> We first demonstrate that conventional training data is not the most effective for arithmetic learning, and simple formatting changes can significantly improve accuracy. This leads to sharp phase transitions as a function of training data scale, which, in some cases, can be explained through connections to low-rank matrix completion. Building on prior work, we then train on chain-of-thought style data that includes intermediate step results. Even in the complete absence of pretraining, this approach significantly and simultaneously improves accuracy, sample complexity, and convergence speed. We also study the interplay between arithmetic and text data during training and examine the effects of few-shot prompting, pretraining, and model scale. Additionally, we discuss length generalization challenges.
A tangent is that positional systems were originally invented with least digit first, I believe.
The Babylonian sexagesimal system was like that as was the Arabic one (where first is on the right).
The most significant digit first convention came when right-to left numbers were used in left-to-right systems without reversing them in writing. To this day we read the more common smaller numbers least significant digit first to varying degrees.
16 = six teen, sech zehn
98 = acht und neunzig, achten negentig, ثمانية وتسعون
17 + 14 = 20 + 11 = 30 + 1 = 31
vs 17 + 14 = 10 + 10 + 10 + 1 = 31