> Pre-training is crucial for this mechanism: models trained from scratch to add numbers only exploit low-frequency features, leading to lower accuracy.
what's the convention on the meaning of "pre-training" vs "training from scratch" ?
Is this a nomenclature shift?