Couple other interesting papers from 2023 to add, specifically for small language models.
TinyStories: https://arxiv.org/abs/2305.07759
Phi-1.5: https://arxiv.org/abs/2309.05463
Both these papers proved that with good/concise data we can get better performance with smaller parameter counts.