This blog post gives more details on the science of training large autoregressive language models that have recently been successful worldwide (ChatGPT, GPT-4). Some thoughts on the similarities between the biological brain and artificial neural networks are also presented.