Very useful info. Didn't realize model releases often reused the same pretrained base-model and just enhanced the post-training RL. And also that contractors are creating tasks and solutions for this post-training. Super interesting and makes a lot of sense.

... And yeah, agree with your statement that this is not emergence. But to me, it's still impressive, because the more post-training tasks they create, the more the model can do, and this set of abilities is only going to keep growing and growing.

... Also, if you wouldn't mind one last question, have read that the AI companies have started to do self-training, where the models are starting to train themselves. Might this lead to even faster and faster increase in model capabilities? ... although, based on what you're saying, maybe it just leads to a model reaching a its max plateau for existing capabilities rather than an explosion in new ones. Because, as you mentioned, for a model to learn an ability, it still needs a human to create a training dataset for it. Is this your understanding?