I think next stage in AI training is as the authors said, synthetic data. I am not worried about the G.I.G.O. curse, you can do synthetic data generation successfully today with GPT-4. For example in the TinyStories dataset, or the Phi-1 & 1.5 models, or the Orca dataset we have seen big jumps in competency on the small models. Phi punches 5x above its weight class.

So how can you generate data at level N+1 when you have a model at level N?

You amplify the model - give it more tokens (CoT), more rounds of LLM interaction, tools like code executor and search engine, you use retrieval to bring in more useful context, or in some cases you can validate by code execution.

But there is a more general framework - by embedding LLMs in larger systems, they act as sources of feedback to the model. From the easiest - a chat interface, where the "external system" is a human, to robotics and AI agents that interact with anything, or simulations. We need to connect AI to feedback sources so it can learn directly, not filtered through human authored language.

From this perspective it is apparent that AI can assimilate much more feedback signal than humans. The road ahead for AI is looking amazing now. What we are seeing is language evolving a secondary system of self replication besides humans - LLMs. Language evolves faster than biology, like the rising tide, lifting both humans and AI.