> A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that are difficult to stumble on by accident. It is hard to say precisely what would count as such a proof, but I think we’ll recognise it when we see it

Agreed. I find that after seeing these results from OpenAI we undeniably have a machine that has:

* General knowledge of nearly every subject humanity has ever learned

* The ability to simulate reasoning (albeit sometimes not very well) with that knowledge

* The ability to reference across the domains of knowledge

To me, this is more or less what I would think "Artificial General Intelligence" is. It's the cumulative knowledge of all general human intelligence, baked into an artificial form, which can then use that knowledge to achieve novel goals.

In many cases of mathematical breakthroughs there is an insight that comes from just happening to know a combination of already existing ideas and then combining them to solve that problem. This is where having that general knowledge seems particularly strong because we can run these machines for weeks on end effectively trying to brute force.

That being said, I could never imagine an LLM in its current form inventing something as elegant as the Fourier transform.