I used to think that to achieve human-level intelligence with AI, we need to have human-sized neural network. It was a conservative guess, but seeing where things are going, I now strongly believe we need a lot LESS parameters with AI, because AI is trained from literally nothing (from noise) on our current culture, from the Internet. It has no baggage, it's built from the ground up to be a modern 21st century human. We're not, we train decades to become ones. And struggle with it. Our genes are mostly tuned for our environment from thousands of years ago.

A human brain carries a lot more baggage with it. Long story short, our biological brain hardware is much superior to current AI hardware, but our biological brain software (where we learn the culture that surrounds us) is much inferior to current AI machine learning. GPT-3 trained 175B on basically the whole internet in 15 days. That's insane. And now there are rumors that ChatGPT is executed on an optimized subset of this 175B model, which may be a bit more than 1/10 that size.

I think going forward we'll see 20-30B parameter models perform much better than an average human in typical tasks. Especially after we evolve the topology of an AI to not be a straight directed acyclic graph, as it is now, but allow structures to loop internally, to invoke other structures with input (as subroutines) and so on.

And also there's new hybrid analog/digital hardware in the pipeline, which makes AI execution 100 times more energy efficient and compact. So above-human level AI on your phone is on the horizon, I'd say.

We're only getting started.