It may also be that we're looking at this the wrong way altogether. If you compare the natural world with what humans have achieved, for instance, both things are qualitatively different, they have basically nothing to do with each other. Humanity isn't "adding nines" to what Nature was doing, we're just doing our own thing. Likewise, whatever "nines" AGI may be singularly good at adding may be in directions that are orthogonal to everything we've been doing.
Progress doesn't really go forward. It goes sideways.
AGI to encompass all of humanity's knowledge in one source and beat every human on every front might be a decade away.
Individual agents with increased agency adequately covering more and more abilities consistently? Seems like a steady path that can be seen into the horizon to put one foot in front of the other.
For me, the grain of salt I'd take Karpathy with is much, much, smaller than average, only because he tries to share how he thinks and examines his own understanding and changes it.
His ability to explain complex things simply is something that for me helps me learn and understand things quicker and see if I arrive at something similar or different, and not immediately assume anything is wrong, or right without my understanding being present.
Anything that exists is part of nature, there can be no exceptions.
If I go burn a forest down on purpose, that is in fact nature doing it. No different than if a dolphin kills another animal for fun or a chimp kills another chimp over a bit of territory. Insects are also every bit as 'vicious' in their conquests.
Even if you don't like that definition, you still have the question of how many nines we are away from having an AI that can contribute to its own development.
I don't think you know the answer to that. And therefore I think your "fast acceleration within two years" is unsupported, just wishful thinking. If you've got actual evidence, I would like to hear it.
Machine learning has been helping with the development of machine learning ever since hyper-parameter optimisers became a thing.
Transformers have been helping with the development of transformer models… I don't know exactly, but it was before ChatGPT came out.
None of the initials in AGI are booleans.
But I do agree that:
> "fast acceleration within two years" is unsupported, just wishful thinking
Nobody has any strong evidence of how close "it" is, or even a really good shared model of what "it" even is.
There's a massive planet-sized CITATION NEEDED here, otherwise that's weapons grade copium.
If you look at it differently, assembly language may have been one nine, compilers may have been the next nine, successive generations of language until ${your favorite language} one more nine, and yet, they didn't get us noticeably closer to AGI.