I think I am coming to agree with the opinions of the author, at least as far as LLMs not being the key to AGI on their own. The sheer impressiveness of what they can do, and the fact they can do it with natural language, made it feel like we were incredibly close for a time. As we adjust to the technology, it starts to feel like we're further away again.

But I still see all the same debates around AGI - how do we define it? what components would it require? could we get there by scaling or do we have to do more? and so on.

I don't see anyone addressing the most truly fundamental question: Why would we want AGI? What need can it fulfill that humans, as generally intelligent creatures, do not already fulfill? And is that moral, or not? Is creating something like this moral?

We are so far down the "asking if we could but not if we should" railroad that it's dazzling to me, and I think we ought to pull back.