It seems to me like the moon is "chatbots which are somewhat convincing" and everybody is landing there in OpenAI's wake. The real problem is Mars - make a computer which can learns as quickly and reason as deeply as, say, a stingray or another somewhat intelligent fish[1].

[1] This task seems far beyond the capability of any transformer ANN absent extensive task-specific training, and it cannot be reasonably explained by stingray instinct: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8971382/

I don't think a pure language model of the sort under consideration here is heading towards AGI. I use language models extensively and the more I use them the more I tend to see them as information retrieval systems whose surprising utility derives from a combination of a lot of data and the ability to produce language. Sometimes patterns in language are sufficient to do some rudimentary reasoning but even GPT4, if pushed beyond simple patternish reasoning and its training data, reveals very quickly that it doesn't really understand anything.

I admit, its hard to use these tools every day and continue to be skeptical about AGI being around the corner. But I feel fairly confident that pure language models like this will not get there.