Hacker News Reader

Comment by PaulRobinson | Hacker News Reader

> as soon as you want them to do something original, they just can't.

An LLM is, famously, a "stochastic parrot". It has no reasoning. It has no creativity. It has no basis or mechanism for true originality: it is regurgitating what it has seen before, based on probabilities, nothing more. It looks more impressive, but that's all is behind the curtain.

It surprises me that so many people expect an LLM to do something "clever" or "original". They're compression machines for knowledge, a tool for summarisation, rephrasing, not for creation.

Why is this not that widely known and accepted across the industry yet, and why do so many people have such high expectations?

View on HN

Because it isn't true according to Hinton, Sutskever, and other leading AI researchers.

Most foxes are herbivores according to the fellow guarding my henhouse.

They do make compelling arguments if you listened to them. Also, from my coding with llms using cursor they obviously understand the code and my request more often than not. Mechanistic interpretability research shows evidence of concepts being represented within the layers. Golden gate claude is evidence of this: https://www.anthropic.com/news/golden-gate-claude

To me this proves that llms learn concepts and multilayeres representations within its network and not just some dumb statistical inference. Even famous llm skeptic like Francois Chollet doesn't invoke this stochastic parrots anymore and has moved on to arguing that they don't generalize well and are just memorizing.

With GPT-2 and 3 I was of the same opinion that they seemed like just sophisticated stochastic parrots, but current llms are a different class from early gpts. Now that o3 has beaten memory resisting benchmark like ARC-AGI I think we can confidently move on from this stochastic parrots notion.

(And before you argue that o3 is not an llm, here is an openai researcher stating that it is an llm https://x.com/__nmca__/status/1870170101091008860?s=46&t=eTe... )

Neural networks are generalizing things as part of their optimization scheme. The current approach is just to dump many layered neural networks (at the core) as in "deep learning" to solve the problems, but the networks are too regular, too "primitive" of sorts. What is needed are network topologies that strongly support this generalization, the creation of "meta abstraction levels", otherwise it will get nowhere.

Biological networks of more intelligent species contain a few billion neurons and upwards from that while even the big LLMs are somewhere in the millions of "equivalent" at best. So, bad topology + much less "neurons" and the resulting capabilities shouldn't be too surprising. Plus it is clear that AGI is nowhere close, because one result of AGI is a proper understanding of "I". Crows have an understanding of "I", for example.

And that is where these "meta abstraction levels" come in: There are many needed to eventually reach the stage of "I". This can also be used to test how well neural networks perform, how far they can abstract things for real, how many levels of generalization are handled by it. But therein lies a problem: Let 2 persons specify abstraction levels and the results will be all across the board. This is also why ARC-AGI, while dives into that, cannot really solve the problem of testing "AI", let alone "AGI": We as humans are currently unable to properly test intelligence in any meaningful way. All the tests we have are mere glimpses into it and often (complex) multivariable + multi(abstraction)layered tests and dealing with the results, consequently, a total mess, even if we throw big formulas at it.

I don't care much for the concept of AGI since it's ill defined.

However, out of distribution generation and generalization are a better more useful metric. In these Yann Lecun has argued that interpolation is meaningless in the high dimensional spaces these "curves" are embedded in.(https://arxiv.org/abs/2110.09485)

ARC has proven they can generalize, and Alpha Go (not an llm but a deep network) has proven it can generate novel/creative solutions. We don't need AI to have a sense of self "I" for them to beat us at every human skill and activity. Infact it might be detrimental and not useful for us if AI developed a sense of self.

Reply on news.ycombinator.com