Hacker News Reader

Comment by menacingly | Hacker News Reader

I’m not an AI zealot, but this is such a strange debate to me.

“AI only _appears_ to be reasoning” is silly, because appearing to reason is our only sane definition of reasoning.

If we can’t say exactly when our random statistical noise emerges into some vague concept of intelligence, why does our standard suddenly become rock solid when applied to other intelligences?

View on HN

My take is when the model can't do the things that you would expect it to do if it were reasoning, and instead does the things that you would expect it to do if it were an approximate retrieval system with an enormous knowledge base.

this is a good paper: https://arxiv.org/abs/2309.12288

The "reversal curse" has nothing todo with inference or the reasoning abilities of a pre-trained model. GPT-4 or whatever has no problem telling you the right deduction with information presented in the context. This is a memory retrieval from training issue and is completely unsurprising.

I'd say the main issue is why does it take ingesting the entire collective knowledge of humanity for LLMs to work if they are actually reasoning and understand things from first principles? Humans don't require that volume of training data to learn how to write or do basic math

Well, I think you underestimate the amount of "training data" that human gets in the early stages of their life. Just learning to speak takes a few years, during which all people around you talk to you for most of your waking hours. Learning to write and doing basic math typically does not start until age of 6.

No chance it comes close to the content of the internet.

You (humanity) don't learn from first principles either.

The 3-year-old toddler doesn't learn the rules of English grammar before they learn to speak.

Sure, we don't know how humans manage to learn with so "little" data yet, but it's definitely not "understand things from first principles". In fact, if you randomly pick an average person from the streets, they wouldn't be able to explain most things from first principles. (eg. How does their car work?)

Reasoning is a process of apprehending facts about objects and inferring how they will relate with other facts. It has nothing to with statistical distributions of words, except in the special case where that distribution has been constructed by humans in a way that reflects their knowledge and inference of those facts. If you feed an LLM garbage, it will happily give you garbage. It doesn't know what garbage is and it doesn't even know that it doesn't know.

Reply on news.ycombinator.com