Hacker News Reader

Comment by TeMPOraL | Hacker News Reader

And LLMs are the opposite of dictionary, actually. They suck at storing facts. They excel at extracting patterns from noise and learning them. It's not obvious to me that this isn't intelligence; on the contrary, I feel it's very much a core component of it.

View on HN

Isn't that a dictionary of patterns?

> They excel at extracting patterns from noise and learning them.

You can argue those are facts too.

Yes, I have noticed that a lot of extreme AI cynics have been arguing that any and every example of reasoning or thinking that an LLM displays is just some variant of memorisation.

The biggest evidence that LLMs can’t reason is hallucinations. If it could reason it would have rejected fictional generated output that make no sense.

Maybe it’s more accurate to say that LLMs lack (self-)awareness. Because when you point out things that make no sense, they do have some limited ability to produce reasoning about that. But I agree that this lack of awareness is a serious and maybe fundamental deficit.

And how often does it get that wrong too?

It’s more likely it’s just, once again, generating the most probable answer - and if you shake the magic 8 ball enough you will get the answer you were expecting.

Yeah, but the thing is, this seems exactly what people are doing too, at the boundary of conscious and unconscious, with the "inner voice" being most directly comparable to LLMs. It too generates language that feels like best completion, regardless of whether the output is logically correct or not.

Wouldn't that be evidence at most that its reasoning was flawed, or could contain errors?

> The biggest evidence that LLMs can’t reason is hallucinations.

If I asked you a question and you had to respond with a stream of consciousness reply, no time to reflect on the question and think about your reply, how inaccurate would your response be? The "hallucinations" aren't a problem with the LLM per se, but how we use them. Papers have shown that feeding the output back into the input, as happens when humans iterate on their own initial thoughts, helps tremendously with accuracy.

In comparing to human minds, LLMs are better understood as the "inner voice" part, not the mind. From that perspective, it's eerie how similar the two are in success and failure modes alike.

Yes, I'm saying here that peoples' inner voices are hallucinating in very similar fashion; "rejecting fictional generated output that makes no sense" is a process that's consciously observable and involves looping the inner voice on itself.

Reply on news.ycombinator.com