Hacker News Reader

Comment by nullstyle | Hacker News Reader

> Why would we expect LLMs to be capable of thought in the first place?

"We" didn't have the expectation, but upon interacting with LLMs, we can see and feel and hear that thought is taking place, and have realized that maybe the special place we held for human thought isn't so special as we had hoped.

> The fact that these models have devoured the contents of the entire internet and still aren’t AGI is an indication that LLMs won’t develop a capacity for cognition.

Why do you assume that training on the internet would lead to AGI?

View on HN

People would’ve said the same thing about ELIZA. Turns out, people are inclined to anthropomorphize non-conscious objects or systems. This is especially true for chatbots, which are able to mimic language.

Again, why should we believe that LLMs are capable of thought, other than the fact that they vaguely “seem” to communicate with conscious intent?

> People would’ve said the same thing about ELIZA.

I believe this to be the case because the appearance of thoughtfulness is a smooth function, not a line in the sand. Why are you bringing consciousness into this discussion? Consciousness isn't thoughtfulness.

> Again, why should we believe that LLMs are capable of thought, other than the fact that they vaguely “seem” to communicate with conscious intent?

I define "to think" as the act of applying context to a model. What do you define it as? Given my definition, llm inference seems plainly to be an act of thought.

> Why do you assume that training on the internet would lead to AGI?

Answer my question, please. edit: let me rephrase to be more clear with what i'm asking. Why are you assuming that consuming the entire internet would lead to either "AGI is here" or "AGI will never be here"? New techniques for developing better intelligence are emerging all the time.

...because that's the story being told?[0]

[0]: https://openai.com/index/planning-for-agi-and-beyond/

I think if you read the article linked and thought: “I can assume that AGI will be birthed by training on the entire Internet” then you didn’t read very carefully or judge the author well.

I read the linked article and thought OpenAI wants to communicate: "We plan our decisions to build AGI in the long term". You know, decisions like the partnership with reddit[0] a year after that announcement.

[0]: https://redditinc.com/blog/reddit-and-oai-partner

> upon interacting with LLMs, we can see and feel and hear that thought is taking place

uh.. no we can't?

Then you aren't paying attention, IMO.

An example for the dense: A friend of mine used his conversations with ChatGPT to get through a tough spot in his marriage. The distinction between what happened in ChatGPT's internal processing and what would have happened in the internal processing of the brain of a family therapist is a meaningless distinction.

My friend was able to benefit from the thoughtwork of ChatGPT.

Calling people dense isn't helping them engage with you

I was responding reciprocally to your attitude. Care to differentiate thought and thoughtwork for us?

Screenshot for you: https://gist.github.com/simonw/5b051aa10e97ac86eeb539f51a828...

That looks like thinking. If you want to argue it's not thinking you are absolutely welcome to, but you need to say more than just "no we can't?".

The sharpest cries of, "that's not thinking!" always seem to have an air of desperation about them. It's as if I'm special, and I think, and maybe thinking is what makes me special, so if LLMs think then I'm less special.

At some point the cry will change to, "It only looks like thinking from the outside!" And some time after that: "It only looks conscious from the outside..."

This paper makes the claim that the “reasoning” output these models produce isn’t a necessary component of these models, and is likely unrelated to the actual final output:

https://arxiv.org/abs/2505.13775

Just because these models are good at outputting text that appears to be a series of thoughts, doesn’t mean that LLMs are thinking.

I'm finding quite often from reading these streams in my own chats there's a lack of strong logical consistency from one chunk to the next where it should clearly be there. That combined with the output not being consistent with the 'thoughts'. I'll screenshot it next time I see a good example

'thinking' here needs a clearer definition. Even for us humans the language is just an after the fact attempt at describing what went on, or is going on inside our heads

Consider: https://www.unsw.edu.au/newsroom/news/2019/03/our-brains-rev...

I’ve got one of those old 1980s chess computers with Kasparov’s face on it.

Before it makes a move, a little LED labeled ‘thinking’ blinks for a while.

Looks like thinking, too.

Reply on news.ycombinator.com