Hacker News Reader

Comment by Salgat | Hacker News Reader

To me LLMs seem to most closely resemble the regions of the brain used for converting speech to abstract thought and vice-versa, because LLMs are very good at generating natural language and knowing the flow of speech. An LLM is similar to if you took the the Wernicke's and Broca's Areas and stuck a regression between them. The problem is that the regression in the middle is just a brute force of the entire world's knowledge instead of a real thought.

View on HN

I think the major lessons from the success of LLMs are two: 1) the astonishing power of a largely trivial association engine based only on the semantic categories inferred by word2vec, and 2) that so much of the communication abilities of the human mind require so little rational thought (since LLMs demonstrate essentially none of the skills in Kahneman's and Tversky's System 2 thinking (logic, circumspection, self-correction, reflection, etc).

I guess this also disproves Minsky's 'Society of Mind' conjecture - a large part of human cognition (System 1) does not require the complex interaction of heterogeneous mental components.

> that so much of the communication abilities of the human mind require so little rational thought

Beyond that fact that human minds created them, I doubt that LLMs can tell us anything about the abilities of the human mind or what language in humans requires.

The most we can learn from LLMs about ourselves will be in how we react to them, or more broadly, what the datasets LLMs use show about who we are.

What makes this tough is that LLMs can show logical thinking and self-correction when specifically prompted (e.g. "think step by step", "double-check and then correct your work"). It seems unlikely that they can truthfully self-reflect, but I don't think it's strictly impossible.

> LLMs can show logical thinking and self-correction

The same way they "show" sadness or contrition or excitement?

We need to be careful with our phrasing here: LLMs can be prompted to provide you associated phrases that usually seem to fit with the rest of the word-soup, but whether the model is actually demonstrating "logical thinking" or "self-correction" is a Chinese Room problem [0]. (Or else a "No, it doesn't, I can tell because I checked the code.")

[0] https://en.wikipedia.org/wiki/Chinese_room

Reply on news.ycombinator.com