Hacker News Reader

Comment by lifeisloving | Hacker News Reader

I dislike how we use the word think or behavior when describing llms. Its just giant neural networks producing logits. There is no thinking. Its a function that takes ~500k words, and returns some words based on those probabilities. I'd argue the GPU is doing more classical "thinking" than the model, but its not doing that either.

Humans think, computers do not.

Humans are not computers/programs do not reply to me with that nonsense turbo nerds.

View on HN

Whenever I see LLM "thinking" or "reasoning" I replace it in my mind with "generating hopefully-related tokens to hopefully steer the conversation to a more related area in vector space to hopefully generate a more useful next token".

But I understand that isn't very catchy.

Very funny, or we could just normalize saying "generating" or "producing" so my normie friends dont think Claude is alive, because they are starting to think that.

Neither of those words are very specific, and everyone in the LLM space knows that "reasoning" tokens are just tokens that don't appear in the answer shown to the user.

The issue isn’t with people in the LLM space knowing what is happening. It’s with normal or less-technical people believing that the LLM is actually thinking. We can understand the difference, but that doesn’t mean we need to encourage the misconception.

Non-technical people believing that these models are actually thinking can lead to all sorts of inappropriate uses.

> everyone in the LLM space

The problem is that the language spills outside of the LLM space, so we should use terminology which is more generally understood in a way which reflects reality.

But what terminology is best than "reasoning" tokens? I think it's perfect because:

- It conveys the idea that they aren't tokens normally visible or part of the final answer.

- It conveys the idea that the longer the models "reasons", the better the output.

What other term best describes both of these ideas? Why should we use a less useful terms just because it can confuse some people? I mean, some in 1952 feared that calling computer memory "memory" was already too much anthropomorphization[1]!

[1] https://3e.org/private/ai_words.pdf

Look in a human head and you don’t see thinking either. It’s not a technical term.

Read Thinking, Fast and Slow. Both system 1 and system 2 thinking has been achieved with modern AI.

Your argument that it is not thinking because it is not biological / like a human is like trying to argue a plane doesn't fly because it uses jet fuel unlike a bird.

I agree with you but our current understanding of the biological brain is pretty much an organic neural network. What's the thing that actually differentiates the two and makes a neural network not thought but our brain is? I don't know.

i can give you a random number between 1 and 10, mean while all the LLM's will say 7 every time... because they are language models that look like they can reason and "think"

There's no "every time" for a LLM. It's weights are frozen. You're asking it for the first time, again and again.

A human will also say 7 disproportionately often. Clearly, our output is statistically constrained and we don't actually think, right?

... Besides, how do you even make the jump from "LLM picks 7 when asked for a number between 1-10" to "llms don't think"?

Just... Nothing actually makes sense in what you're saying. You didn't define "think", you didn't actually make an argument, you didn't present evidence, there's just nothing but empty words.

>our current understanding of the biological brain is pretty much an organic neural network

This is just plainly wrong.

So what is correct?

Hormones, neurotransmitters, genetic expression, and a mountain of biology.

Thinking sure, but behavior? People talk about the behavior of functions all the time. Like, ones written with normal, traditional programming languages. So why not models? They're just annoyingly complex functions.

Computer science, in general, has always had a habit of not being precise with language. That's only getting worse with AI.

It's pretty wild to see people uncritically accept the marketing speak like this, yeah. LLMs cannot reason, period. The supposed "reasoning" they do is just another statistical correlation of tokens, not actual reasoning. But apparently all you have to do is claim that it's thinking and people will buy the lie hook, line and sinker.

This is not a marketing problem, many ML scientists have also fallen into this trap.

> LLMs cannot reason, period.

You cannot be serious.

... Can you quote any existing definition of "reasoning" that LLMs don't fit?

Very excited to hear that you've solved the fields of neuroscience and phenomenology!

You seem to have stumbled upon the hard problem of consciousness.

Saying the GPU is doing thinking but the model is not is a WILD take.

What is your definition of thinking?

Reply on news.ycombinator.com