Comment by ben_w | Hacker News Reader

Comment by ben_w | Hacker News Reader

Parent Full thread

> If you feed it enough flat-earther content, as well a physics books, an LLM will happily tells you that the earth is flat, and explain you with lots of physics why it cannot be flat.

This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.

And not just that, common misconceptions abound everywhere and not just conspiracy theories, religion, and politics. My dad absolutely insisted that the water draining in toilets or sinks are meaningfully influenced by the Coriolis effect, used an example of one time he went to the equator and saw a demonstration of this on both sides of the equator. University education and lifetime career in STEM, should have been able to figure out from first principles why the Coriolis effect is exactly zero on the equator itself, didn't.

> A human will learn one or the other first, and once the initial learning is made, it will disregards all the evidence of the contrary, until maybe at some point it doesn't and switches side entirely.

We don't have any way to know what a human would do if they could read the entire internet, because we don't live long enough to try.

The only bet I'd make is that we'd be more competent than any AI doing the same, because we learn faster from fewer examples, but that's about it.

> LLMs don't have an inner representation of the world and as such they don't have an opinion about the world.

There is evidence that they do have some inner representation of the world, e.g.:

• https://arxiv.org/abs/2506.02996

• https://arxiv.org/abs/2404.18202

> This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.

You completely misread my point.

The key thing with humans isn't that they cannot believe in bullshit. They can definitely do. But we don't usually believe in both the bullshit and in the fact the BS is actually BS. We have opinions on the BS. And we, as a species, routinely die or kill for these opinions, by the way. LLM don't care about anything.

> But we don't usually believe in both the bullshit and in the fact the the BS is actually BS.

I can't parse what you mean by this.

> LLM don't care about anything.

"Care" is ill-defined. LLMs are functions that have local optima (the outputs); those functions are trained to approximate other functions (e.g. RLHF) that optimise other things that can be described with functions (what humans care about). It's a game of telephone, like how Leonard Nimoy was approximating what the script writers were imagining Spock to be like when given the goal of "logical and unemotional alien" (ditto Brent Spiner, Data, "logical and unemotional android"), and yet humans are bad at writing such characters: https://tvtropes.org/pmwiki/pmwiki.php/Main/StrawVulcan

But rather more importantly in this discussion, I don't know what you care about when you're criticising AI for not caring, especially in this context. How, *mechanistically*, does "caring" matter to "learning abstract thought", and the question of how closely LLMs do or don't manage it relative to humans?

I mean, in a sense, I could see why someone might argue the exact opposite, that LLMs (as opposed to VLMs or anything embodied in a robot, or even pure-text agents trained on how tools act in response to the tokens emitted) *only* have abstract "thought", in so far as it's all book-learned knowledge.

If you cant parse the sentence about BS i dont even know how they should respond to you. Its a very simple sentence.

>> But we don't usually believe in both the bullshit and in the fact the the BS is actually BS.

> I can't parse what you mean by this.

The point is that humans care about the state of a distributed shared world model and use language to perform partial updates to it according to their preferences about that state.

Humans who prefer one state (the earth is flat) do not -- as a rule -- use language to undermine it. Flat earthers don't tell you all the reasons the earth cannot be flat.

But even further than this, humans also have complex meta-preferences of the state, and their use of language reflects those too. Your example is relevant here:

> My dad absolutely insisted that the water draining in toilets or sinks are meaningfully influenced by the Coriolis effect [...]

> [...] should have been able to figure out from first principles why the Coriolis effect is exactly zero on the equator itself, didn't.

This is an exemplar of human behavior. Humans act like this. LLMs don't. If your dad did figure out from first principles and expressed it and continued insisting the position, I would suspect them of being an LLM, because that's how LLMs 'communicate'.

Now that the what is clear -- why? Humans experience social missteps like that as part of the loss surface. Being caught in a lie sucks, so people learn to not lie or be better at it. That and a million other tiny aspects of how humans use language in an overarching social context.

The loss surface that LLMs see doesn't have that feedback except in the long tail of doing Regularized General Document Corpora prediction perfectly. But it's so far away compared to just training on the social signal, where honesty is immediately available as a solution and is established very early in training instead of at the limit of low loss.

How humans learn (embedded in a social context from day one) is very effective at teaching foundational abilities fast. Natural selection cooked hard. LLM training recipes do not compare, they're just worse in so many different ways.

Reply on news.ycombinator.com