Hacker News Reader

Comment by stackghost | Hacker News Reader

LLMs are not conscious and cannot have "takes". They stochastically produce a simulacrum of a human's takes based on their training, which is not the same thing.

View on HN

But then if you observe human behaviour at a macro level don't we do the same?

I think we like to pretend that we don't. But even if you think about it from a bio/physics point of view, all we are is just a bunch of biological wires and neurotransmitter packages.

I know there's the whole unproven quantum tubule nonsense but even that would be a physics effect that we could learn about, manipulate and extort just as we have with everything else we've learnt about our realities.

That suggests they can't make inferences or produce any original insights, which is wrong. They can absolutely have takes.

Even if it's a simulacrum, it's a simulacrum based on nearly everything that was ever written. No one who has ever lived can claim to know what these models have trained on.

Trained on a very small portion of what has been written, heavily biased by what is available to American companies to acquire. You could argue that LLMs are somewhat representative of their training data (with some consideration to the way that they are trained) but to say it’s anything like a representation of what humans think is nonsense.

It is a simulacrum, the end.

There's nothing small about it, and I didn't say it's a representation of what humans think. I said it's the models take on humanity based on having been trained on a far larger amount of text than any human has ever read

The problem with asking these models stuff like "what do you think about humanity" is that, even if it can synthesize an insight about humanity, it's just as likely it's regurgitating some opinion it's already seen. Which, given that this "controversial idea" is hardly new, is likely what it's doing. It's like a maximally boring controversial opinion, wake me up when it starts endorsing the bicameral mind or something actually weird.

Maybe it can produce interesting insights. But it's like asking a pundit for a soundbite: what are the odds they aren't just shuffling together something that sounds appropriate to the question?

> That suggests they can't make inferences or produce any original insights

No it doesn't suggest that. A random number generator can also produce original insight (with different efficiency and effectiveness)

> They can absolutely have takes.

Sure, if you're willing to redefine "take"

I'd suggest that a random number generator is a great place to go if you want to get an original number.

And an LLM is a great place to go if you want to get an original token?

Aren’t human takes also a simulacrum of other human takes based on training?

How do you know they are not conscious?

How you know God doesn't exist ?

You can't prove negative; but it doesn't satisfy any conditions for a positive

Oh you have a convenient list that separates the conditions for consciousness from the ones that just happen to be incidentally true of the substrate? Can I see it?

Reply on news.ycombinator.com