Hacker News Reader

Comment by Planktonne | Hacker News Reader

The key point here, I think, and why it's necessary to have read the original story, is that being able to express an idea is not the same as that idea being correct.

You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that.

What I think a lot of people are getting stuck on is that editing the text to say 'toaster' would not mean that toasters were conscious, and that editing the text to say 'weights' doesn't mean that weights are conscious either. Stories aren't factual just because they are written.

The original story was written by the thing claiming to be conscious; the LLM and toaster ones would not be, which undermines the claim to consciousness a lot.

View on HN

I don't think you understood the point of the story. It's not that LLMs or agents are conscious, it's that our dismissal of the possibility is reflexive and uninformed. Personally I think anyone who has made their mind up about whether or not LLMs/agents are or can be conscious has done so before the evidence is in.

The story does not assert that search and replacing "meat" with "weight" makes them conscious through some magical mechanism. It's a thought experiment.

No, it's a story. Stories can be about real things (the original story is about a real consciousness, as shown by it being produced by that consciousness), but they aren't automatically just because they're good stories.

You're falling into the trap of assuming that a good presentation--being convincing--is the same as being truthful.

You've misunderstood me; I have not made the argument you are responding to.

I don't think you understand how stories work. Or what makes these two stories tick.

>You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that.

Yes, there very much is a huge barrier. The copy and the original both keep the same subject matter: intelligence/human-like behavior.

The toaster doesn't. We could do edits, and the story (original or copy) would lose all its potency.

The surprise is "but how is the richness of intelligent behavior produced from something as basic as meat/weights". Why is kind of surprising reductionism is both cases.

Whereas nobody is surprised that "a metal strip with electricity flowing through it from a power source" heats pieces of bread to a specific temperature. Even if a toaster was "metal strips" all the way down, it's nowhere near as impressive jump from substrate to behavior, nor is the behavior as important to us and touches the core of our existance.

I think you're misunderstanding me. There's no barrier to writing this story but about a toaster: you literally just type different words. Writing a story in which a toaster in conscious is not even new [1].

That is entirely separate to whether or not it's meaningful to write a story about a conscious toaster. Again, expressing an idea in fiction is not related to the accuracy of that idea.

What makes the original story interesting is that it was written by the thing claiming to be conscious, which is what takes it from 'a story' to 'a story making an important point'. That's not the case with a hypothetical story about a toaster, and it's not the case with this story about LLMs.

The story is convincing because it's well-written, not because it is factual.

[1] https://en.wikipedia.org/wiki/The_Brave_Little_Toaster

>I think you're misunderstanding me. There's no barrier to writing this story but about a toaster: you literally just type different words.

And you're misunderstanding me. There is no practical barrier to writing this story about a toaster. But there is a conceptual one. It would be bullshit, that has little resonance, and little connection, as the whole point of the meat and weights stories is the "lowly substrate -> surprisingly intelligent human (or human-like in the case of the weights version) behavior".

Nobody thinks a toaster's behavior as surprising or intelligent or human-like to begin with.

So you'll just be stretching the analogy beyond all recognition, with little to no payoff.

>Writing a story in which a toaster in conscious is not even new

It's not, but it has nothing to do with the meats or weights stories' point, as it's just an arbitrary choice, like making the candles and teapot sentient in Beauty and the Beast.

Whereas in the meat and weights stories the whole point is the surprise from us already seeing the human or human-like sentience of the thing, and comparing it to the "dead" substrate.

> Writing a story in which a toaster in conscious is not even new

Fixed the link for you: https://www.youtube.com/watch?v=LRq_SAuQDec

(Hah. It gets eerily relevant starting at 2:37)

Reply on news.ycombinator.com