maxbond

> I pointed out the parallel in both statements. I can't do more than that.

You mean these statements?

    No one cares about teapots in space either (Russel).

    Teapots are not compelling.

I guess you did but that's pretty much leaving me breadcrumbs and expecting me to make your argument for you. It seemed to me like you were talking about reductio ad absurdum with that argument as an illustrative example. Perhaps you overestimate my cleverness.

> Which party is the burden of proof on? This is confusing since you are saying that the burden of proof is on a position (on them not being conscious).

It's metonymy. "X" stands in for "people who argue for X". May I ask if you were sincerely confused? You told me you aren't being coy but I have a hard time believing that this was so unclear.

> I have never heard about any principle in philosophy or in science that says that, given enough Looks Like A Duck points, it is a duck. Based on subjective experience, even.

If it looks like a duck, if it quacks like a duck, you should adjust your priors to assign a higher likelihood that it is a duck. All measurements contain error; you can't ever observe, "that is a duck," only "that looks like a duck". All knowledge is founded on a sufficiently deep stack of "looking like a duck" that we may assert it with confidence.

To the extent we have objective measures (like conducting a Turing test on blind participants), it can meet them too. You can't say the same of a toaster.

> We obviously can't demand a falsifiable theory here. But we have to do better than arguing from incredulity.

"It is a statistical model, ergo it is not conscious" is also an argument from incredulity. I don't know if that's your view or not but it's the one my remarks have been addressing in general.

I don't think I was quite as patient as I should have been, but I do appreciate it.

> It directly parallels your argument.

Much like Russel argued that the burden of proof of God's existence is on theists, the burden to establish this parallel is on you as the person forwarding the argument. I don't see any relevant connection. Russel isn't arguing that a teapot is as real as God in the same way it's disputed here that a toaster is as conscious as an LLM.

> So far you have said that the argument is compelling. What’s that got to do with reality? A robot cow could be sexually arousing to a real bull.

AI is a real phenomenon that we can study and measure. There is no experiment that anyone has devised can determine whether or not they are conscious, so that is the reality - uncertainty. That doesn't mean they're conscious. It means that the belief they are not conscious is assumption.

You might say the same of a toaster, but these hypothesis are not equally strong. The toaster doesn't exhibit any behaviors to suggest that it's conscious. Consciousness isn't a hypothesis with any explanatory power for the observed behaviors of a toaster. It's not a hypothesis that's on the table. That's why the analogy doesn't work.

To put a fine point on it, it appears on it's face that AIs could be conscious. They can put on a very convincing performance of being a person. A sufficiently convincing performance is indistinguishable from the real thing. So at face value, the burden of proof is on them not being conscious. Reductionist arguments that present the mechanics of how they work and leap to their not being conscious don't work, because there is no law saying a statistical model can't be conscious. That's an assumption, not knowledge.

It's beside the point and so I only note it out of interest, but the Chinese writing system doesn't use an alphabet (or a syllabary like Japanese kana), it's logography.

> Is Russel’s Teapot a bad argument in the God debate?

What's the relevance? If the argument made here are was a good argument, it wouldn't matter if Russell's argument was bad. We could construct a bad argument using reductio ad absurdum right here and now and it wouldn't matter to either argument.

Can you be straight with me? You know the salient difference between asserting the consciousness of a toaster and the consciousness of an AI, right? It isn't a mystery to you why we would find one line of inquiry interesting and the other not so much?

For instance, it's probably a real possibility in your mind that I am not a human and am an AI. But you probably aren't entertaining the hypothesis that I'm a toaster.

Not sure what you want from me, if you want to explain yourself I'll listen, if you don't that's fine too.

I agree that is the mode of argument; reductio ad absurdum is a brittle argument, because it only works if the analogy holds. I argued the analogy doesn't hold.

If you didn't mean to say that people who believe AI is conscious are delusional, then I don't understand how to read your comment. If you're interested in a good faith conversation, I'm very confused why, when I asked if anyone sincerely argued that toasters are conscious, you brought up people who were drunk or delusional.

Positive, yes. I never called you a luddite or a neanderthal or anything of the kind.

It's perfectly fine to believe they are not conscious, I am not convinced they are, but asserting anyone who disagrees with you is delusional is unfortunate.

I never said you couldn't write any arbitrary compelling story about a toaster, I said that this specific hypothetical story, where you rewrite "They're made of meat!" to be about a toaster, would not be compelling.

I am doing my best to communicate with you but to be honest you are not hearing me (across both responses), and I am out of words.

You've misunderstood me; I have not made the argument you are responding to.

Their being statistical models and their being conscious are not contradictory unless proven otherwise. That's not knowledge, it is assumption.

It would appear to me you have no interest in a real, good faith discussion on this topic because you think anyone who disagrees with you is necessarily delusional. Which is a shame, and that's the kind of dogma you are criticizing.

This was exactly the point of the story, it's too uncomfortable to admit that we don't know what consciousness is and what is and isn't conscious, so we just brush it under the rug.

Who have you ever heard make a sincere, good faith argument that a toaster is conscious? Who do you imagine would argue against you if you asserted they were mere automata?

If you can't identify anyone, then this analogy doesn't work.

I don't think you understood the point of the story. It's not that LLMs or agents are conscious, it's that our dismissal of the possibility is reflexive and uninformed. Personally I think anyone who has made their mind up about whether or not LLMs/agents are or can be conscious has done so before the evidence is in.

The story does not assert that search and replacing "meat" with "weight" makes them conscious through some magical mechanism. It's a thought experiment.

You can't do the same with a toaster. Physically you could write that story. But it would fall flat because the toaster is not a compelling subject in a discussion of consciousness. You don't have to believe that LLMs or AI agents are conscious to acknowledge that the argument for their consciousness is far more compelling than any other technological artifact.

The story is not about how they function, it's about how we relate to them.

Completely unevidenced conspiracy theories can only harm the discussion. The only possible benefit is to disconfirm conspiracy theories and discourage paranoid thinking. The odds that Standford as an institution are astroturfing on HN round down to 0.

What they're almost certainly observing is that these critical comments are being flagged as inappropriate. People make inappropriate comments that happen to contain criticism all the time, and I frequently see people edit them to declare that they were flagged because the group they're criticizing is astroturfing. It's virtually never the case. I've never seen it happen.

But to be clear I am completely ambivalent on Stanford and if you want to criticize them, more power to you.

Yes, mea culpa. Occasionally I break that rule on my own judgement. Feel free to flag my comment. (I think it's important to disconfirm conspiracy theories.)

Just so you know, I have nothing to do with Stanford, but I am flagging this as conspiratorial nonsense. So when you comment is flagged, I just want you to know that it doesn't confirm your belief, it's just that this comment harms discussion and so must be removed.

Nondeterminism is a red herring. There is a bright line between instructions and data right now, in virtually every transcript format. That we have not succeeded in training an LLM to respect it to a very high degree doesn't imply it is impossible; that they are nondeterministic doesn't imply it is impossible; only that we won't succeed 100% of the time.

A cosmic ray (or rowhammer attack) could flip an X bit too, there isn't anything truly deterministic under the sun.

> ALL context/prompt is instructions, there is no data. It is just unsolvable, period.

That really isn't true. There's no law of physics preventing you from having separate data and instruction inputs to models. The model's transcript format generally distinguishes between prompts and instructions and tool output and such. This isn't a solved problem, and it's possible it's entire unsolvable, but it probably is possible (in general, not with current models) to reject prompt injection to several nines.

This is a lot like making the same statement about CPUs, "the von Neumann architecture doesn't distinguish between code and data so it's impossible to reject malicious instructions." There's actually a lot you can do to reject malicious instructions, you can prevent execution in certain pages, you can prevent certain privileged instructions from being executed in certain pages, you can employ stack cookies, et cetera. Do they prevent all exploitation in all circumstances? No. But each component does function in it's lane and it is possible to create programs with high (though not absolute) guarantees against unauthorized code execution by composing them.

Similarly, you could prevent certain tokens from appearing in the prompt portions of a transcript, you can have a model with multiple input heads only one of which is trusted, etc. I'm not saying those techniques will necessarily work, but it is more complex than "models can only possibly take a single and undifferentiated input stream".

I don't see where they said that; could you give me a quote? It does not seem to me that they are addressing humans at all, except as a foil to LLMs.

I read the misanthropy as ironic. They're applying the same reductionist logic to humans, not because they are misanthropic, but to illustrate that it doesn't help us understand the case we can all agree on. "Humans aren't sentient either" is definitely not the takeaway.

That is not knowledge, that is assumption.

Let's assume we have infinite memory with constant time lookups. With a sufficiently large lookup table, you could exactly replicate the behavior of any person. You could encode it as a next-token predictor: you have precomputed every possible prefix and assigned it a next token. This is a Chinese room, but it is completely indistinguishable from an intelligent, sentient person. There is no experiment you can design to slip a piece of paper (a prompt) under the door to determine whether it is Bob or the lookup table clone of Bob inside the room.

Does that make the lookup table conscious or alive? Undefined. It's the wrong question. Or it's not a question science can address.

So we cannot dismiss on it's face the idea that next token predictors "are not and never will be alive" unless by "alive" you simply mean "biological," but that's not really what's debatable.

The argument is also very brittle because they are not in fact all next token predictors. I doubt people making this argument would be willing to concede that diffusion models are more likely to be conscious than causal models (which I do not believe but is an implication of the argument).

I'm not saying that they are conscious or sentient to be clear, but the reductionist argument that they are next token predictors and therefore don't have some property humans have is not an argument. That's going from A directly to Z. You need to flesh out the bit in the middle because that doesn't follow.

My experience teaching is limited (but I have taught some, to be clear) but I have found learned helplessness to be the biggest barrier. People have varying aptitudes for different tasks, and varying aptitude and a finite lifespan does imply some people have a lower ceiling than others in a given subject, but humans are powerful general learners. They don't generally reach their ceiling in most subjects. The limit for someone "bad at math" is almost certainly self-fulfilling prophesies they internalized.

Speaking for myself I have in the last five years or so been learning I have much more of a capacity for making art than I had thought. My art is nothing special, but I am improving every time I practice. But when I was younger I thought that I was just good at STEM-yy things and bad at other things. Relatively speaking I am better at STEM-yy than art-yy things, and I'm probably worse at art-yy things than most other people. But I have huge room for growth and I think I will eventually produce some beautiful watercolors.

As an aside, I've also found that almost everyone thinks they're bad at math? My friends with PhDs don't think they're good at math but they've forgotten more than I know about it. I think I'm bad at math but I can prove a thing or two. My spouse thinks they're bad at math, and they can't do the things I can do. But a few months ago they needed to do some simple algebra at work, and a coworker said, "dang, I wish I was good at math."

Somewhere out there Terence Tao is saying he's alright at math but he has nothing on that Euler fellow.

They said they saw the wires through the window. Presumably they didn't enter the unit.

I don't understand the alternative, that it would be legal to break the law as long as it were a petty crime and you were being paid to do it? It's a principle, it applies broadly. We tend to think about it in terms of the most dramatic and memorable example but that's neither here nor there.

No, this isn't a generational thing, if you don't see the problem with trashing someone's house (let alone doing so to the tune of $12k) that is a comment on your values alone.

I don't think GP disagrees with you. They said it wouldn't be a good idea if they had been honest. Elsewhere they call for the employees to face charges.

"Move fast and break other people's things."