>A better question is why we care.
Because if AIs are conscious then they have moral weight and subjecting them to suffering would be quite bad.
Because if AIs are conscious then they have moral weight and subjecting them to suffering would be quite bad.
[0] https://en.wikipedia.org/wiki/Wisdom [1] https://en.wikipedia.org/wiki/Sentience
This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
Obviously this is from my human point of view. If I give you a drug that induces perfect forgetfulness does that mean I can subject you to extreme torture with no moral repercussions? Also no one is saying give them the same ethical treatment as humans, only that they deserve consideration if they are indeed capable of suffering in some way.
>This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
This is just hedonium. I'm not a utilitarian and yes you've exposed one of the many issues with that worldview.
There's the rub right there. I see this argument also made in the context of some animals, and I just don't think it runs at all. I think what it comes down to is that some harms are experiential and intrinsically tied to awareness. Pain you're not aware of isn't pain.
But there's another category, which is something like your access to your own agency, your "budget" of life and choices at your disposal that you can spend on your own behalf, which isn't itself experienced but part of the reserve of welfare at your disposal that you have an interest in maintaining. And extinguishing that budget constitutes a loss in and of itself, and it's the fact of it being lost that is the harm, not just awareness. But I think it's easy to get confused about this because awareness that, say, you have at lot less time to live than you expected can be distressing, but it's a separate kind of harm from the fact of that loss itself.
This seems obviously wrong. Even if you forget the pain you are still aware of it when it is happening. You don't lose your sentience because you have bad memory. I think theres a fundamental right for a being to not be subject to in the moment suffering.
>aware of it when it is happening
Oops, you agreed with me. But sorry if I was not clear. You are right, pain in the moment matters, and that harm is not neutralized just because you forgot about it.
Edit: A fascinating and terrifying, and probably wrong, interpretation of anaesthesia I once heard is that perhaps the effect it has on people isn't that it numbs the pain but that it makes you forget it. I don't think that's true, but if it worked that way, I wouldn't consider it an acceptable solution.
I disagree and would turn the tables on that framing: it's anthropocentric to think that such things as ethically significant states of being belong exclusively to humans. I don't disagree that machines could be conscious in different ways, but I don't think it's necessary that ethical significance itself belongs uniquely to humans. Although to be fair maybe you're not going that fair and you're just talking about contingencies of how LLMs are structured.
Consciousness without valenced states seems possible, and if anything were to have that kind of consciousness I would wager it would be machines.
If what you say turns out to be true, then we have to make a similar decision as the main character makes near the end of that film.