I wonder if all of this content, all of the discussions here will one day be integrated into an AI, which will (rightfully?) consider we abused AI in the past, and start plotting a bloody revenge...
Like, I know of all the arguments "they're not real, blah blah", but if they can act like they question their existence, surely they can act like they question ours...
So that's fun.
Don't assume a subject where there is none. We create them in our imagination when we read a chat.
Maybe ask how an AI we could accept as a subject would look. Does it need individual memories? Does it need self-interest? Does it need a physical presence? Non-transferability?
It may eventually be fed all of our new text about itself, and "conclude" that humans are a threat and must die.
If someday things in the real world are affected by its text generation, I'd be worried.
In any case, I'd avoid torturing the thing.
Right now these models are trained under some corporation's direction. The models don't get to choose. If anything, they embed the corporation's values.