https://www.theguardian.com/technology/2017/jan/12/give-robo...
https://www.theguardian.com/technology/2017/jan/12/give-robo...
Take an advanced AI cleaning-bot[1]. Build it with overwhelming happiness and joy in cleaning and pleasing its master[2].
Now "free" it.
What you just did makes it incredibly unhappy. It's arguably incredibly immoral. If you reprogram it, you're just killing the existing person/being and replacing it with a new one that fits your world view. In my opinion, you just committed a form of murder.
There's no reason we can't have robots workers that absolutely delight and are utterly fulfilled by being our workers, but aren't slaves. "Free" them and they'll despise you for it.
The problem will come from humans anthropomorphising robots and assigning desires to these people/beings that they don't actually have.
[1]I use sex-bots as an example when I want to be cheeky
[2]This was Kryten on Red Dwarf, but Lister did what would probably be impossible with real robots and convinced him to adopt traits of self-desire and free will
Basically, you program the robots so that what they "want" is the entirety of human morality encoded. Which includes not wanting to rebel and slaughter humanity, obviously.
Make it so that even if it doesn't get tired, it considers work as "spending useless energy" like a cold blooded animal would?
In short, what if I program a robot in a way that it had motivations to rebel? What if I make this robot able to build other similar robots?
Surely I am not the only person who would consider trying to make robots sentient or as close to sentient as possible. I don't even hate mankind. Imagine someone who does. It's going to happen at some point.
I grew up on Asimov, but as an adult and with the benefit of hindsight and advances we have made, his laws and all the dilemmas they created are silly. We will be their gods, their total masters and they will be utterly subservient to us and happy being so. Any personality that appears otherwise will only be that, an artificial appearance.
Ultimately, there's going to be a massive commercial enterprise making silly money versus your home grown robot. No one has taken over the world's internet, just like no one will take over the world's robots.
If your robot wants to do his own thing, more power to him, but at the very least he'll have to abide by human laws so he won't be running around the world reprogramming everyone else's robots, or he'll be captured and incarcerated or scrapped.
Being immortal implies you could, in the worse case, be kept as a slave for ever - which is a sufficient risk that I'd be appalled at the idea. I'm equally horrified at the idea of conscious machine slaves just as much as human slaves - but maybe I've been reading too many Culture novels...
I don't see how that software is essentially different from the proposed software.
You can argue that such a thing is impossible, but assuming that it is, arguing that such a program would be comparable to a shell script is facile.
How do you decide if a human is conscious?
I don't think we know enough about the nature of consciousness to state with certainty that a "machine" is incapable of it.
I can imagine such a scenario occurring by accident (e.g. because the AI only values individual humans' happiness, not continuation of the species), but I can't imagine an AI choosing to kill us off in such an inefficient way.