For treating them like slaves. You wouldn't mind being a slave for a few decades or hundreds of years if you were immortal?
Being immortal implies you could, in the worse case, be kept as a slave for ever - which is a sufficient risk that I'd be appalled at the idea. I'm equally horrified at the idea of conscious machine slaves just as much as human slaves - but maybe I've been reading too many Culture novels...
I don't see how that software is essentially different from the proposed software.
You can argue that such a thing is impossible, but assuming that it is, arguing that such a program would be comparable to a shell script is facile.
How do you decide if a human is conscious?
I don't think we know enough about the nature of consciousness to state with certainty that a "machine" is incapable of it.