This is different from how we treat rocks, or pieces of metal, or trees. AI is more conceptually similar to these things than to animals. I.e., if someone made a convincing argument that rocks are actually conscious, I don't think we would stop using them to build stuff. We'd just come up with a more useful concept than consciousness.
> even if we don't act as perfectly as we could/should be.
We don't even treat them anywhere close to perfectly, in fact we treat them in the most horrific way imaginable. To most that is tolerable, but it bears repeating.
If not, would you be ok with torturing simulated individuals to make them work?
If yes, then how would you draw a line between a full-brain simulation and a "synthetic" AI?
That doesn’t mean it would be okay to “torture” the simulation and I’m not sure how that would work, but at the very least it would be a different ethical situation from real humans.
We clearly aren't close to implementing them, but why do you think they are outright impossible? Brain is a physical object. Why isn't it possible to simulate its behavior?
> That doesn’t mean it would be okay to “torture” the simulation
So it would mean that they are ethically significant, even if they don't have the same rights as a human being. If you agree, then would this also apply to synthetic conscious AIs?
> it would be a different ethical situation from real humans.
I agree. Their way of living would be very different: you can speed up or pause the simulation, fork it or archive it. Questions like "is it ok for an individual to clone themselves and then kill the copy?" don't even make sense for regular humans, but can apply to simulations.
That said, if for some reason my brain gets scanned one day, I would like my virtual copy to be treated humanely.
The effects of millions of years of evolution on perception are a good example. We can't perceive the "roads not taken" in terms of development, nor can we know if there is "something we're missing" in terms of understanding the brain. So, at best, the brain simulation will just be a mediocre imitation, not a true copy.
Beyond that, though, I think I just reject the notion that the brain is equivalent to identity. This idea is very traceable to European culture circa the Enlightenment, which to me indicates that it's not an accurate understanding of the human Self. This means that the Self includes the body, which implies that merely copying the brain, no matter how perfectly, is not an accurate copy.
Side note: I'm not really sure how you're connecting this to my original comment, which was not normative in nature. I wasn't suggesting that "doing away with consciousness" was a good or correct thing, just that it will pragmatically happen. Fact-value distinction and all that.
> We can't perceive the "roads not taken" in terms of development, nor can we know if there is "something we're missing" in terms of understanding the brain.
What do you mean "we can't"? Of course we do know that "something is missing", because we can't build a good predictive theory of how consciousness work.
> Self includes the body, which implies that merely copying the brain, no matter how perfectly, is not an accurate copy.
So you consider a person with an amputated arm a different person, not just the same person, who just can't use their arm anymore?
2. Evolution has taken us down certain perceptual pathways. We don't know what other pathways exist. They are "unknown unknowns." Ergo, again, we can't have a complete picture of the brain, so we can't have a perfect copy of it.
3. That's not really a good counterexample to the point I'm making, which is that the act of embodiment is intertwined with the concept of the Self. The idea that the Self developed independently of the body is false. So again, if we were to copy the brain and put it in a digital environment, it would not be a copy of the person, nor would it remain the same, as it no longer has a body. Patterns of thought would change, concepts of Self would change.
https://plato.stanford.edu/entries/embodied-cognition/
All of these are just various ways of saying that you could not make a 100% digital copy of a flesh-and-blood human being. It would not be the same person.
edit: Oh, I think I understand a bit more now: we're not talking about creating a simulation of "a brain", but a simulation of a specific brain, and whether that's still the same "person" as the original person the brain belonged to. That's a very different discussion.
2. Doesn't the same logic apply to basically everything else? It doesn't rely on any specific properties of the brain.
3. Replace an arm by a car. Would a person sitting at the steering wheel of a car be different from the same person outside of it? If no, then what's the salient difference between an arm and a car?
Watching the advance in AI, we should depart from a substance based approach and move to an experience based, it doesn't matter if your molecules form DNA or semiconductors (or future stuff), if this hypothetical being has "an experience", it should be treated accordingly.
We should treat all (including animals and other being) kindly, and if we can prove they don't have an experience, treat them as a rock/metal.
1: https://www.nytimes.com/1987/12/17/opinion/l-why-infant-surg...
The present moment is the most important moment in the world for every live being, and yet amongst humans it's pretty low on the totem pole of "things that are important to consider when interacting with other independent beings".