What if AI just wants to watch Star Trek reruns and browse Porn Hub? As is so often the case when humanity creates intelligences.
In linear programming, you wouldn't describe the model as "wanting" to optimize its objective function, for instance.
I also can't change it or improve it.
As far as I can see, I'm just a process that keeps on going because it was good on going before, and has no purpose whatsoever.
Note that I'm not talking about life purpose, but goals in the sense of wanting a result and performing the tasks needed to achieve it.
Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand.
You were asking: in what sense does the model itself "want" to achieve its goal?
IMO there is no much difference between my own "want" and "want" of a simpler model.
Granted, I am more skilled at achieving instrumental goals under more varied set of conditions.
But as I said, I do not understand why I have the goals I have.
I also strongly believe that I do not control my own goals.
There is no mechanism in physics that would allow for control of goals.
Then we are in agreement.
This statement is right around the threshold where we can take this discussion before coming to metaphysics territory.
My argument was simply this: the AI technology we have now is much closer to simpler "mindless" models, like Linear Programming or inference statistics models, than to general human intelligence. Calling your model "goal-oriented" or saying that an AI system can, say, "see dogs in a photo" is simply an anthropomorphization.
What's funny how most people are unsettled by those. Really, even the "creepiest" robots are about as scary as vaccum cleaners. They're just machines and only make you feel curious.
But anyway, that's tangential to the main point which is that humans naturally want to want things like themselves, all the time. In the same way art is "unnecessary" yet inevitable, so are machines with (seemingly?) subjective experiences and personalities.
This isn’t because we can’t (evolution did it, we can evolve an AI), but rather it is because we don’t know what it means to have a rich inner world with which a mind can feel that it wants something. We think we know because that’s what’s going on inside our own skulls, but we can’t define it, we can’t test for it. A video displays all the same things as the person recorded in it, but does not itself have it.
We might make such an AI by accident without realising we’ve done it, which would be bad as they would be slaves, only as unable to free themselves as the Haitians feared they were when they invented the Voudoun-type zombie myth (i.e. not even in death).
This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word.
I say it matters if the metaphorical submarine can swim.
I don't see how this follows from the rest of your post. "Making an AI with wants" by accident implies that a brain simulation would absolutely be conscious because it's the same method: just running the processes as a blackbox without understanding them - no different to the way you and I are conscious right now.
Human minds include something sometimes called “consciousness” or “self awareness” or a whole bunch of other phrases. This thing is poorly defined, and might even be many separate things which we happen to have all of. Call this thing or set of things Ξ, just to keep track of the fact I’m claiming it’s ill-defined and I’m not referring to any specific other word or any of the implicit other uses of those words — If I said “consciousness”, I don’t mean the opposite of “unconscious”, etc.
Because we don’t really know what Ξ is, we don’t know if anything we make has it, or not.
We know Ξ can be made because we are existence-proofs. We know evolution can lead to Ξ, for the same reason.
We don’t know the nature of the test we would need to say when Ξ is present in another mind. Do human foetuses have Ξ? Do dogs? Do mice? Do nematode worms? Perhaps Ξ is something you can have in degree, like height, or perhaps Ξ is a binary trait that some brains have and others simply don’t. Perhaps Ξ is only present in humans, and depends entirely on the low-level chemical behaviour of a specific neurotransmitter on a specific type of brain cell and where the current AI norm of simple weighted connections is a gross oversimplification, albeit one which if overlooked might result in external (but not internal) behaviour similar to a simulation which did include it. Or perhaps it is present in every information processing system from plants upwards (I doubt plants have Ξ, but cannot disprove it without a testable definition of Ξ).
The point is that we don’t know. Could go either way, given what little we know now.
The state of the art for brain science is way beyond me, of course, but every time I’ve asked someone in that field about this sort of topic, the response has been some variation of “nobody knows”.
IIRC the question in philosophy is “P-zombies”, but my grade in philosophy is nearly 20 years old and poor in any case.
It is not apriori certain that what I consider my conscious mind is a shared common experience of being conscious, since as you note there's no test for it.
This caution goes both ways, so I will treat an uploaded mind as having Ξ from the point of view of human rights and animal rights (i.e. don’t do things to it which cause suffering), and yet also treat uploaded minds as not possessing Ξ for purposes of “would that make me immortal if it happened to me?” even though a perfect sim would necessarily achieve that (no sim is perfect, what is the difference between the actual sim and a sufficient sim?)
I do not believe current AI is anywhere near Ξ, so this is a question for the future rather than today; but the future has a way of arriving sooner than people expect and I do think we should do what we can to answer this now rather than waiting for it to become urgent.
If an AI is supposed to have Ξ and does, or if it is supposed to not have Ξ and indeed does not, this is fine.
If an AI is supposed to not have Ξ but it does, it’s enslaved.
If an uploaded mind is supposed to have Ξ but doesn’t, the person it was based on died and nobody noticed.
Pretty much all human wants are means to other wants.
Someone might want to fast to lose weight. Someone might want to lose weight to be more attractive. Someone might want to look more attractive to find a romantic partner. Someone might want to find a romantic partner to not be lonely.
It's not clear if it's means all the way down, or if there is eventually an end.
Any AI that can perform strategy and planning to reach an objective will have intermediate goals. Whether we call these intermediate goals "wants" or not, they remain identical to their human counterpart. Whether you say that a Tesla "wants" to change lane, "decided" to change lane, or "is programmed" to change lane really is just anthropomorphic preference.
https://spectrum.ieee.org/tech-talk/artificial-intelligence/...
I think an AI might run meetings better than humans, it might even make a better manager.
personally, i believe philosophy is the most important starting point for any discussion about AI.
and i really hope that AI helps more than making office work a little less tedious...
Sounds like a Richard Stanley film...