The world is actually pretty messed up with problems right now; some of which we could actually do something meaningful about. Continuing the naughty-AI bunfight isn't one of them.
AI is inevitable. It's very likely to happen in our lifetime. And it's very likely to kill us all. And explaining every reason why I believe those things could fill an entire book, so I seriously recommend reading Superintelligence by Nick Bostrom.
This is absolutely absurd. Could you elaborate on that instead of just pointing to a book?
The second point is that they would have no reason to keep us around. They would do whatever we programmed them to do, to the point of absurdity. An AI programmed to collect as many paperclips as possible, would convert the mass of the solar system into paperclips. An AI programmed to solve the Riemann hypothesis would build the biggest computer possible to do more calculations. An AI programmed to value self preservation would try to destroy anything that was even a tiny percent chance of being a threat to it. It would try to preserve as much energy and mass as possible to last through the heat death of the universe. Etc.
The only AIs which actually do things we want them to do would have to be explicitly programmed to want to do that. And we have no idea how to do that. We can't just hope that arbitrary AIs will happen to value humans instead of something else. We need to figure out how to control them, which requires solving something called "the control problem".
This is where this line of reasoning completely falls apart for me. Any complex system is going to do unexpected things that weren't deliberately designed for. My laptop computed doesn't do "whatever it was programmed to, to the point of absurdity." Why would you expect a program complex enough to simulate the human mind to act in such a predictable way, when none of the much simpler programs we already have act that way?
> Why would you expect a program complex enough to simulate the human mind to act in such a predictable way, when none of the much simpler programs we already have act that way?
Exactly. We suck at making complex programs do what we want them to. This is the source of worry.
A program "complex enough to simulate the human mind" would probably have even more bugs that result in unintentional behavior. Not fewer.
Questions:
(1) What reason do we have to keep chimpanzee's around? Why don't we just kill them all? They can't even do calculus.
(2) Whatever the answer for question 1, wouldn't an AI consider that same scenario regarding humans? Assuming an AI is truly smarter than a human?
(3) If AI gets to the point where they are much smarter than humans, wouldn't it follow that there would be AI philosophers? Or activists? Or pacifists? Would there be such a thing as an AI civilization? Or is your scenario just a few AI overlords that some how slipped through the cracks of regulation by humans?
2. Empathy, which is a part of our emotions, is an incredibly complex thing we can't even begin to formalize yet - it's extremely unlikely that a random AI we'd build would happen to have the exact same emotional makeup as we do.
3. "Smart" has a particular meaning here when used in discussing AI. It implies raw intelligence, not morals. A smart AI may be incredibly good at evaluating information, inferring new facts and doing complex tasks. None of it means it will necessarily have to ask itself philosophical questions that we do - those actually mostly come from our emotions, from our desire to understand our place in the universe.
The core of the AI-threat issue is that for an AI to not be dangerous it would have to value us and the same things we value. We can't agree about our exact values even among ourselves, and we're nowhere near even trying to formalize it on a level of detail that would be useful for programming.
Read a book recently that contained an interesting AI attack. Basically, it set a lot of small objects in motion, with complex orbits all computed to result in a large impact at some point in the future.
We would not notice something done this way, by analogy.
Most technologies we build, and use, surpass humans in some capacity. Thinking is no exceptions and we allready have machines far better than humans at some thnking tasks, such as arithmetic.
We are now working on systems surpassing humans in various recognition and prediction tasks.
I suspect the catalog of tasks far better done by machines will continue to expand to finally encompass all tasks required to form a general intelligence. Only, the trick to assemble them, into a complete system could very well prove tricky.
The result, though, is that there will never exist such a thing as human level artificial intelligence because, if and when, a system of general intelligence is created all its components will far surpass human equivalent capabilities.
So "smarter" simply means far superior to any human in any task requiring thinking, in the same sense that computers are far superior to any human at arithmetic.
Even some odd genius savant of today cannot compete with computers at number crunching. More interestingly it's not even interesting to compare, the difference in potential performance is just at different scales all together.
In fact, by relying on a secondary AI it's quite possible we're even more likely to accidentally end up with AI that isn't benevolent.
For instance the other comment brings up the uncertainty around giving an Oracle AI the task of finding what benevolence means to humans, to then plug into the final AI hoping it will be Friendly. You don't know how to precisely define 'benevolent' so let the Oracle AI do it (or help you do it), though you think you can at least program the Oracle AI to do a good job in finding answers to vague, complex, fragile problems. (Why? Past success in AI tech?) Is what the Oracle AI outputs actually good? How do you know? What did the Oracle AI have to do to reach its answer, or gain a last bit of certainty in it? (e.g. Did it reason it needed more data and so built or hastened the arrival of brain-scanning technology, started scanning brains from volunteers, the very recently deceased, cryonics patients, or just without its creators knowledge through black market deals, and at some point did it simulate trillions of human minds under all sorts of gruesome scenarios to test responses? Do you even morally care about sentience on silicon? Would you care more if you were an upload?) You've got a lot of problems just building an Oracle AI (or anything powerful enough that can help you solve the hardest problems). Even supposing the output it gives is good and the costs (practical and ethical) were acceptable, will a seed AI allowed to recursively self-improve be guaranteed to preserve this Friendliness property in all subsequent improvements?
With the idea of using "AI tech", you've basically drawn a line in the outcome space that on one side says something like "let's just program the thing" and the other says "hold on, let's just program the thing but also use all these other relevant computer tools we've developed over the decades to help program the thing". It's the start of an approach, but does it really prune all that much? What do these relevant tools actually buy you in safety, if a safety-guarantee-tool is not among them? Another line which may be what you were heading towards, could be on one side say something like "we might not be able to solve all the necessary problems with our current intelligence, so let's also spend time looking for ways to augment human intelligence -- brain-computer interfaces, brain emulations, tweaks to our genetic code, that sort of thing" and the other side "no, we're capable, let's try our best right now". Using ems based on friendly-ish human minds that become better than human could very well be a better approach to solving the superintelligent problems correctly and efficiently than doing so on raw human levels, on the other hand it could turn very bad if one of those ems instead just solves the problem of recursively improving themselves and their goal and value system is insufficiently friendly. Or many other failure modes. (Though an interesting "most likely scenario" given certain assumptions and only looking at a span of 2 years which by itself doesn't look too terrible is Robin Hanson's Age of Em idea.) Just like approaches involving molecular nanotech to help us, there's reasons why it could be seen to help the best outcome along, but it also opens the door to a lot of other risks that aren't necessarily there if you took the other side, so again how much is really pruned?
What I take as your general idea of "maybe we need and should employ additional help and knowledge that we've yet to gain to even really start" isn't bad, though, much better than "Benevolence is easy, a King and his subjects all know he is a great ruler when everyone is smiling frequently", it's just lacking in detail to constitute a plan. Throwing in the positive feedback loop idea (which might even manifest as impressive accelerating returns, who knows) to me seems only relevant to figuring out time scales, I don't think it says anything about the fraction of good vs. bad outcomes apart from whether short or long time scales matter.
My general take is that, AI or not, the world is continuing to progress towards a future where extinction level capabilities are not only within reach, but also within reach for more and more people. It is, to me, not a question of if, but when, such capabilities will be in the hands of an individual, or organization, that either by malice, or incompetence, would be a serious problem.
Last year a pilot intentionally crashed a plane in act of sucicde, taking 150 passengers with him. Consider a likeminded person capable of engineering a virus.
So, my optimistic take on the future is that while such capabilities are developed, one would hope, that in parallel capabilities to mitigate those threats are also developed, so that when the problems arise the combined capabilities of all benign actors is sufficient to evade extinction.
In the case of AI it just seems implausible that any debate could manage to stop the technological race. Just look at the climate change debate, it has been going on for decades and is just gaining political traction. AI isn't even on the radar yet...
So the plausible futures too me are either that AI will wipe us out, or not. In the event that it does not I think one plausible future is that the potentially malign AI will operate in an environment where other AI, or near AI-systems, will also perceive it as a threat. Hopefully aggregating into the capacity to suppress the dangers in a benign way.
Also, I believe any path from here to AI will consist of a complex system of AI-like technology interwoven with human systems resulting in some aggregates with considerable capablites itself not to be discounted.
In some sense I believe we can extrapolate how things will play out just by looking at some systems that might resemble AI today. Consider corporations, these are actors created from a complex system of legal, economic, social and other constructs. While ostensibly owned by, and operated by, humans, the reality is that there is little any individual human can influence them in view of all other forces directing their actions, making them, in a sense, allready existing, AI:s.
Creating actual AI, as a useful thing, will probably involve similar complexity and influences. So in some sense there are probably similar patterns that will play out. Just as corporations can act to influence the world in ways malign to humans, the AI systems would. And just as corporations, the AI would find it self acting in an environment doing its best to contain the malice.
An alien species would have to be observing this planet at very frequent intervals to be able to catch any of these recent developments -- because if millions of years ago they determined nothing interesting was happening, and decided to only look in every five hundred thousand years or so, it's very likely the last time they looked our species wasn't even around.
Now you take the development of an AI that is roughly human level intelligent with a given amount of resources. If you give it the same amount of resources again, it can "breed", copy itself perfectly onto them, and now we have a "population" of two AIs whose combined intelligence should roughly match that of two twin humans. Maybe it doesn't even copy perfectly, maybe it switches some things about to see if it can create something smarter without modifying its own code directly. In any case its breeding is only restricted by its ability to breed and desire to breed in the first place and the compute and power resources it needs to run a new copy, and these can always get lower than initially. What does the long-term look like? Even if the initial resources for the first one are astronomical (e.g. all computing power in use on the planet as of 2016), I would still bet you that left to breed without any other restriction it would take the AI family far less time than 100 thousand years to reach 7 billion instances. So you're looking at many, many more AIs than non-AIs, plus each with human level intelligence that isn't distributed normally (assuming soft eugenics doesn't become widespread in non-AI populations) but is mainly all the same with perhaps increases here and there (assuming a soft takeoff), each coordinating with the same goals in mind. This situation without further specification can be either extremely good for the non-AIs or extremely bad for the non-AIs. The question of whether this will happen (and whether it happens in that exact form, personally I think hard takeoff from a single AI is likely) in the next 100 years or the next hundred thousand years (perhaps a near-extinction-level event occurs forcing our species to basically start over but with an even harder battle to survive since many low-hanging-fruit resources have been depleted) might help you determine whether to worry now about doing actions that make the extremely-good more likely than the extremely-bad, but if one thinks human-level AI can't happen at all in any amount of time, that there's something special about our squishy brains or something inherently limiting about our so-far-general intelligence such that we can't solve the engineering problem of creating another intelligence directly and must always go through breeding, the argument needs to take place at a different level.
The paperclip maximizer argument (not sure if Bostrom repeats it in his book) is yet another level of argument but it's meant for those who already buy the premise of human level AI and self-improving AI but who also think increased intelligence will naturally converge to a human-loving benevolent AI that sees how self-evidently obvious it is to love all life or whatever, that we don't have to worry about all the messy details around Friendliness because they'll just fall out of the entity naturally. No one is actually worried about Clippy tiling the solar system with himself, its probability is epsilon. But a lot of people just see this one argument out of context and infer the arguer believes it to be a relevant possibility to worry about and write off the whole group as crazy...
That's interesting. How could it? Please present a plausible scenario (more plausible than "The US president decides to cover the globe with nuclear explosions, and everyone with any degree of influence over the process just lets him do it").
https://en.wikipedia.org/wiki/Cobalt_bomb
[Of course, they don't exist at the moment - but they wouldn't be that difficult to make as one advantage of a Doomsday bomb is that you don't need to deliver it so it can be made as large as you want through staging].
I said that it would take a book to explain. You are parodying an extremely shortened summary of a complicated subject.
The difference between AI and "alien" is no one is trying to make alien. There aren't research labs and billions of dollars in funding. There is no incentive at all. And the problem is far more difficult. AI is just algorithms.
Second, aliens arent actually that powerful. Superintelligent AI would be godlike in power to us. I'd much rather have aliens.
The purpose of my facetious approach is to make clear that statements such as, "Superintelligent AI would be godlike in power to us," are more likely to have originated by analogy to science fiction than from a grounded understanding of what is possible or realistic in the relevant future.
Being "just algorithms" doesn't really mean very much. It's like saying breeding the Alien is "just genetics". It's something we imagine might be possible, but is far beyond our current capabilities.
We live in a world where the only intelligences are humans. The idea of superintelligent minds seems absurd to us, because it isn't something we've ever encountered in reality before. But there is absolutely no law of nature that says it's impossible, or even difficult. If you look at history, it'd actually be really surprising if it was. Human minds were just the very first intelligence coughed up by evolution. We are far from the limits of what is possible.
And there is a difference between genetic engineering and AI research. Biology is incredibly complicated. Like millions of complicated, interconnected, moving parts. It really is impossible to make Alien, or really anything beyond modifications to existing biology.
Algorithms are a whole nother ball game. Our current AI algorithms are actually reasonably intelligent at many tasks, even smarter than humans at some things. And they are extremely simple. Could be described in a line or two of equations, or a few paragraphs of text. Not because they are primitive, but because simple algorithms work. And they have been improving every single year for a long time, and will likely improve well into the future.
But you don't have to take my word for it. If you survey AI researchers and experts, they almost unanimously agree that AI will be invented in our lifetime. If you survey biologists, I highly doubt they would make similar claims about Alien.
What's the point of arguing about the potential threats something we have so little knowledge of? Maybe it will come too fast, but maybe that unkillable virus will, too. At this point in time, we simply have too little information to have an intelligent assessment of the risks posed by AI. We can and should, however, discuss the more immediate related dangers of our real-world machine learning, such as self-reinforcing bias.
I'd be interested if you said more about that.
Think what could happen if your brain worked much faster. The world will mostly seem to slow down. A very conceivable outcome is one of boredom and possibly madness. You could perhaps multi-task and think of many different things, but we are generally terrible at multitasking, so we have no idea whether an intelligence even could multitask well.
Another problem is that we don't know anything about the information capacity of a mind. A faster mind of the same size could potentially have trouble dealing with all that extra information (again -- madness, maybe?)
One thing that hints at those capacity limits is the need for sleep. We don't know why, but some hypotheses say that it's necessary to do some "information cleanup" operations in the brain. It is possible that an AI would need to sleep, too. The AI may need to sleep a lot more to handle much more information.
It is therefore conceivable at least that a mind has to work at a speed that is commensurate with the speed of the world around it, and one that is appropriate for its capacity.
Quoting to emphasize.
It is one thing to acknowledge that there is a non-zero probability that an uncontrollable AI might be created, but that we have other things to worry about and so why hand-wring over it.
What should be the strong counterpoint to this hand-wringing is that what we should be doing is better understanding the behavior of the "weak" AI we have now. We need grasp where it does and doesn't work, a difficult problem to "sell" since it lacks that alien-like doomsday aspect or predatory agency.
These kinds of problems are the problems of our own creation and thus we must take ownership.