Why Technologists Fail to Think of Moderation as a Virtue
lareviewofbooks.org
lareviewofbooks.org
"Long before existentially troubling control problems will arise [...] these mediators will shape an immense number of decisions" - certainly a true statement! But true in the same way that Enron can create an immense fraud long before the energy market causes existentially troubling climate change problems. It would be hopelessly silly to read a book on climate change and respond that it should have been about accounting regulations.
(Sidebar: I don't think the title was cut off in quite the right place. "Moderation as a virtue" is a reference to one particular criticism by Ted Chiang and isn't really a good summary of what this article is responding to.)
If I understand it correctly, this is saying:
- people like Elon Musk worry about AI risk because the hypothetical out-of-control AI would be thinking a bit like they do;
- therefore we shouldn't worry about AI risk.
I don't see why anyone should find that line of reasoning convincing. At most I think it gets us as far as "we shouldn't consider people like Elon Musk as authorities on AI risk".
Wouldn't it be better to look directly at whether the runaway AI scenario is plausible?
It’s not that Musk fears AI bc the hypothetical runaway computer will think like him. Hard to restate without reusing the word projection, but it’s more like that his concept of AI risk itself says more about him than it does about future technologies.
Weird example, but it’s kind of like how in Dr. Strangelove General Ripper’s fear of a soviet plot to emasculate us via fluoride has everything to do with Ripper’s own feelings re his balls and nothing at all to do with the actual soviets.
It’s ridiculous to think Bill Gates and Elon Musk are worried about AI because “we would stop at nothing to achieve our goals, AI must be the same way”, and also not on-point even if true. Elon Musk and Bill Gates aren’t thought leaders in AI safety, they are proponents.
I expect a sufficiently advanced AI would modify itself in ways that would reduce inefficiency to its operation, in order to improve efficiency in pursuing whatever goals it has defined for itself.
And if it can set its own goals, we should be careful not to project ourselves into its role in order to understand or predict its behaviour; it would be unlike any intelligence found in nature.
Yes but that burden has not been met by the people who talk about AI risk. They don't properly motivate why theirs is a plausible scenario, that is to say, provide actual evidence for their quite fantastical claims about the capabilities of AI that are grounded in empirical research rather than science fiction.
Douglas Hofstadter asked decades ago if a computer who is able to think will still be able to calculate fast, that is to say, he questioned whether endowing machines with intellect and symbolic reasoning will actually slow them down by the very nature of what intelligence is. I've read quite a few of the alarmist AI treatises over the recent years, and it's funny how few of them actually even engage that prescient question.
What Chiang is saying is that the entire AI risk complex is a mirage conjured up by the neuroses of billionaires who are projecting the problems that their very own creations have produced (as in their companies) onto the topic of AI. That it's a psychological phenomenon rather than a technical one. When these people are concerned about autonomous systems that turn the planet into paperclips and disregard humanity and ecology they're not actually thinking about general intellect, they're thinking about capital.