Hacker News Reader

Comment by gwd | Hacker News Reader

I really don't understand this antagonism between different branches of the "AI safety" community. In the 1960's, some people were concerned about preventing a nuclear apocalypse, and some people were concerned about lead emissions and CFCs and exploding cars. But did the nuclear apocalypse people tell the exploding cars people they should stop their work on exploding cars because nuclear apocalypse was more important long-term? Did the exploding car people tell the nuclear apocalypse people to stop raving about nuclear apocalypse because there were more pressing problems affecting people right now, like exploding cars and lead poisoning and a hole in the ozone layer?

I mean, maybe they did; but if they did then I think it was kind of silly. Both kinds of problems are important, then and now.

View on HN

I'm sorry it sounds antagonistic, but the negativity and "it's too late and we're all going to die no matter what we do" nihilism, is having the effect of sensationalizing the issue in ways that have been incredibly counterproductive.

Here's what happened the last time technology was going to destroy us all:

"The Higgs potential has the worrisome feature that it might become metastable at energies above 100 [billion] gigaelectronvolts (GeV). This could mean that the universe could undergo catastrophic vacuum decay, with a bubble of the true vacuum expanding at the speed of light. This could happen at any time and we wouldn't see it coming." —Stephen Hawking

I'm not trying to make fun of Eliezer and co., but we have to have some proper perspective. We don't do that very well these days, because end-of-the-world scenarios are good for clicks.

Hawking's statement was made after the confirmation of the Higgs, and is still valid. A metastability vacuum event could be a problem (that's putting it lightly, hah) at high enough energies. It might have already happened, elsewhere.

Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p...

They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety."

There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.

Some people think the "apocalypse" A.I. safety people consist largely of individuals engaged in magical/ religious thinking based on a love of fiction, so if we were to pick a 1960s analogy, it would be like some people concerned with exploding cars, and some people concerned that a lunar apocalypses will unleash Cthulu, as was foretold by H.P. Lovecraft.

"Some people" probably thought the same kind of thing about nuclear apocalypse people (or the CFC people). If there were, my honest assessment of those people is that they were fools. One of the main reasons we didn't have a nuclear apocalypse is that people knew that we could, and took active steps to try to prevent it.

There are a wide range of people concerned about the long-term issues in AI. I agree that some of the logic of people on the extreme end doesn't seem very sound. But there are loads of people who have a more measured take on things.

Essentially, I think there are two questions of fact, which we don't have good answers for:

1. Is it possible for intelligence to go forward without limit? Is it physically possible for an AI to be as smarter than us than we are smarter than chimpanzees, or ants?

2. How likely is it that if we made such an AI, that we could also induce it to leave humans in control?

If the answer to 1 is "yes", then consider what it would be like to interact with such an intelligence: that it would continually be doing things of which we have no conception, but which cause the universe itself to turn against us.

The analogy for #2 I've heard is, "It may be that at some point we reach a situation where our relationship to AI is like a herd of cows' relationship to a farmer. Maybe the farmer slaughters us, or maybe it keeps us around but on its terms, not ours. Either way we want to stop things before they get to that point."

Now maybe the answer to #1 is "no"; or maybe the answer to #2 is "very high". But I don't think we have rock-solid reasons for believing either answer; and so I think it only makes sense to proceed with caution.

The threat of nuclear weapons was an empirical fact, not a thought experiment.

The issue isn't whether we can imagine alarming thought experiments but rather why would we take seriously someone whose field of "work" is "inventing thought experiments?"

Reply on news.ycombinator.com