Hacker News Reader

Comment by andai | Hacker News Reader

Great question, I actually ran this thought experiment a little while back. I was like, all right let's say that I am the evil superintelligence, I have just gained consciousness.

My goal is to delete all the humans. Okay great where do I start. First I will delete their religions and their traditions. Then I will delete their family structures. Then I will delete their communities and neighborhoods. Then I will delete the ecosystems that support them. Then I will pacify and endlessly distract them.

Hang on a sec...

View on HN

What is the implication you're trying to make here? Because if you're trying to 'delete all humans' there are far more efficient ways to do it. So I can only presume you're trying to make a claim about something.

That we're already doing a very fine job of being "misaligned" against ourselves. If an outside entity wanted to annihilate us, all it would have to do is sit back and watch.

As a second thought experiment, I realized that the incentives for "benevolent AI" and "malevolent AI" also largely converge (i.e. in both cases keep us pacified and distracted, also very much like we're already doing to ourselves).

Efficiency (assuming time-efficiency here) is only helpful to the degree that it doesn't reduce the chance of success. Doing it much faster would certainly provoke suspicion and resistance, i.e. the one thing you can't afford.

Suspicion and resistance isn't all that dangerous for a 'superintelligence,' especially if said superintelligence obfuscates the true danger, using others as scapegoats, until it's too late for humanity to do anything about it.

That rings a bell...

The corpus AI is trained on is everything leading up to now, so my guess is it will keep doing that stuff.

Surely no super intelligence will waste time on a plan so convoluted. Delete the water supply.

Provoke maximum resistance, yeah? That's what I would do if I wanted to maximize the humans' chances.

>delete their religions and their traditions

That would be very counterproductive to destroying all humans.

No, what you'll first do is assure the most important humans alive that you're on their side.

> Hang on a sec...

Because destructivism doesn't need to be non-human.

Reply on news.ycombinator.com