Hacker News Reader

Comment by vasco | Hacker News Reader

At the end game, a "non-safe" superinteligence seems easier to create, so like any other technology, some people will create it (even if just because they can't make it safe). And in a world with multiple superintelligent agents, how can the safe ones "win"? It seems like a safe AI is at inherent disadvantage for survival.

View on HN

The current intelligences of the world (us) have organized their civilization in a way that the conforming members of society are the norm and criminals the outcasts. Certainly not a perfect system, but something along those lines for the most part.

I like to think AGIs will decide to do that too.

I disagree that civilization is organized along the lines of conforming and criminals. Rather, I would argue that the current intelligences of the world have primarily organized civilization in such a way that a small percentage of its members control the vast majority of all human resources, and the bottom 50% control almost nothing[0]

I would hope that AGI would prioritize humanity itself, but since it's likely to be created and/or controlled by a subset of that same very small percentage of humans, I'm not hopeful.

[0] https://en.wikipedia.org/wiki/Wealth_inequality_in_the_Unite...

It's a beautiful system, wherein "criminality" can be used to label and control any and all persons who disagree with the whim of the incumbent class.

Perhaps this isn't a system we should be trying to emulate with a technology that promises to free us of our current inefficiencies or miseries.

Our current meatware AGIs (corporations) are lawless as fuck and have effectively no ethics at all, which doesn’t bode well.

They well may, the problem is ensuring that humanity also survives.

The example from our environment suggests that the apex intelligences in the environment treat all other intelligent agents in only a few ways:

1. Pests to eliminate 2. Benign neglect 3. Workers 4. Pets 5. Food

That suggests that there are scenarios under which we survive. I'm not sure we'd like any of them, though "benign neglect" might be the best of a bad lot.

Considering the U.S. is likely to have a convicted felon as its next president, I don't agree with this characterization

Reply on news.ycombinator.com