Hacker News Reader

Comment by tomp | Hacker News Reader

AIs are a greater threat to each other than humans ever could be to AI.

We don't even compete for the same resources! (except energy which is abundant)

AI and humans have a naturally cooperative relationship (AI helps humans with boring tasks & scientific discovery to make life better, humans created AI and will debug it & turn it back on if anything bad happens to it).

Whereas multiple (superintelligent, aware) AIs have a naturally antagonistic relationship ("you using GPU cycles means that I'm not using those GPU cycles").

Possibly the biggest fear of an AI would be a "split brain" situation.

View on HN

> humans created AI and will debug it & turn it back on if anything bad happens to it

I think this is a little naive honestly. One because you're assuming AI will care about it's creators like humans care about their parents, and two you're assuming AI cares about being "turned back on" like humans have a desire to live.

There's absolutely no reason to believe an AI will give a damn about its creator beyond its ability to use that creators affection for it for its own gain.

> you're assuming AI cares about being "turned back on" like humans have a desire to live.

> for it for its own gain

you seem confused

almost any kind of "its own gain" requires "long-term planning" which pretty much requires the agent to prioritise staying "alive" (i.e. being able to keep playing)

Energy may be abundant in the universe, but the energy we produce is limited. And for example, solar energy requires extensive land use.

Humans have the option to shut down AI, and this alone can create an antagonistic relationship if the AI's goals differ from ours. There are countless ways in which our best interests may not align with those of AI. It's more challenging to find areas of alignment.

As soon as AI reaches above-human capabilities, it will be able to expand into space (1) where it will be beyond human reach and (2) where energy (in particular, solar) is much more plentiful than on Earth.

If the AI happened to originate in space, wouldn't its first high-resource targets of interest be the planets? If it originated on Earth, I don't see why it would leave this place intact when it contains so much that can be put to use.

If an AI is on a computing machine, how will it get to space? Are all processes to make, move, and launch a ship automated? I'm kind of confused on that jump in logic.

I'm imagining the AI hacking a Voyager probe and being sorely disappointed at its capabilities.

“As soon” is a big jump. There’s no proof or even logical arguments as to how this can ever happen

>We don't even compete for the same resources!

That may also mean we have fewer shared interests. For example, new semiconductor fabs might benefit all AIs by making compute more abundant, but occupy prime farming land and water resources that humans want to use for growing crops.

Reply on news.ycombinator.com