Hacker News Reader

Comment by Aurornis | Hacker News Reader

> Humanity's strategy so far has been to pull marbles out of the bag as fast as we can.

> But why are we rushing so much?

Talking about these things with elaborate analogies only obscuring the point you already stumbled into: Humanity advances because humans are curious, driven, like to discover things, and like to push the boundaries.

Talking about humanity like one entity that operates together misses the point: There are billions of people in the world and some of them are going to discover new things.

There is no general counsel where humanity gets together and decides what humanity works on. There is no singular bag with a singular “humanity” pulling marbles out one at a time.

The analogy is too broken to be useful for the discussion.

View on HN

There’s a point at which logic/reason/wisdom overrides our curiosity, and there are quite literally multi-nation groups that collaborate on matters of global importance.

There are plenty of examples of this, e.g. when we discovered how to harness nuclear energy, we quickly realized that extreme measures needed to be taken to ensure we don’t collectively destroy ourselves.

> There are plenty of examples of this, e.g. when we discovered how to harness nuclear energy, we quickly realized that extreme measures needed to be taken to ensure we don’t collectively destroy ourselves.

You’re speaking of “we” as if the whole world was operating in unison, while there’s a war going on in Iran over nuclear disagreements.

The nuclear situation only reached stability after multiple countries fully developed it. I don’t see the parallels to halting AI.

It's hard for me to understand how the parallels wouldn't be abundantly clear.

The nuclear stabilization that occurred was the result of a collective understanding of the existential risk it posed leading to anti-proliferation agreements and superpowers going to great lengths to prevent nations deemed as unstable from obtaining such technology.

Some portion of the global community may collectively discover/decide that the risk AI presents is worthy of anti-proliferation treaties and wars may be fought over it.

The main gap in the comparison is that we have a much clearer understanding of the risks of unchecked nuclear weapons use than we do about what current and future AI tech might or might not unleash.

The underlying point is that we have historically placed limits on our curiosity when we deem certain paths of exploration existentially dangerous. It has yet to be seen if AI will reach those levels of existential risk, but if they do, there is plenty of precedent for stepping on the brakes.

> There is no general counsel where humanity gets together and decides what humanity works on.

Nuclear Non-Proliferation Treaty (1968)

Biological Weapons Convention (1972)

Asilomar Conference on Recombinant DNA (1975)

We've done this sort of thing plenty of times around specific, dangerous technologies.

And countries still violate these treaties. See Iran and NK.

Also the bio/DNA ones are quickly falling apart as the technology to do these kinds of things on a small scale are readily available. To the point where we're finding illegal wet labs in the US.

Nuclear was more of an exception because of many of difficulties in purifying stuff for weapons and its detectability.

AI is a much, much, much harder problem is it uses the already existing infrastructure as a base. Unless we regulate, monitor, and ban all private usage of technology AI can be developed at a small scale.

Is your argument essentially that because global treaties haven't been universally effective in all cases, we should just YOLO AI even if it has the potential to cause great harm?

No, we shouldn't yolo AI, but because we started the ball rolling and it may be impossible to stop.

> Humanity advances because humans are curious, driven, like to discover things, and like to push the boundaries.

This is true, but not all the time, and transparently not the case for the current leaders in AI development. Their motivator is money.

Reply on news.ycombinator.com