Academic research has always progressed and pushed the boundaries. Is that a $badthing to you?
Humanity's strategy so far has been to pull marbles out of the bag as fast as we can. And that's been working out, because most marbles are white. But we're insanely lucky we haven't had a nuclear war. The USSR's early warning systems picked up 5 incoming american missiles in 1983. They didn't fire back because one of the officers on shift didn't believe the reports (why only 5 missiles?), and he declared the reports were fake.[1] If a different person was on shift, there's a good chance nuclear war would have broken out because of a sensor glitch. And that's far from the only instance. The list is terrifying: [2]
And now we're improving AI capabilities as fast as we can, despite it being pretty clearly another grey ball. AI has a lot of amazing uses. But why are we rushing so much? Doesn't seem wise.
[1] https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...
> But why are we rushing so much?
Talking about these things with elaborate analogies only obscuring the point you already stumbled into: Humanity advances because humans are curious, driven, like to discover things, and like to push the boundaries.
Talking about humanity like one entity that operates together misses the point: There are billions of people in the world and some of them are going to discover new things.
There is no general counsel where humanity gets together and decides what humanity works on. There is no singular bag with a singular “humanity” pulling marbles out one at a time.
The analogy is too broken to be useful for the discussion.
This is true, but not all the time, and transparently not the case for the current leaders in AI development. Their motivator is money.
There are plenty of examples of this, e.g. when we discovered how to harness nuclear energy, we quickly realized that extreme measures needed to be taken to ensure we don’t collectively destroy ourselves.
You’re speaking of “we” as if the whole world was operating in unison, while there’s a war going on in Iran over nuclear disagreements.
The nuclear situation only reached stability after multiple countries fully developed it. I don’t see the parallels to halting AI.
The nuclear stabilization that occurred was the result of a collective understanding of the existential risk it posed leading to anti-proliferation agreements and superpowers going to great lengths to prevent nations deemed as unstable from obtaining such technology.
Some portion of the global community may collectively discover/decide that the risk AI presents is worthy of anti-proliferation treaties and wars may be fought over it.
The main gap in the comparison is that we have a much clearer understanding of the risks of unchecked nuclear weapons use than we do about what current and future AI tech might or might not unleash.
The underlying point is that we have historically placed limits on our curiosity when we deem certain paths of exploration existentially dangerous. It has yet to be seen if AI will reach those levels of existential risk, but if they do, there is plenty of precedent for stepping on the brakes.
Nuclear Non-Proliferation Treaty (1968)
Biological Weapons Convention (1972)
Asilomar Conference on Recombinant DNA (1975)
We've done this sort of thing plenty of times around specific, dangerous technologies.
Also the bio/DNA ones are quickly falling apart as the technology to do these kinds of things on a small scale are readily available. To the point where we're finding illegal wet labs in the US.
Nuclear was more of an exception because of many of difficulties in purifying stuff for weapons and its detectability.
AI is a much, much, much harder problem is it uses the already existing infrastructure as a base. Unless we regulate, monitor, and ban all private usage of technology AI can be developed at a small scale.
With AI, the point is who uses it for what.
Think about it like a magic wand that makes your wishes come true. Would you really want everybody to have one? Who should and who shouldn't? Who gets to decide?