Hacker News Reader

Comment by catigula | Hacker News Reader

The second anyone develops an AI that is more capable than humans they will use them to completely cripple opposing threat actor's attempts to develop AI. Full power grid, economic, social attacks are definitely coming, not sure how you could think otherwise.

View on HN

If superintelligence gives you superpowers, then why isn't the world trembling at the feet of Mensa nerds? There are rapidly diminishing returns on "excess" intelligence. Life is constrained chiefly by resources. There's a baseline of intelligence needed to function in a modern society, but anything above that isn't necessarily all that advantageous.

Transport young Albert Einstein back in time to the Middle Ages? I don't think that would give you Special Relativity.

I didn't say superintelligence.

Even modest intelligence clearly gives you the ability to develop a superior intelligence, let alone many other wonders and marvels that exist in the realm of sorcery to someone living 100 years ago.

That's quite remarkable.

Great way to get yourself nuked, no?

The problem I have with intelligence is that intelligence alone doesn't win a land war in Asia.

Is it?

We've had cyber hostilities with China for a long time now.

An AI could easily be used in a decapitation strike of military systems.

Strike with what?

I don't think we've seen any cyber decapitation strikes, particularly against military systems, but that's exactly the type of action I think many would want to respond to as an outright attack. We have not seen any outright attacks on nuclear powers for obvious reasons. Maybe Stuxnet, but note the goal of that attack was explicitly to prevent Iran from developing nuclear strike capabilities.

By definition you can only respond to a decapitation strike if you haven't been decapitated.

Anyways, I think the nuclear deterrent is an okay argument, I'm not totally convinced by it because I think you could also get right up to the line and even push it without your opponent being crazy enough to push the big red button.

I think we're doing this right now to each other all the time. For example, China is systematically targeting US trade secrets. This seems existentially frightening, but we haven't yet levelled China with nukes.

In fact, if I were a government, I'd secretly have a policy to never push the button, but that's just me.

We put a lot of effort into making sure our nuclear infrastructure isn't vulnerable to a decapitation strike. I have no idea how AI might cripple all subs, for example.

I completely agree that the line is the dangerous part, and it's an unclear target to aim for! I think that's exactly why we don't see major strikes against nuclear powers. Nobody has a public policy stating "we won't nuke you until exactly X, Y, and Z happen" so everyone else is left guessing. This is far from a complete deterrent, which is why we still see probing attacks like Solarwinds, but they do not strike me as decapitation attempts.

What if AI could rapidly develop hypersonic interception technology?

That doesn't seem unreasonable.

That's exactly what I'd expect to happen, but I suspect we may have different definitions of "rapidly" here. Aerospace development requires building and testing stuff, and AI can only help so much there.

Regardless, we're back to MAD doctrines again. How many and how effective are your interceptors? How many nukes do you expect to be launched? What's your tolerance for failure? Margins for error are slim and consequences are rather dire.

I think you've made reasonably good points. My least favorite aspect of discussing AI is the undercurrent that we're monkeys discussing the Sistine chapel. Ultimately, it's difficult to engage with the concept of superior intelligence.

Generally I think the outline that such a device would be a doomsday scenario for competitors remains quite plausible. This is somewhat easy to falsify because we would start seeing direct military action soon.

Taiwan especially is very vulnerable under this worldview and would be a pretty quick falsification.

I agree entirely. I think both our arguments are entirely within the real of possibility, and I don't have the faculty to gain much more confidence either way...

> My least favorite aspect of discussing AI is the undercurrent that we're monkeys discussing the Sistine chapel.

This resonates.

Reply on news.ycombinator.com