Hacker News Reader

Comment by netflixandkill | Hacker News Reader

The only thing that will address that is a broad international agreement with sanctions and political consequences for aggressor AIs that violate it, just like agreements that mostly keep horrifying biological weapons off the table and mostly kept nuclear weapons programs in check.

"mostly" still leaves a lot of room for bad actors, but these sorts of compromises have worked surprisingly well enough that the failures were notable. Political problems are only going to have political solutions, and almost by definition those are imperfect.

View on HN

Firstly, agreements do not keep biological weapons off the table or nuclear programs in check.

The first is done largely by the great powers agreeing that it's a bad idea from a military standpoint. That is, biological weapons are not stockpiled not because of all the obvious horrible consequences, but because the common wisdom is that they are ineffective weapons.

Keeping nuclear programs in check is mostly done through expense. Everyone who has them has made enough of them to feel safe, and doesn't want to make more because they are really expensive. After states have decided they don't want any more nukes, they sign agreements with each other to make sure no-one else is building a massive stockpile. If any of them wanted a larger stockpile, they would make it.

In general, powerful states can not be limited by agreements. They only ever work as a framework that allows everyone to do what they would have wanted to do anyway, while feeling confident of what everyone else wants to do and how to plan for it. The American obsession of thinking of the UN as world government and international agreements as laws really distorts your view on the world. That is simply not how any of this works.

But regardless of all that, even if broad international agreements would actually work to limit weaponry, it still wouldn't work, because AI is deniable. The initial AI isn't going to replace the pilot, they are going to augment them. And those are developed further until the pilots are no longer needed, and past that. So you can develop all the way to the point where humans have become a detriment, without needing to show your hand. And only deploy completely unmanned platforms once the proverbial shit hits the fan, which is after the point where everyone has stopped caring about international agreements anyway.

> The first is done largely by the great powers agreeing that it's a bad idea from a military standpoint. That is, biological weapons are not stockpiled not because of all the obvious horrible consequences, but because the common wisdom is that they are ineffective weapons.

Can you point to any evidence that this is the case? It doesn't sound plausible to me.

https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...

This is about chemical weapons but a lot of it applies to biological as well. Biological weapons also have the added benefit of possibly becoming a global pandemic and infecting your own military and economic base.

The American obsession of thinking of the UN as world government

I don't know if it's because of Trump, but as far as I can tell Americans don't think of the UN at all anymore.

At all. There have been so many recent events where the UN could have been involved, but wasn't.

E.g. when the USA recently smoked General Soleimani with Hellfire missles, where was the UN?

I watch local and network evening news on a regular basis, but didn't hear anything about this being discussed in the UN.

Perhaps it was debated, but it went unnoticed by about 99% of the US population.

> when the USA recently smoked General Soleimani with Hellfire missles, where was the UN?

This is a good example of the weird attitude where Americans think of the UN as a world government.

UN is a place where diplomats of different countries keep constantly in touch and maintain relations. That's all it is. UN does not have opinions. The member countries have, UN is only a place where they are presented. UN does not do things. The member countries do things, UN is the place where this is organized.

To take your "not perfect" results into a conspiracy theory arena.

Genetic engineering on humans.

Politically there is a ban. But given your "not perfect enforcement" and 2000 billionaires I guess at least one of them has this goal - whether it is considered kooky or not. Someone is trying to engineer their offspring for intelligence/looks/health.

Whether they achieve it remains to be seen and will no doubt come out in history.

> mostly kept nuclear weapons programs in check.

It has completely failed to keep nuclear weapons in check, imo.

Keeping AI technology out of existence has the same problem as keeping nuclear weapons out of existence - the incentives are too great for any nation to have them.

If a nation is armed with nuclear weapons, the only possible check on them is another nation that is armed with nuclear weapons. Thus even in the existing non-proliferation framework, the largest nations (the United States, Russia, China, several European powers) ALL still have nuclear weapons. The reason why is that if any nation was the only nuclear armed nation it would be impossible to defeat with any quantity of conventional force.

The same argument applies for AI. It would be impossible avoid the existence of militarized AI if it provides the same level of force multiplication as nuclear weapons. It would be necessary for any nation to try to gain the same technology as a deterrent against attack by a rival that possesses the technology.

> The reason why is that if any nation was the only nuclear armed nation it would be impossible to defeat with any quantity of conventional force

Really debatable. Bribe enemy generals, drop commando paratroopers right on top of enemy's political leadership, even at 90% attrition rates, bomb vital military infrastructure, like strategic stockpiles, to demoralise the enemy (making them understand that the war will be long, hungry, and ugly,) even if it means sacrificing the entirety of your airforce, and wait until your land force finally reaches enemy territory amid this complete chaos.

Do you have any historical examples of someone accomplishing what you’ve described?

It seems like if what you are arguing is possible, nukes don’t need to be invoked at all. Why wouldn’t you use the same technique against a conventional enemy, if it is possible at all?

Well, all of the above is so far purely hypothetical.

There are exactly 0 countries on record who went on offensive against nuclear states, with intention to invade them.

To execute a strategy like this, one needs to be quite a monster to rationalise such losses in their sane mind.

It is as ruthless to your own soldiers, as it is to the enemy: forcing a stronger enemy into the most ugly, meat grinding form of ground war, with exact intention to exact maximum attrition losses, while negating advantages of high tech weaponry. Sending them a message "All your bombers, and missiles are useless at the bayonet range, you will have to bury us with your bodies"

Even Mao only sent covert cues to Soviets that he is ready to repeat Zhenbaotai, but even he had no real intention to follow up on them. That was a pure theatre.

What you're saying sounds like a strong argument in favor of my original post.

It is insane to go to war against a nuclear armed opponent. Sure, you could kill half your able bodied population of military age and maybe win (what if something goes wrong in your battle plan? Now you're nuked and the opponent is still combat ready), but now any of your neighbors can easily take advantage of your now devastated country.

One country having nuclear weapons means that they can prevent all of what you propose with mere threats. If a few your planes are in position to drop paratroopers or bombs, they can and will threaten to nuke your capital in retaliation the moment the first commando or bomb gets dropped; and the expected result is that you will fold.

You can't bomb much of vital military infrastructure if your own infrastructure - not only airfields, but also population centres - have been nuked. Your bombers can drop a single full payload, after which they will have no bases left where they can return to reload. I mean, we have Cold War studies of all of that - the only way to ensure that bombers really have the ability to bomb enemy infrastructure was to ensure that they are up in the air patrolling, or alternatively scramble to launch within minutes of an alert, since an hour after the start of WW3 you only have a single strike with the airforce that you have in the air.

You can't demoralise the enemy by "making them understand that the war will be long, hungry, and ugly" because the war will end in a single day, after which half of your population will be dead, and the other half in need of food, electricity and supplies. Sure, you will have people willing and able to defend the country, but nobody will be invading; but after being on the receiving end of a nuclear strike your ability to deploy and supply a massed overseas invasion is destroyed, so you're not a meaningful threat anymore, the conflict is over.

Nuclear deterrent is the ultimate defensive weapon because it ensures that any war is unwinnable for the attacker - if the attack fails, then it fails, but if the attack succeeds, there's no good response to an ultimatum to accept ceasefire or have your cities nuked. All your soldiers and commanders know that if they invade and win, then their families and their homeland will die anyway.

> I mean, we have Cold War studies of all of that

And I read much of them, and what I wrote above is a restatement of what I read.

> Your bombers can drop a single full payload, after which they will have no bases left where they can return to reload.

Yes, but even this will likely be enough to bomb biggest C3 centres, bases, stockpiles of food, ammo, and fuel. Just enough to ensure 1.) initial shock, and C3 paralysis, 2.) making sure that the adversary will be running on the leased time too.

Even at the height of cold war, each side had at most 10 strategic level stockpiles. Without them, the huge force would've ran out of supplies from regional bases in under 6 month.

> after being on the receiving end of a nuclear strike your ability to deploy and supply a massed overseas invasion is destroyed,

If you have your entire tank, and mechanised force on the go, and one leg on foreign soil already. It doesn't matter. The enemy will either have to nuke own cities captured by your force, or contest them with conventional force. Similarly, they will have to either set their equally diminished supplier on fire to deny them to invader, or have to contest them.

> All your soldiers and commanders know that if they invade and win, then their families and their homeland will die anyway.

Yes, it requires one to be a madman to accept such "victory." Especially if you already are a somehow major power, as you know that in today's free-for-all world, a third power will swoop in to claim spoils in no time. But think of it as a "counter-MAD," or "asymmetric MAD," or "madman's MAD"

The problem isn't so much an AI involved in military planning so much as an AI raised by the military, similar to how we are more okay with children who grow up to join the military rather than child soldiers (or children raised to be in the military). Defense (defense!) should be a tool taught and employed only after one has developed an understanding and appreciation of what they should be defending.

Frankly, we shouldn't be developing AI outside of institutions with STRONG ties to humanist missions (possibly including religion). AI is not for war, and it's not for "efficient" resource extraction and development; it's for helping humanity to safeguard itself and its potential.

Reply on news.ycombinator.com