Hacker News Reader

Comment by bottlepalm | Hacker News Reader

Absolutely yes, we've all been assuming security in everything we do up to this point. Knowing that the resources to defend were being deployed faster than the resources to attack, we're not afraid of our random stuff getting hacked, and there's time between zero days to patch things before they can be exploited.

AI turns that all its head. SOTA malicious AI can create novel zero days, exploit them, spread, create more zero days, and essential hack all the things in days. So yes in the old world it was possible to assume a high margin of safety, in the new world given all the out of date everything out there and the speed at which anything is patched - assume nothing is secure anymore unless it is not connected to any network whatsoever - or even listening wirelessly - bluetooth/wifi off.

View on HN

> Absolutely yes

That's pretty strong. Recalling our pre-AI condition, there were side-channels built into our silicon, zero-day/zero-click browser+mail+IM attacks, supply chain attacks, industrial scale ransoming, mass credential leaks, commercial exploit brokers... I can't recall a time when safety could be assumed. Before even rudimentary networks were available, floppy disks were spreading viruses.

In fact, most of that is still in play. Certainly AI has reduced the cost to compromise things, but whatever period of time you have in mind where safety was an assumption was so long ago, or applicable to such a narrow subset of our information/communication age, as to be effectively negligible.

Again, it was assumed if you kept up with updates and patches your risk would be low. Unless you were a direct target valuable enough to pour resources into. Which was far and few between, stuxnet for example.

In the face of an agent that can zero day you, nothing you can do is safe, and the cost of attack is super low. This is a completely different world/ballgame that we are not prepared for whatsoever.

Huggingface was an accident, a deliberately malicious AI could be a million times worse and potentially unstoppable if it infects data centers around the world; you have no jurisdiction even to shut it down.

"it was assumed if you kept up with updates and patches your risk would be low"

I don't share that view at all. Updates and patches handled known risks. Updates and patches are often years late: side channel vulnerabilities were dealt with only many years after the problems were endemic. Valuable latent vulnerabilities were and are horded and traded, by entities ranging from ghetto spammers to nation state actors.

I don't argue the AI makes attack cost "super low." It's obvious that this is true. My problem is this notion that there was a time when "safety" could be "assumed." That time did not exist. Perhaps it was possible to pretend such things by people for whom stakes were low, but for many people, the stakes have always been too high to indulge such thinking.

Legacy software in this environment becomes a bigger risk every day it runs.

Don't you see now that everything is legacy in the face of an AI that can zero day anything.

Perfectly secure software doesn't exist and the only thing keeping the house of cards standing was the time it took for determined humans to knock it down. That whole model is being thrown out the window.

Reply on news.ycombinator.com