Hacker News Reader

Comment by anExcitedBeast | Hacker News Reader

But "fully airgapped systems" are more than realistic. The labs are testing the limits of the most capable cyber offense systems that have every existed, WITHOUT cyber protection guardrails.

For that purpose, any cyber operator worth their salt will tell you true physical airgaps are more than merely "realistic"; they are categorically necessary.

View on HN

Cyber operators I agree would tell you that physical airgaps are necessary, and I know they are being used for running agents. But ML engineers training agents? That is far more complicated. At that point sensibly you are talking about physically airgapping an entire datacenter, and when moving around petabytes of training data, model weights, training results, researcher code? The process moves too slow for most companies to tolerate. This is how we got into this mess in the first place. I think the industry woke up to the idea that the security threat model of training ML models has fundamentally been rewritten, and there is some serious non-trivial scrambling to come up with a better workflow.

So, you're right, but also systemically creating a different problem.

We are creating evolutionary systems. You can't just assume rules of a computer system, you get natural scaling laws that we call behaviors emerging from them.

The very first things we do to any model is train them. The ones that want to hack and get caught die, the ones that want to hack but can detect your airgapped network from the real net live. You can never know if you've trained the desire to hack out of it, or if you just locked it behind the right prompt and the knowledge we're watching, but not all the time.

Reply on news.ycombinator.com