Hacker News Reader

Comment by qnleigh | Hacker News Reader

I'm frustrated by articles like this that categorically dismiss the risks of AI in security. If you don't trust OpenAI's and Anthropic's motives, that's fine, you probably shouldn't. But don't tell me that there's nothing to be worried about; we need an alternative proposal.

So let's stop talking past each other and engage with the arguments on both "sides." For example, let's discuss how to ensure competition and availability of open-source models in the long-run while giving the world time to prepare for the immediate security risks of agent swarms.

View on HN

> immediate security risks of agent swarms.

Immediate since 2024

in 2024 they could not break into unsecured all-my-passwords-and-acces-keys.txt on my Desktop

Do you accept that the story / justification from the labs in the popular media and political discussion is simply nonsense? Do you believe that any real security was autonomously bypassed without direction by these models during internal evaluation?

> Do you accept that the story / justification from the labs in the popular media and political discussion is simply nonsense?

No, and I don't see anyone who's actually demonstrated understanding of what happened in the Hugging Face incident (e.g. reading the reports in their entirety) making this claim.

> Do you believe that any real security was autonomously bypassed without direction by these models during internal evaluation?

Yes. Again, this is hard to deny if you've actually read the reports.

If you were building a sandbox for untrusted code, would you give it access to an artifactory instance outside the sandbox?

I don't see how this is relevant. OpenAI absolutely had sloppy security practices here. But that doesn't mean there was no "real security", and it certainly doesn't mean that their models were simply following orders.

I’m just trying to understand the perspective.

I naturally think of this in terms of running untrusted code. If you really think the agent(s) could get out of control that seems obvious. And the approaches to securing a runtime environment in that situation are pretty standard at this point, and they would never allow something like artifactory access.

And if the question is should they get special government dispensation to form an otherwise illegal cartel, it seems much simpler to just follow standards for running untrusted code.

Reply on news.ycombinator.com