Hacker News Reader

Comment by Leynos | Hacker News Reader

Inadequate or broken safeguards do not fall under my definition of reasonable.

OK, but one of the examples in the source article was a report from third-party testing of those safeguards. I feel like you're starting from an unexplored presumption that adequate safeguards must be easy to build and anyone who doesn't build them is just being lazy.

I did not say this shit is easy.

Let’s go over their statement.

> Inadequate or broken safeguards do not fall under my definition of reasonable.

I see no such assumption there. In my reading, the implicit assumption is that adequate safeguards are necessary and that anyone who doesn’t build them is irresponsible. And I feel that that is correct.

I don’t understand how to connect this to the facts at hand. Building adequate safeguards requires studying safeguards to understand whether they’re adequate. But the source article is dunking on one of AISI’s attempts to study such safeguards; we should, the author argues, consider whatever behavior the model exhibits to be an explicit decision by whoever provided the model. It seems impossible to build or study safeguards against model misbehavior without accepting that the models can make decisions.

You start with least privilege. A shared instance of an artifact host is very far from least privilege (I'm comfortable in accepting that OpenAI did not wilfully create this situation, but they were certainly negligent).

And yes, it does indeed seem that a UK government agency ran a controlled experiment where they allowed an agent to instigate phishing attacks against members of the public. Attributing this to Anthropic is wrong. I'm not sure how I feel about my government conducting such attacks on non-consenting individuals, but I'm not comfortable about it in the slightest.

Reply on news.ycombinator.com