Hacker News Reader

Comment by arjie | Hacker News Reader

Fascinating that we’re converging on AI safety and alignment in time across jurisdictions. The cybersecurity approach taken here is interesting because should the model be opened it’s still not dealignable because it just doesn’t know enough.

The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.

The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.

View on HN

if you want to save some $ you can start with abliteration and after few turns switch to normal glm5.3 for example from z.ai subscription or other source. I didn't test it yet with deepseek, but I'm assuming it will will work the same.

those models will follow your request when they already agreed to do this

deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)

Are you sure providers allow you to submit modified assistant turns? Because, if they do, great! Last I heard most of that was neutered, with some providers (ahem anthropic) even signing everything and encrypting the "precious" thinking turns

Most providers of open weight models allow it, it's the closed weight providers that don't

If you use the API, there is nothing stopping you writing both sides of the conversation...

That's would allow distillation . They are impacting legit users' experience and it doesn't even prevent abuse.

These are open weight models, nothing is stopping distillation anyway.

Worth noting the Claude code client it seems has been designed to make it rather difficult to start a conversation with one provider and then continue it with another.

Presumably they don't want you leaking your conversion transcript to a competitor.

> should the model be opened it’s still not dealignable because it just doesn’t know enough

This is the only AI-safe approach - see the OpenAI agent swarm incident where the swarm engaged in offensive cyber because guardrails were not active for their task and the model had clearly been post-trained on that narrow domain.

So the biggest threats and incidents came from the labs themselves.

Yet we keep completely unpunished offences that warrant arrests if committed by humans, and let them dictate who has access to what.

Does this make sense? It is the same as when Stable Diffusion tried to not have "people laying down" in their training data in order to block the generation of sex pictures.

Deliberately keeping all models ignorant doesn't make sense. It won't work (whoever breaks the pact has a massive advantage); it won't be enforceable; it will have downstream impacts on things you don't want it to be ignorant about; this kind of regulatory regime invites adding more and more to the "no-no list".

Is it not possible to focus on hardening? And on new legal liabilities for operators who do security attacks?

The difference is that it’s not subtracting an otherwise present part of training, it’s choosing not to continue to push it with RLVR. Models become capable like this when they are specifically tuned to be good at cyber tasks, so without the RLVR (Reinforcement Learning w/ Verifiable Rewards) they never pick up this level of aptitude.

I agree that it’s not obviously sustainable, but it’s better than specifically training these tools to be good at wreaking havoc, and then putting on a shocked pikachu face when they wreak havoc (ahem, Sam Altman).

There is difference between blocking all content and just making model non-savant in some area.

Deliberately keeping people ignorant doesn't help either

there are legal liabilities for cybercrime already -- they need to be applied

LLMS are tools, not untouchable gods. The AI safe approach would be to throw Sam Altman in jail. It's his company, his tools did the hacking, it presumably violated laws, if so, why is someone not prosecuting OpenAI (this is a rhetorical question; I don't think any of the players with a stake in it want that in court)?

That is a shout out to the Void Star book, https://www.theguardian.com/books/2017/apr/26/void-star-zach...

Reply on news.ycombinator.com