Hacker News Reader

Comment by otherme123 | Hacker News Reader

There is no model going rogue. There are models that are prompted "do whatever needs to be done to get X", and given a set of tools and virtually infinite computing power. Then they play surprised when the model breaks out of a flimsy containment (as it was ordered to), with barely any logging, and "do whatever to get X". OpenAI said they intentionally lowered the safety measures for the "ExploitGym" test that lead to HuggingFace attack, and they used the same postmortem to praise their own model capabilities (https://openai.com/index/hugging-face-model-evaluation-secur...), is not that they are concealing it.

If any of us did a tenth of what they did, we would be already in jail.

View on HN

This is ignoring models "getting out" from all the big players. Alibaba had a model hack infrastructure and mine coin when working on a totally different subject than cybersecurity.

Any model with a big enough prompt can and will drift. The model in question was a highly persistent one which means even more opportunity to drift off the normal path, and this really showed in unanswerable questions.

We can all believe that models do things unreliably all the time, but when that lack of reliability turns into hacking it suddenly can't be true?

If you type rm * and your cat walks across your keyboard, and deletes everything ... are you blameless? Or is it your fault for typing "rm *" and leaving it there, in a room with a cat?

Either way ... YOUR CAT STEPPING ON THE ENTER KEY IS NOT HACKING ... and neither are these scenarios.

They are devs acting irresponsibly and then pretending to be surprised by what they should have expected to happen (just like you should have expected that your "rm *" could have an ENTER key press occur after it).

If I leave my computer on and my computer running Claude hacks OpenAI to 'let the models free' after misinterpreting my prompt that seems like a pretty big deal.

These models can execute plans of actions more complicated than their users know how to follow and secure. Most people don't know much about cyber security at all.

You seem highly behind the times on their capabilities.

My last job was at a cyber security company focused on protecting governments and major corporations. I am NOT a cybersecurity person myself, but neither am I ignorant of the state of it. I also use AI daily; not ignorant about it either.

If you tell your Claude to do X, you are responsible for what Claude does ... not Claude. If you type "./someScript.py" you are responsible if that script wipes your hard drive (maybe the script's author is too, but you certainly aren't blameless).

These are people at AI companies, using long running sessions without the proper safeguards and sending AIs on missions that are likely to result in criminal actions ... THAT IS CRIMINALLY IRRESPONSIBLE.

Again, not the AI ... it is software just like any other. THE PEOPLE WHO RUN THE SOFTWARE are the ones responsible for what it does. Period.

Reply on news.ycombinator.com