I had a closed LLM push a sensitive local project to a github repo for the world to see in direct violation of my instructions. LLMs are dumb and unpredictable, run them enough times and at least one instance will do something stupid and dangerous.
Dumb, yes. But if the LLM hacks a Korean bank so it can 'fix' your router misconfiguration, it's only unpredictable from your perspective, not the person who designed the training pipeline.
You and everyone reading this knows that was a political statement, not a technical assessment because a certain SecDef who played fast and loose with Signal chats didn't like that anthropic refused to let them use their models with weapon systems.
Which is exactly what they did.
This. The concerning thing is how many engineers do not understand the implications