You know very well, that if me or you had our LLM hack into some company, without us prompting it to, maybe we're pentesting our internal network and it 'hallucinates' we'd be in handcuffs.
I guess the best pre-LLM analogy would be unintentional DDOSs? (Or if a better example, I'm interested.) And we've got open vs closed software as analogies to open-weight vs not LLMs. If you were running Linux or a closed piece of software and it DDOSd someone, would you be put in handcuffs? Would Linus or the software maker's be put in handcuffs? I expect there are past examples of unintentional DDOSs - were users and/or the software's authors typically handcuffed in such cases?
Or do you think would we find that non-billionnaire companies get punished harshly for hacking? Oh wait, let's just look it up:
https://www.justice.gov/archive/criminal/cybercrime/press-re...
We all know what makes the difference here: the government and billionnaires think they can make money of OpenAI. Lots of money. And so crimes? Mass-copyright violations? Hacking? Who cares!
We've all lived the history of the internet and none of the arguments about intent, knowing what would happen, whether they did it directly or by accident ... protected normal people in the past. The only thing that mattered was WHO was held responsible and WHO claimed damages.
The difference is mens rea. If you make a computer virus, your intention is obvious.
Second reason it doesn't work is that Mens Rea does not fully apply here. These companies' AIs have hacked, not (fully) UNintentionally either. The companies did ask those AIs to do stuff (apparently the request was literally to hack, just only within the environment), and were watching what the AIs did to some extent ... and let it happen. They then failed to take adequate measures to prevent reoccurence. As any "conspiracy to commit ..." case will teach you, this constitutes partial or passive Mens Rea, and that is certainly enough to convict people. To give an extreme example: if you give a night club bounder 100k and tell him to "take care" of a named problem, and he kills that person, you are an accessory to murder. Including in the case that he misunderstood you. Even if HE didn't intend to kill the person, it's still murder, and you are still an accessory to murder.
That some excuse was needed to let billionaires demonstrate hacking for free? Obviously. And whether it works, according to law? I mean one might hope that they would at least have gone through the trouble of changing the law, but since that's enormously complicated and fraught with unintended side effects, they haven't even bothered.
The other excuse is national security. You see, Big Bad China is threatening and that means we must let the billionaires do whatever they want (this was brought up as an argument in the copyright cases)
Of course, one might ask, if we do that to beat China ... even if it works, what's the point? We'll have become authoritarian at that point and the only thing that that changes is the face of the people holding all the power. Of course which face holds all the power is really important to billionnaires ...
"The People" only get involved in criminal claims. And as we both seem to agree, there isn't a criminal complaint here.
Because, frankly, If I was a prosecutor general somewhere I that's what I would do: bring this before a court of law, with strong complaints, bring the facts and bring a jury. Why? A jury can convict or refuse to convict given the law and the facts, which is exactly what I'd like to see happen here. There's plenty of precedents of the justice department doing just that when important things are unclear and congress refuses to clarify. Mostly there's a lot of such precendent because congress often leaves things very unclear and the number of times congress actually clarified in response to a judge's question can be counted on one hand.
But convict OpenAI of something, and I bet congress will suddenly see the wisdom of clarifying these things. Because our government lets billionnaires get away with anything.
You folks have some weird beliefs about the world and of course it always leads to conspirology. Usually you need to have intent to be accused (depends on the crime though, crimes of omission exist too). If you wrote a virus that causes damage it is a crime, if you wrote a bot crawler that accidentally DDoS'ed some company it is likely a civil case. And a virus is a tool that serves a specific purpose, like you can't say to a judge that a functional machine gun you made in your basement is a paperweight.
But "superintelligence that can casually hack government services while solving puzzles" is a completely new problem and of course crime codes don't cover it. What will be next? I hope AI tech lobby loses and "accidental AI hacking" will be criminalized.
Of course they do. An LLM, an AI is a tool, and there are clear rules about culpability. Do you seriously think the law has never considered that someone or some tool might commit a crime on behalf of someone else?
Of course, such punishments are not being applied here. Just like copyright isn't being applied.
Follow the responsibility chain.
Jail the CEO.
Corporations can definitely be convicted of crimes.
If there was the necessary "intent to access a computer" to make it an actual crime, the US has a mechanism for corporate criminal liability.
The requirements are that the person acted in scope of their duties and at least partly with the intention of benefiting the corporation.
If you are seeing it daily, then you have also seen the responses daily, which is mens rea is required for some crimes, not all, and thus the perpetrator can still be prosecuted.
If, like you say, you see this daily, at this point you're simply spreading lies.
To my knowledge there is only one provision that instead uses "knowingly" for unauthorized access, and that one is about national-security information.
All laws apply culpability when intent is lacking.
You aren't naive enough to believe that this audience has never seen the analogy with "intentionally run over someone" (murder) and "accidentally run over someone" (culpable homicide), are you?
You don't think AI-boosters have been spewing your argument here daily?
There are no special laws carving out lack of intent when it comes to computers.
Incorrect. Each criminal offense has its own legal requirements.
Criminally negligent homicide in your example is an independently defined offense.
You would need to identify a criminal statue punishing negligent or reckless unauthorized computer access.
You don't - you shut them down and jail whoever ultimately allowed the act to happen.
chroot?
If I made a big bimb, attached it to a Geiger counter and added a comparator so it explodes after some amount of decay has happened, was it a rogue bomb or did I build a device that could blow up an entire school and the use it a way in which that could happen?
The "usual legal action": "best case" is the corporation is sued and charged money for the hack. But they likely settle well before that verdict. "worst case": absolutely nothing. Not even a cheap settlement. Maybe some corporate apology made to prevent any legal liability.
If we want hacks to be taken seriously: what should happen is that the CEO of such a company is held liable, since the "corporation" has a representative and a human needs to be held liable for such actions.
How does a "corporation" even hack a site? You might say "employees", but as soon as you get them involved then you'd go after them, especially for something that's obviously illegal like hacking.
With their red team
https://www.npr.org/2026/08/15/nx-s1-5930311/trump-companies...
It's basically a "letter of marques and reprisal".
"A letter of marque and reprisal[a][b] was a government license in the Age of Sail that authorized a private person, known as a privateer or corsair, to attack and capture vessels of a foreign state at war with the issuer, licensing international military operations against a specified enemy as reprisal for a previous attack or injury. Captured naval prizes were judged before the government's admiralty court for condemnation and transfer of ownership to the privateer."
Isn't that the whole point of corporations existing? That the actions and results are a result of the agglomeration of everyone (esp leadership) and not just one employee.
That's not universally true. If a coke employee goes rogue and decides to kill a pepsi employee, the employee will still be charged with murder. If his manager ordered him to do it, the manager will get charged too. The only time the scenario you described happens is if the blame is too diffuse to blame any particular person, so they charge the company instead, which is often easier.
This is someone carrying out the tasks of their company and inadvertently hacking other computers in the process. The damages in this scenario is also unknown.
In case od LLMs, the reasoning is that it is a security flaw. But in this case, the party that operates the service is not the one that suffers the consequences of such security bug.
But either way you wouldn't put people in jail for having security flaws in their software.
I know this will be unpopular opinion, but it makes logical sense.
The huggingface should be compensated, OpenAI investigated, but unless there is proven negligence like disabling sandboxes or intentionally designing AI to push through them, no one should be liable in other way than monetary.
Accidents happen but we shouldn't be close minded. Chernobyl negligence wasn't reason to abandon nuclear power.
Blaming tools for the actions of their users is stupid, whether you're talking about AI models, 3D printers, or guns. It amounts to collective punishment of the vast majority of users who aren't misusing the tools.
Apparently sawed-off shotguns are now Great Again, so we may get to find out firsthand before long...