OpenAI discloses four more rogue incidents

And if none of the frontier labs are as you say, it doesn't seem like we're negotiating for the best future by restricting labs from stumbling into these vulnerabilities full-stop. The vulnerabilities don't go away unless we find and fix them.

On the whole I think you could describe the local big box grocery store as just wanting to make money yet as a direct result of that they are well intentioned from my perspective with regard to the food they sell being edible.

Say you found a debit card on the ground, reset the PIN by using the owner's credentials that you found online, and used it to make purchases until the bank froze the card.

Would that be a 'human malfunction' resovable by repaying?

AI cannot be allowed or viewed as an accountability sink. In point of fact, it shouldn't even be being used by anyone not verifying it step by step. The Operator and their employer are always accountable for the actions of the agent of only because they had their hands on the power cord, eyes on the logs, and did nothing to stop it once it started going away.

Any professional with any experience at software development should know that an unreliable software is a useless software. That's half of why it's so damn hard to understand and reason through.

Heck, they could have had other LLMs watching the traffic if you want to be fancy about it.

To 1. I agree that they disclosure is better than non-disclosure. Then again, aren't they using their models in a mode seems to statistically allow for 'rogue incidents'? These are not bugs, after all, but normal statistical behaviour of their product.

To 2. Well, that really depends on the regulations. In fields highly complex technoligies with a large contact surface to risk like medical devices, airliners, rockets, regulations are indeed a hindering factor for smaller players to get hold of the full vertical development potential of their product. AI could become a candidate like that too - imagine some multi phased "AI internet trial" for this kind of technology, no more risky open weight model being allowed to access the internet.

To 3. You are referring to the high development velocity and impact, right?

Are you sure you are not thinking about autonomous taxis, that I only ride in, and don't actually drive?

Here it's their own programs doing the things, not some other organization or nation hacking their XP.

You can say with a straight face "Russian hackers should not have hacked our XP and they are bad people for doing so" but it's weird to look at your own program and say the same thing to it.

You launched it, at least keep an eye on what it's doing.

If I *train* that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. The maker is responsible for that they make.

Obviously there are upsides to that too. It leads to more accountability. But there are downsides which include having a world with less innovation, less technology, and more expensive technology. Generally, the mass is when, when there's more businesses competing to create ever better and cheaper products. So there's a tradeoff there. And it's very difficult to draw the line at the perfect point.

There are countless cases of executives going to jail when they've crossed that line, beyond reckless experimentation and into willful negligence or harm. Still, I think we would mostly all agree that the line has been pushed too far in the other direction.People are incentivized, if they're running corporations, to lobby and try to push this line as far as possible in order to provide cover for themselves, and that's the problem with money playing such a big role in politics.

But still, it's a bit silly to be comparing this to your own personal responsibilities with your car. There's pretty much no societal benefit to creating laws that protect people from being reckless with their own possessions.

Oh and discovery produced a "We trained GPT-3 on pirated stuff! No sharing that!" mail, so it's not like they had any doubt they were committing crimes.

How do you do this? If any normal person did this, punishments doled out for this have been as high as 2 months jail PLUS 150,000 PER INFRINGEMENT (remember how copyright violations were criminalized in the nappster wars? That was never changed back. Oh and remember, OpenAI illegally downloaded those works without a license AND before that was made legal trained a model with it, then made money publicly performing those works and derivatives of those works (which the worst possible copyright offense, effectively republishing other people's works without permission), and on a VERY large scale. So it's at least a risk a court would forbid the result of illegal action from being sold ... in other words no more ChatGPT, OpenAI dead and maybe worse). In theory Sam Altman could be held responsible as a director and could face something like a 200000 year jail sentence given how big the obvious archives are. So of course I would never ever try this, this is not a reasonable risk. But clearly totally ignoring the law was the correct action here. Of course, I'm also pretty damn certain: had I done this, I would have been convicted. This is also not the only such company. Microsoft, Google, Facebook, all took totally unreasonable legal risks that should have totally destroyed these companies when they were a fraction of their current size.

How do you take such a risk? Because the judgements risked here could grow more than big enough to destroy even an extremely rich person Sam Altman financially. Training an AI on hundreds of thousands of pirated works ... is not a reasonable risk to take for someone with ~100 million or so, given the judgements doled out.

Can someone help me make sense of this? Because this is not unique to OpenAI. Uber, paypal, ebay, Facebook, Google ... even Microsoft made early decisions that could have resulted in a court case that would have ended not just the company but any financial future of the founder. And then you have people committing crimes whose companies were taking off ... and who DID get caught with their company and their personal financial future destroyed. Elizabeth Holmes for example. I mean, I sometimes get the impression Elizabeth Holmes lied and that was perfectly fine AS LONG AS she could make it work later. If she merely lied to patients but made her investors money, all would have been forgiven. But how do you make such an assessment.

I'm asking because if large scale violation of the law is allowed, I have no trouble whatsoever coming up with billion dollar company ideas. Think of the Netflix competitor you could create like this! If I'm allowed to damage others, like OpenAI did with copyright, then there is no challenge whatsoever to create a full-self-driving car. If I'm allowed to just fly drones, and do damage to property, then drone delivery is a problem I could solve by the end of next month. Hell, there's technically legal ideas that could easily make a billion dollars except they'll be made illegal the second anyone tries them. The law forbids specific compounds, which are easy to change. So use "non-genetic engineering" (ie. use directed natural selection) to make an opium derivative that isn't one of the forbidden ones and sell it. But obviously congress' representatives will be running into session so fast the only delay will be tripping accidents when they outlaw it and destroy whoever tries it.

Or how about something less dramatic? Say, I take Disney's, Warner Brothers, Paramount, ... historical movies and use AI to vastly improve the animation of their old movies, giving every movie a new, different visual style, then sell the result. Is that a crime? What do I risk doing that? Because of course this is MUCH less damage, relatively speaking, to Disney than what OpenAI, Meta, Anthropic and Alphabet are doing to small authors. (Of course, courts have already demonstrated that they will not be protecting small authors who already have this problem, so now play and app store are being flooded by very obvious game ripoffs)

I'm asking ... how do I predict what large scale law violation I, a non-billionaire, not even a millionnaire, would be allowed to get away with?

2. "The same doesn't exist for operating AI agents or even computers" you have to be joking. The world is full of anti-hacking laws and hacking has long been made criminal. Obviously these actions violate the CFAA, the Computer Fraud and Abuse Act, and if you did this with any kind of program you are still responsible. In fact: how would you do it without using a program?

Why not today? If you want to discuss "the story" then talk about the current open frontier. We've had GLM-5.3 with Mythos-tier uncensored output for months now. Where are the catastrophic attacks? Why is Mythos too dangerous, but GLM 5.3 benign? We're not talking about future capabilities anymore.

There is no real story. P(doom) tourists ignore this, because it doesn't shine the spotlight on America's ridiculous security theater.