All these "safety" bills are to stop open source models by painting them as "dangerous". Regulatory capture.
Companies are responsible for their agents; you don't blame the car for running over the person.
Companies are responsible for their agents; you don't blame the car for running over the person.
I’m still… hopeful… that “good enough” models will be abundantly available to be run anywhere as to limit the low water mark for market capture by a few big ones.
The biggest frustration to me is that investment dollars are all tied up in the shovel company and no one is funding the digging.
Instead, we've seen the opposite: all the big scary stuff has been from their own models (in a few notable cases, also demonstrating that they're actually quite irresponsible in their monitoring and security).
"We need to outlaw single-prop planes, they're a risk to the public: only Boeing has the ability to produce planes safely. How do we know single-prop planes are dangerous? Because Boeing planes keep falling out the sky."
You'll find aviation politics is not too far off. But basically all pilots train single engine first abd America invented the plane.
That is to say, anthropic should be considered responsible, not (or not just) the users.
I'm not a lawyer, but I imagine there's not a lot of precedent or law around this, so it'll be up to the court to decide on the finer details until we do have meaningful laws/regs. Anthropic will probably have better lawyers than the user.
This is why it's important to get ahead of issues like this, and anthropic should not be involved in discussions around regulating itself...
None of that requires trying to impose limits on open models, unlike what Anthropic is aiming for.
I never said we should impose limits on open models.
We have seen these companies repeatedly ship coding harnesses without any attempt at real sandboxing, and encourage people to let them run free. Seems that if they've somehow solved coding, they'd be able to solve making it easy for people to sandbox agents, and to be fully aware when they've enabled some LLM the ability to hack companies. Why don't they put 10,000 agents on that instead of ripping off Navier-Stokes?
I wasn't trying to imply you were advocating to restrict them, just that that is what Anthropic is doing. Legislating away all competition is worth whatever small concessions they would need to make.
Well, you shoehorned it into a completely unrelated conversation, so forgive me.
If anything, their public messaging has been to entirely pretend that open models don't exist, that they're mere toys or curiosities, that only the big dogs with eleven zeros of capex have capabilities which could be considered dangerous.
You could certainly argue that the various "regulate AIs like airplanes" bills might shut down some open model development (domestically; they're certainly not calling for international weapons treaties). But the big firms haven't exactly been jumping backflips in support of Sanders' Ban Artificial Superintelligence Act.
https://www.anthropic.com/research/self-replicating-ai (https://news.ycombinator.com/item?id=50061603)
That's still making the rounds in mainstream news media.
Such laws attach to the human operators of such cars, if they recklessly endanger others by driving them on shared, public roads.
But they're perfectly legal to own (and operate off-road). And, there's no technical barrier stopping people from illegally driving them on public roads: just fear of laws.
(There's entire subcultures of people who modify cars to do insanely dangerous things, on private race tracks. Perfectly legal).
If we were to apply Amodei's safety standard to cars—do they have the theoretical risk of killing over 50 people—well, cars empirically kill tens of thousands of people in the US annually, so we'd have to categorically ban them. That standard is bad-faith and absurd.
(Would we also need a real ID to operate curl(1)?)
Regulations should attach only at the point humans willfully or recklessly cause their agents to commit crimes.
What Amodei's trying to do is outlaw free, open-weight models. That's two distinctions: (1) non-commercial, and (2) abstract, intangible mathematics. Pure speech. Same as Phil Zimmerman and PGP.
Even the EU's draconian open-source regulations (which HN criticizes endlessly) exempt free software which is purely non-commercial.
AI is much more comparable to that than to a car.
You can still run someone over with a car. The analogy doesn't require absolute lack of capability to do harm.
And, more importantly, we certainly don't try to limit the capabilities of computing that might be used to do those things. Given how AIs are very general purpose tools, the fact that they can be used to do something potentially harmful is too narrow a threat.
Nobody is trying to push for cars that automatically stop themselves from running over someone either, which would make for a more comparable analogy than cars with flamethrowers.
If a person closes his eyes and hits the gas pedal on the car and runs over someone, the person is to blame, not the car. AI companies do the same. Your eyes may be closed but you're still operating/hosting the car/AI.
Keep in mind that those open source/weight models are not as powerful as whatever closed source Claude/GPT version exists behind the scenes that hack companies. As per the Hugging Face hack - open weight GLM saved the day as closed source ChatGPT refused to help Hugging Face fix their systems. An example of my point that "sidestepping any third-party checks that could be present" IS good.
Part of point of open source/weight AI is that it allows what those model providers can't: uncensored models for whatever reason you may need it. You are responsible for being legal with it.
Until then, I propose we use the 'unplug the server' rule. Can [insert bad thing] be prevented by unplugging the server? Then these things are fully in our control and no, we don't need to re-build hundreds of years of legal precedent from the ground up.
Liability in the event of harm is already something we have a pretty good handle on from living through the industrial revolution 150 years ago, a time when much more activity that companies did existed in the real world with real harms, as opposed to digital companies doing imagined "mental" harm to soft desk chair people who work at email jobs.
No, only metaphorical AI spinning blades and flamethrowers should be regulated. Actual AIs with spinning blades and flamethrowers will be subsidized and unleashed in your town.
Regulatory capture is certainly something to be wary of, and obviously Anthropic is only going to be in favor of regulation that helps them in some way. But some form of regulation is coming. There is nothing that can stop it, unless you somehow make more money than these guys do and fund a massive counter-lobby. Spoiler alert: the only guys that have more money than Anthropic and OpenAI have all invested in them.
Regardless of anyone's instance on gun control, it's probably wiser to compare AIs with something that's a little more similar to it, and try to reach a conclusion from there.
While the point is well taken, it might be well to consider that according to some of the major rhetoric, the AIs are perhaps (far?) more dangerous than guns.
And so it may be an interesting position to adopt or consider in argumentation.
In the sense of: "ok if we concede the danger for sake of argument, then how shall we treat this?"
My point was that it can't be stopped unless we kill the industry entirely. Once the cat is out of the bag, you can no longer control it.
Ou-la-la. Spicy. But actually we already have those kinds of AI.