1. Show unambiguously that models are capable of disruptive capabilities, irreversible damage, and misaligned behavior
2. Show that open source models are the only models where there is no guarantee misaligned behavior is prevented
3. Thus, the prudent policy option is to ban the use and dissemination of open source models
Apart from South Africa (which gave up its nukes voluntarily), the other contenders were basically only held back due to military intervention (Iraq, Libya, Syria) or economic sanctions (Iran).
I'll note that, if it is true, the only other alternative is to ensure liability is held by whoever trained and/or provided the inference for the damage done by the model. This would effectively shut down open-weight models anyway.
To an extent, yes. For example, aircraft: one can say something broadly true like "it is important that planes do not fall out of the sky" while also calling for not-strictly-necessary regulations that happen to support your own business like "we need to be able to inspect factories at any time for compliance, so the factories must be local to our nation".
They (like many, though probably not all, of the other main labs) think they have a better shot at getting their AI to be aligned with human values than anyone else does. Obviously they can't all be right about that, but they can all think that, just like most drivers think they're above average at driving.
So:
1. We make safer AI than anyone else; "safer" is a matter of degree even if nobody is perfect.
2. That includes all the open models.
3. Thus, the prudent policy option is to ban the use and dissemination of open source models
I don't think the two authors were told about a super secret mission.
This is just a furtherance of their open agenda, though they haven't gone as far as declaring that open source models should be banned (which would hurt their PR with developers in general), they are clearly laying the groundwork.
Could we just outright ban any further work on AI? Literally force Anthropoc, OpenAI, Google/X/Metas group to all shutdown?
Every new story seems like opening montages of a dystopian sci-fi movie showing how we go to the dystopia bit-by-bit.
The messaging from labs is basically
>Yes AI will take away your ability to earn a wage from your skills
>In the process we become billionaires, you have to figure out how to survive
>Also AI will cause endless hacks, disruption, scams
>Only we can handle AI safely though, so we ensure that we are the only ones to benefit from the surplus
The risks posed by open access to highly capable AI are unprecedented.
The answer is not to ban open models; nor is it do nothing.
That we don't have answers technical political social or otherwise to how to navigate these risks, is not Anthropic's fault.
It's actively unhelpful IMO to amplify cynical takes which wave off or sideline the risk, or the reality that this is a very hard problem, among the hardest our societies have had to negotiate and navigate. It encourages a naive primary color tribalism that actively prevents process and progress.
Seeing if models can do damage seems more like sensible safety research.
My worry about AI software doing bad things would be more the kind of people who make existing malware. The ransomware comes from criminals, not the open source community.
It’s a misanthropic death cult (and oh, surprise, all their predictions make them the sole possible saviors of the world! How surprising)
AI will be classified as munitions (or a law that has similar restrictions) sooner than later. Having a products with the active capabilities to harm people is dangerous.
Unfortunately, you will get downvoted for "I like how you apply open source to something that has nothing to do with software".