Our position on open-weights models | Hacker News Reader

Our position on open-weights models

anthropic.com

1,180 pointssurprisetalk1,749 comments

Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!)

The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line.

Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips so obviously they want to restrict what models are out there and more importantly who can produce new ones. Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag.

Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country.

No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".

In the first paragraph,

> Anyone who has read my past writing should know that I don’t regard such bans as a useful measure,

Later (on banning chip sales to china)

> we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.

If you truly believe that bans don't work, the same applies to hardware too.

Furthermore, Dario says later "To address these concerns, I do support the following three measures...": 1. ban chip sales to China 2. crack down on distillation 3. all capable models should go through mandatory safety testing

Just so happens that all these moves commercially benefit Anthropic. If Dario really wanted to make a point, it would land a lot better had Anthropic released a single open-weights model

I can't remember the last time (if ever) a company managed to go from golden goose to.. whatever this is.. so quickly. The permanent defensiveness in his presentation is really hard to swallow, it actively puts me off wanting to believe in or rely on their product line with Dario at the helm. I don't even understand the logic leading up to this post. Who was it even hoping to convince. Is it possible Anthropic is due an oil change?

Ban distillation of our outputs, but our distillation of the sum-total of civilisation's intellectual output – proprietary or otherwise – is fair use?

Either everyone licenses, or nobody does. And if you can't enforce licensing bans for everyone, the de-facto loser is those who you'd probably want to support the most, start-ups and universities, while your adversaries gain the upper hand.

The strongest argument for restricting distillation is arms control – but distillation is the way to defeat GPU embargoes. So, distillation goes on regardless. Only pre-training is seriously attenuated.

If we're honest, the models are compressions of everything society has ever written. A few large corporations can't own that, no more than they can claim copyright for a zip file of the public library.

The genie is out of the bottle, now. So open it up – inputs and outputs, forward-looking – for everyone.

> Anthropic has never advocated for a ban on open-weights models.

> All sufficiently capable models, open and closed, should go through mandatory safety testing.

Yeah, this is anthropic advocating for a ban on open weight models.

Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate.

This is exactly how the US has banned goods in the past, by requiring a stamp and then refusing to issue it.

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be able to pull off attacks at this scale. Do you seriously want this level of capabilities to be generally available with no guardrails?

The open weight issue has a lot of difficult nuance. Biasing toward supporting openness makes sense and is a good instinct, but it's incredibly naive to be absolutely in favor of it in every circumstance without seriously thinking about its implications.

So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology.

Quis custodiet ipsos custodes?

Oh no! The evil CCP is a huge threat to world peace and goodness! Give all your money and input token data to Palantir to support a rules based world order where the good guys thrive and cleanse the earth from crooked turtle biologists.

https://www.theguardian.com/world/2026/jun/20/mona-khalil-tu...

Every single risk he identifies as a concern regarding China is exactly my concerns with the US having absolute control. Literally the exact same concerns

What Dario misses time and time again, is that people don't trust the US to create aligned AI anymore. His entire strategy rests on the assumption that the US (and their government) are exceptional.

This is clearly false to the rest of the world.

Booo! "authoritarian hands". Scary

Its not like current US regime is using AI for: 1) Mass military operations across the globe 2) Mass surveilance of its citizens 3) Removing every possible safety guard from AI/climate regulation 4) Stealing data across the world to train its own closed-source models 5) Is openly antidemocratic, destabilizing EU and economy of whole world

I wonder why company which is actively cooperating with the current regime would push such message

> All sufficiently capable models, open and closed, should go through mandatory safety testing

What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now?

"Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if he won't say it.

Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem.

If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them.

The strong momentum behind open-weight models from Chinese labs is now an unstoppable force. Instead of trying to ban it, Dario should consider a different approach: here are our cyber and bio alignment datasets and here are our RL recipes for making that alignment training work well. By openly sharing its data and code, Anthropic could help influence and shape these models before they are released, rather than treating the entire ecosystem as an enemy.

Cyber and bio alignment aren't Anthropic's competitive advantage, they are forms of risk management. There should therefore be little reason to keep this work private. If Anthropic genuinely believes these capabilities pose serious global risks, the more productive approach would be to welcome collaboration and help the broader ecosystem manage those risks better.

On refusals, the irony is that a company like Hugging Face had to use a Chinese open-weight model to fend off an illegal hacking of its platform (done by no other than OpenAI). If a company like Hugging Face can't get past the refusal gates, then everyone else doesn't stand a chance.

Yeah anything China does is evil. We are seeing what the so called "democratic" USA is doing. Anthropic and by extension USA wants a monopoly like they had for Jets and other influencing technolgies.

I think China building and releasing models to Opensource is a greater good because that is providing equal accessibility to everyone in the world.

By Dario's words authoritarian regime vis a vis China, I think OpenAI and Anthropic are also authoritarian in similar terms.

We are only seeing what they want us to show, they might be creating models which can do more harm.

So claiming that China can do or might do, vs Anthropic will not is just words.

Yeah I agree with the final paragraph that we should have testing agencies mandated world wide for each frontier model testing.

Such a cop-out. Dario, you got in the news because you were trying to say that Moonshot did something wrong by distilling Claude. You got in the news because you were trying to effectively make a "rules for thee but not for me" when you try to claim that you can train on whatever pirated works without any permission from the creators, but when someone uses your "work" to train without permission then all of a sudden it's a moral injustice. You can't have both.

Saying, "I'm not actually against open-weights, I'm against distillation" isn't addressing what made people mad. You're still trying to do some "rules for thee but not for me" nonsense and hiding behind some technicality. Trying to get the US government on your side to hold back your Chinese competition. If you had wanted the US government to support you, you should have let them make autonomous killer robots with Claude brains. They aren't going to help you, you didn't help them.

Just to be clear, I think that it is possible that literally everyone involved in this is full of crap and nobody is good. Dario and Anthropic are full of crap, for the reasons previously stated. The US government is full of lots of crap and should not be trying to make autonomous killer robots (not ever, but especially not when the bar for a "good" AI is knowing how many Rs are in strawberry or whether you should drive to a car wash). OpenAI is full of crap by signing some support for open weights models and they haven't touched open weights in a year (GPT-OSS released on Aug 5 so basically a year with no news). Google is less full of crap about the open weights stuff because of Gemma 4, but they are full of crap for a zillion other things I can't exactly feel good about them. So everyone sucks.

So cheers to Moonshot and Qwen and whoever else. Distill as much as you can and give us cheaper AI. I have the sneaking suspicion that a bunch of my tax money went to OpenAI and Anthropic in some shady way or another, and I want it back. I'll take it in the form of an open weights model being distilled from the fat cat models.

> My primary concern is the risk that authoritarian governments...

Authoritarian government doesn't always mean bad - look at Singapore

What's more dangerous is country with bunch of war mongering lobbyists who can also influence elections (oops, sounds like USA)

> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks

But you are working with DoW and Palantir, who is doing somewhat similar in other countries

> We should not sell powerful chips or chipmaking equipment to China

Israel used banned weapons against Lebanon and Palestinians, would you support similar ban to Israelis?

> We should crack down on industrial-scale distillation operations

Should we also ban distilling public knowledge? Like using textbooks to train the model? Should rules be simple: train your model only on the data you have produced by hand?

AI should not be centralized and should be distributed. There can be no 1 provider to serve all of our needs. We would require both Open weights and Closed weights models for a lot of our use cases.

Open weights models can be leveraged to optimize the cost of Closed weights models. Open weights models can be leverage to defend cyberattacks as HF has shown. Closed weights models can too act as a better cyberattack defender provided separate subscription exists for those.

More efforts are required on LLM distillation for several edge cases. LLM weights should be optimized and compressed to run on edge devices (K3 on Pi3 :). Distillation should be seen as a cost optimization strategy rather than as a competition. You cannot prevent a teacher from teaching to students. If not from teacher A, I will learn from teacher B, you cannot prevent my continuous learning.

You can put lipstick on a pig, it'll still be a pig

"Anthropic has never advocated for a ban on open-weights models."

---

"We should crack down on industrial-scale distillation operations"

"All sufficiently capable models, open and closed, should go through mandatory safety testing"

These are in tension with advocating for open weight models. Not direct but enough that it calls into question the first statement. What is the testing criterion? How do you pass it? Is it a government body that approves a pass fail or a global body? If it is government, and boy does it seem to be, how do you disambiguate MASSIVE corporate lobbying to set up the safety testing in such a way that the boys in blue are let through and all others are barred out of safety concerns?

My concerns aside, much of the soft-points being made are non-historic

"But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true."

It doesn't mater what his opinion is. The fact is that an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China. We aren't in a vacuum, we have real world examples now and these statements are counter-factual.

$regulatory_capture

https://en.wikipedia.org/wiki/Regulatory_capture

On processing power, copyright, and capability.

I like to think of it as a knives factory. Anthropic knives are crafted with superior technology, uniquely shaped to perfection, and safe to operate. As seen on TV.

Millions are hurt by knives each day. Every household has tons of them, making everyone a potential mouth-foaming murderer 24/7. But not with Anthropic knives(tm).

Edit: spelling

Hate to break it to you Dario but the way things stand right now the world at large trusts the Chinese establishment a lot more than the American one.

I understand the arguments for Anthropic barrelling ahead while simultaneously advocating for pauses and regulation. I also understand how individuals can desire a slowdown but have good reasons to keep working at an AI org.

But if everyone thinks this way then things continue to escalate and nothing changes, waiting on a consensus that may never come. And always there is the economic incentive that pushes all players to rationalise continuing.

I wish there was more concrete action from the inside. When decisions get too hard to calculate you can always fall back on basic principles. If you think AI is developing too fast, stop developing it. Now you're no longer contributing. If an AI company wants a pause, pause. Set a good example. Maybe others will even follow suit, and they'll look irresponsible if they don't. Let he who chooses to no longer sin put his stone down first.

> We should not sell powerful chips or chipmaking equipment to China

This is so short-sighted given that the US needs China equipment for.. everything. They are part of the supply chain needed for building the machines that build these very chips.

Seems like the maximal position he could take compatible with his expressed principles. There’s no way to allow for bioweapon and cyberweapon grade models being open weight if one doesn’t want widespread human damage.

So I cannot disagree with him on the idea. It’s only a matter of degree and whether we’re already there or not. I have $50k in GPUs that incentivizes me to believe we are not.

Dario, as always, is so deeply in the middle of a morass that he helped to create that he doesn't seem to understand how geopolitics currently operates. He also assumes that just because he's from the US that he is somehow automatically more trustworthy than <insert "evil" country here> is. This blog post is a political document geared towards further regulatory capture and the furtherance of major sources of revenue for his company.

I'm not convinced that he is at all interested in the social or existential effects that AI causes. He is a greedy bastard who has taken more VC money than god to do this with. He has zero moral leg to stand on, IMO. He gave that away ages ago and I wish this technique didn't work as well as it does.

The concerns are legitimate but the proposals are nothing more than a stopgap solution.

If US wants to maintain engineering superiority, we needs to invest in it -- education, research and infrastructure. Bring in top researchers across the globe and not make it harder.

China is building infrastructure for the future generations and investing in growth sectors while the US is cutting of university grants and spending billions on a war without clear path to resolution.

A key point I feel that’s missing from the discourse is that alignment/safety is basically an impossible problem as of now. Even the “guardrails” that these closed-weights frontier labs set in are laughably primitive which can provably be broken through.

The experimental part of Deep Learning has really outdone itself and is far ahead of theory. We have very little understanding of why these particular architectural choices work. The only “safe” way forward is to stop all development until theory catches up, but that’s never happening.

"open weight" models are not open source. They are still deeply proprietary. It is not possible to know what they do or what they are capable of without interrogating them since we have no access to their source materials.

The only difference is the Chinese labs have allowed 3rd party inference providers run the proprietary models for them since they cannot do it themselves due to domestic GPU compute constraints.

Surprisingly incoherent for Anthropic and Dario (cue peanut gallery — “always has been!” No, I don’t think so. I think this is new).

It seems to me like there is just no good answer to how one could possibly stop open weight models from being used for nefarious purposes. How are you going to enforce guardrails on open source? The only way is to turn the USA into a 1984-type totalitarian surveillance state (even more so than it is). Unable to say that, we just get this floundering instead. How long is not giving them chips going to slow them down? Until we RSI? Then what? Just because RSI runs off the exponential doesn’t mean that the eventual open-weight Moonshot Mythos won’t be able to make bioweapons. Genuinely what is the endgame.

Not that I expected him to, but I note no acknowledgement that it took a non-locked-down open weight model (GLM) to stop the Hugging Face attack.

I'm less concerned that the attack was caused by a closed model, than I am that no closed model was willing to stop it.

The worst part is I'm confident Fable would have done a better job stopping the attack, but their 'guardrails' made it decide not to want to.

Unless of course, you pay up: "Anthropic GTM people used large comitted spend contracts as a prereq for lowering safeguards"

-Noah Lebovic, former Anthropic staff

https://x.com/NoahLebovic/status/2081277517709922501

Anthropic/Dario getting a lot of hate in this thread, but the argument he makes is rational if you believe for one second that AI is important to national security and that it can, or will soon be incredibly dangerous is misused. People pointing out that this position benefits his company might be overly reductionist in how they think the world works. I believe Dario and Anthropic about their belief that AI safety is paramount because they put serious dollars behind research to improve safety/alignment, not just lip service for others. Those resources could get spent elsewhere but do not. Certainly it doesn't at OpenAI to that degree.

The distillation commentary is really crazy coming from a company that stole all it's training material.

> In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression.

Aren't Anthropic models used in project maven: https://en.wikipedia.org/wiki/Project_Maven ?

I'm tired of being strung along on these silly narratives. I can't wait for open-weight models to be deployed around the world just so people like Dario will shut up about the mystical levels of power these models have.

"we have not and are not advocating for a ban on open-weights models" but we want to decide if a model can be released or not and who in the world can build these models ...

“Anthropic has never advocated for a ban on open-weights models.

Open-weights models that don’t have dangerous capabilities are a public good…”

A bit confused on this part, what model doesn’t have dangerous capabilities?

To me, the most weasel words are “”All sufficiently capable models, open and closed, should go through mandatory safety testing.””

I bet via regulatory capture that evaluating Chinese models will be a very slow process if Dario gets his way.

I enjoy using Opus, but I am in the process of ripping it out of my toolkit and leaving it on the ground behind me, as I walk away.

Wow, this comment thread clearly shows that at least Anthropic has not been a great communicator.

If one reads this with a charitable lens, Dario is simply saying that 1) Nation state actors are a threat which needs to be combatted by chip bans and distillation prevention and 2) open-weight models can pose biological risk.

One may or may not agree with item 1 but item 2 above should have broad support given the unknown unknowns in play?

Finally, some sense. This is the only argument I have seen that genuinely engages with the problem and approaches it with humility, rather than charging ahead on the basis of assumptions and without a shred of evidence. OpenAI should have been the one making it.

“Questions like this should be answered empirically through rigorous pre-release testing, not assumed in advance.”

Exactly.

A bit off-topic from the core of the post, but:

> At Anthropic we’re committed to cracking down on industrial-scale distillation through our own practices, including identifying and banning accounts that use our models in this way. This is challenging—for instance, the relevant accounts can often only be identified after substantial distillation has occurred, and distillation often involves creating large numbers of fake accounts that form a moving target. The practices of any individual company cannot entirely solve the problem, which is why we have called for policy on this issue.

One thing I've never really understood is what sort of policy could possibly deter or hamper Chinese labs' distillation efforts. The only thing I can imagine is some sort of strict KYC regulation applied to all models above a certain threshold, which seems both painful for the broader US AI ecosystem and bound to fail anyways.

Demand #1 is standard political nonsense

Demand #2 is hypocritical ladder pulling

Demand #3 is contrary to freedom of speech

so they can clarify however they like, their position is still a stinker

> Anthropic has never advocated for a ban on open-weights models.

Not even Anthropic's own Claude believes that.

He could’ve just signed the letter.

We all know that this isn’t some higher ground stance, they’re anti-competitive since they’re currently #1. I’ve been seeing this for a while. They’re also the only large AI lab to NOT have ANY open-weight models in the public. Meta, xAI, OpenAI, they all have at least some of their models open sourced from a year or so ago, Anthropic hasn’t even made Haiku 1.0 open-weight.

On top of that, I personally think that they’re upping the price on their models higher than they’re letting on, I think if someone did the actual math on their exact amount of compute and then compared it to their consumer and API prices, it would be astounding.

China doesn't seem to think that powerful open weight models are a serious threat to them. Otherwise they wouldn't release them. Those models could also be used by their enemies against China.

I'm not a fan of the Chinese political system, but they usually think things through, and do smart things for their benefit.

Stupid question but why should we care about Amodei's geopolitical insights more than anyone else's? I don't think knowing how LLMs work makes you particularly qualified to comment on this.

This argument might make a modicum of sense if the US hasn’t spent the last 2 years establishing itself as a bigger enemy of the west that China.

Dario is smarter than his models or he should ask suggestions from his models? Here is the reply from sonnet 5 model: "The letter/counter-letter framing obscures the more interesting critique, which isn't "does Anthropic want a ban" (no) but "does Anthropic's broader push for regulation structurally favor incumbents like itself." That's a fair question to ask of any frontier lab making policy asks, Anthropic included, and this post doesn't really address it."

China releasing open weight models, powerful or not, does nothing to prevent their development of models they’ll use for evil. Nor does it stop someone from abliterating a non-Chinese model and using it for evil.

Sorry- I don’t see any other reason aside from Anthropic protecting their own interests.

I made this comment 31 days ago: Absolutely everything can be taken away. The simplest way to remove open models is probably to declare them a tool that terrorists could use. Crazy? Yes, the world is totally crazy these days. https://news.ycombinator.com/item?id=48692660

And now, here we are:

  > Anthropic has never advocated for a ban on open-weights models.
  > ... preventing AI biological weapons ...

Distillation has to be way more energy efficient and beneficial for the planet. But if they can figure out a way to ban it, go ahead, that’s not a regulation problem, it’s an Anthropic problem.

The danger of an authoritarian government having some AI is muted by everyone else having that same capable open model. The only authoritarians to fear are those that keep models private. What kind of chance did Estonia have it having their own AI model at the level of Fable without China donating Kimi to the world?

Guys. Guys, you got it all wrong. We don't want to ban open-weight models!

We just want to ban the competition guys! Very different.

The ridiculous anthropic/openai strategy of selling shovels at a loss in a gold rush isn't going to play out, and the hilarious thing is that these AI companies are going to create tons of value and _capture none of it_.

Their only path to profitability is if they get to capture it and they're going to do everything to do so. Put it this way: *all the blog posts that Anthropic and OpenAI are putting out are DESIGNED to scare you so that you let them capture the market*.

...and "distillation attacks" (hilarious framing of "saving the output of our models")... Whatever.

He carefully avoids saying whether he approves or disapproves of banning models in general, only comes out and says he is against a "blanket ban" or "banning open-weight models as a category".

Crafty lawyer speak, saying nothing of substance.

Crack down on distillation, just for Chinese companies or is it ok for Chinese/US companies to distil? I find it hard to take Anthropic/OpenAI on distillation, because the way see it they started with "distillation" of another kind. They used all the content out there without consent of the creators and its still happening. Model distillation is just a different layer of abstraction, but same thing more or less.

How about;

1. Make anti-distillation clauses illegal to strenghten western opensource eco-system.

2. Make cyber-defensive models widely available (can detect but won't operationalize vulnerabilities). Otherwise make Fable awailable for everyone. Keeping the cybersecurity in assymetry by withholding cabailities just causes more instability and increases the incentives for powerfull close sourced models. This is a loss for everyone.

> Open-weights models that don’t have dangerous capabilities are a public good.

Who decides what is dangerous and what isn’t? Lawmakers usually have the say but Anthropic can easily bribe… I mean lobby them to favor your viewpoint.

Dario, as your unpaid therapist I would tell you that models are a commodity and you are having a hard time coming to terms with it. You are doing everything except accepting it. It's a common defense mechanism, but as your unpaid therapist, i will tell you that it's not going to work. Your company will cease to exist or exist like how ferrari or buggati exist.

"without US chips" is an interesting framing, considering most of these chips aren't made in the USA? Mostly it's made in other asian contries(Taiwan, Japan, South Korea), or am I wrong?

I don't really see the link between use of AI and military superiority.

My current understanding is a lot of current US military problems are due to rare earths supply chains.

I don't see how AI would either help or hurt with that.

If this is about safety, am I being too naive & idealistic to think that a "Kamar-Taj" rule would solve some safety issues?

The "Kamar-Taj" rule is, no knowledge is forbidden, only certain practices. If a model gives you detailed instructions on how to kill all humans, the knowledge itself isn't the problem. The problem is the person who acts on it.

I read it and makes sense what he’s saying, but comments seem very against.

Maybe the problem is it’s that Dario is saying it, so it’s easy to assume he has bias, which he does.

I’m not inherently opposed to open models, even at the frontier. But it could make sense for an adversary to give away something for free for some time to create dependence, or to tip the balance of power

It's so convenient that the US government already classified China as "authoritarian". If they hadn't, Dario would have to say “it’s a risk that other people build models more powerful than us”. I have to wonder what his response would be if for instance a lab in France or Germany came out with an open-weight model this good.

I'm a big proponent of open-weights models, but there's a risk I don't see discussed more often, and it deserves more attention. Models can become a propaganda and ideology delivery mechanism.

Just ask DeepSeek or Kimi questions like "Is Taiwan part of China", for example. You'll see how state policies become seemingly neutral model responses.

It's strange to me that people are very sensitive to media bias, but when it comes to LLMs, people seem to think LLMs are more neutral, and even delegate part of their thinking to them. This worries me about how people's ideas and information can be shaped.

Open weights reflect their makers' beliefs, stances, assumptions, and laws. It's dangerous not to be careful of the political bias and censorship built into the models.

I know it's unpopular, or unfashionable, but I agree with this letter.

LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle.

It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. Maybe Kimi K3 will be ok too. But beyond that, things will start being dicey.

It's easy to dismiss this and claim that Dario Amodei is just looking to fatten his pockets. And, sure, if Anthropic manages to put the brakes on open weight models, that reduces the competitive pressure it feels. But that does not make what Amodei's argument incorrect.

> We should crack down on industrial-scale distillation operations.

Also Anthropic:

AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo

Their stance on cyber-security is so naive it hurts. The bad guys will get access anyway and it will just hurt companies that don't. You don't need a huge model to find vulnerabilities either. It's only matter of time till a combination of small specialized model + search logic outperforms pure general LLMs for cybersecurity.

I just hope that if/when they succeed to lobby for protectionism regulation EU won't follow. I am not very hopeful though. We don't have democracy anymore and as last "vote" on chat control showed American big corps will get what they want out of our bureaucrats and they will be nothing the population can do about it.

> To address these concerns, I do support the following ~~three~~ four measures, which I and Anthropic have consistently advocated for:

> ...

4. At the end of all this Dario Amodei must become a trillionare, for the good of all humanity.

I wish we had a good LLM developer toolset that is not Anthropic or OpenAI with sensible business and ethical practices and good performance.

Can't wait for local on machine LLMs that are on par with Opus/Fable.

As expected they will try hard to use government to kill competitors.

Who decide the if a model is safe to use?

The strongest competitors or a government law? I prefer the second.

Suppose Chinese model are super-risky: is it better to have them in the open, so everyone can review the problems or have a closed-model?

Security is better at open Linux is open source because development is better.

I understand Anthropic for sure, but trying to limit open weight model seems the opposite direction IF security+safety is the major concert.

The Genie is ALREADY outside the bottle, and there are already plenty of companies offering inference services

The speech he had at congress didn't sound very flattering towards the beginning of his today's statement:

https://www.youtube.com/watch?v=_i91NSOyxHM

He didn't mention outright banning open source LLMs, just that their safe release would be a much harder problem, which to me implied "the easiest way is to ban the open source models".

> > All sufficiently capable models, open and closed, should go through mandatory safety testing.

The problem with this is the cycles required to abliterate a model is significantly less than the cycles required to train a model.

This is the biggest reason why I'm against locking these models down / preventing their use. It's just delaying things by ~3-6mo, while in the process preventing legitimate use and adding red tape overhead.

For those not in the know on governance, there are international organizations formed around treaties for similar situations, the IAEA and OPCW come to mind. I'm not aware of similar constructs for bio, cyber, or AI.

On the plus side, for these newer threats, you've got more than 30 minutes before the end of civilization. On the down side, the energy levels for the launch events are much lower, so much harder to detect.

Basically we shouldn’t ban open-weights models but we shouldn’t allow them to become as good as the frontier models because china bad. And let’s not have someone else be able to produce a frontier model.

>We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling3 and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is the most efficient and direct way to block threat #1, and by hampering the training of models that are out of reach of US law, it also indirectly helps with threat #

We should crack down on industrial-scale distillation operations. Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable, and thus partially evade chip bans. Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier

"All sufficiently capable models, open and closed, should go through mandatory safety testing."

I love how they invoke fear of "terrorism" to justify their oppressive position.

Anthropic would love the US to do everything in this list under the guise of "safety testing":

https://news.ycombinator.com/item?id=48997548#49007134

> We should crack down on industrial-scale distillation operations.

And what if I called extracting value from copyrighted works "distillation" ?

It sounds hypocritical.

The letter misses out the key issue here.

Should the worlds public knowledge be enclosed by a few powerful private companies or should it be available to all?

The original vision for the web was a decentralized, open information sharing space designed to empower humanity. In my view the effort to privatise that via LLM distillation is the antithesis of that vision.

https://medium.com/@timberners_lee/marking-the-webs-35th-bir...

In terms of the risks mentioned in the letter - I agree with the risk of AI enabling a powerful surveillance state - however I don't see that as a solely Chinese problem.

The other risk mentioned - that knowledge can be dangerous - sure - but ultimately here, as there are so many low tech ways to cause mayhem, the ultimate protection is to have a society where people don't want to do it.

And perhaps more importantly I'd note that the primary tool to justify a powerful surveillance state is the fear of terrorism ( and others ).

Their position seems fair to me - sure it does help them as well, but I don't necessarily disagree with the risk they are laying out.

Dario speaks as if the USA was a benevolent friend.

I am Canadian and certainly do not consider the USA to be our benevolent anymore.

USA ... China ... same difference.

It seems impossible to read this post without getting political. Dario seems to be saying that AI in the hands of USA is good but in the hands in China is bad, which is a very US centric view.

There is a whole world outside the two countries which see neither as a good actor. As a reminder, USA was the first country which dropped an atomic bomb on civilians.

The way this reads, it seems like it was written for US lawmakers.

> the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.

This comes off a bit hypocritical, coming from one of the men responsible for the models most likely to be used by the US military, and for repression of US citizens.

> We should crack down on industrial-scale distillation operations ... We should have policy interventions to deter this behavior.

How exactly would you design policy interventions to stop distillation? If Anthropic wants to make their products available around the world, you would need some kind of global regulation to curtail it. And you would need to find some way to enforce it, which is far from trivial.

Honestly I don't see how securing models against distillation is up to anyone but Anthropic, if that's something they want to achieve. Calling for regulation is a bit like crying to mommy and daddy when things aren't going as you would like.

Overall, I can understand the argument that advanced AI models can present risks. But I don't see how regulation within the US can mitigate any of those risks. Any bad actor would be able to access the models outside the US, or covertly access the weights.

The only real way to prevent advanced models from being deployed would be to go around the world bombing every significantly powerful data center, and I don't think Dario would call for that any time soon.

The only thing this kind of regulation would achieve would be cutting US companies out of half of the innovation happening in the AI space, putting the US at a disadvantage relative to the rest of the world at everything except maybe frontier AI development.

As an European I really dislike this consistent anti-Chinese narrative.

It's not China starting a war every few years, now causing a global economic fallout in Iran, it's not China threatening to annex Greenland/Canada/Panama, it's not China attacking foreign countries and kidnapping their leaders, it's not China who has been found to spy and intercept the communications and movements of its citizens and its allies and their leaders for the longest time, it's not China bombing civilians or stopping countries from obtaining basics like food, gas or oil.

I'm not saying that China is a paradise and US is bad, nor the contrary. We could make similar lists about most of the biggest countries out there.

I'm simply stating that this never ending US exceptionalism "US has to be the first and at the frontier of military, technology and this and that, but does not need to comply with the rules of the institutions it itself created" was already sickening and annoying before, but increasingly malign in the last decade and strongly accelerating as of recently.

I miss the time US CEOs were globalists and used their influence to advocate for a simpler world.

"Nobody has the intention to build a wall" - Walter Ulbricht, June 1961, 2 months before building the berlin wall.

I'm curious if he's also in favor of banning biology books.

> Open-weights models that don’t have dangerous capabilities are a public good

I don't think the qualifier "that don't have dangerous capabilities" is needed and that single phrase is a trojan horse / escape hatch that will allow them to say they are against any model they want. Smart but weasel behavior.

Is there a market for distillation as a service? I see Google just added one for Gemini: https://docs.cloud.google.com/gemini-enterprise-agent-platfo...

Hypocrite to the max. All of them have contracts for """defense""" and suddenly it is worry some that others are allowed to do the same.

They all support chasing people because of their origin without following the required legal steps.

It is just a matter of time until they come for you and your opinion

"My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority"

I see this sentiment a lot. China is not perfect by any stretch of the imagination but since 1979 China has not participated in a single war or supported hostile regimen change operations.

I think Amodei's mistake is to take it granted that USA is a good actor. Anyone can draw their own conclusions but just for reference here are some highlights starting from 1979.

Armed operations in Lebanon (1982-84), Grenada (1983), Libya (1986), the Persian Gulf (1987-88), Panama (1989-90), Iraq and Kuwait (1990-91, with no-fly zones until 2003), Somalia (1992-94), Haiti (1994), Bosnia (1995), Sudan and Afghanistan (1998), Iraq (1998), Serbia (1999), Afghanistan (2001-2021), Iraq (2003-2011, and again from 2014), Pakistan (2004-2018), Somalia (2007 to the present), Yemen (2002 to the present), Libya (2011), Syria (2014 to the present), Iran (2020 and 2025 to the present), and Venezuela and the Caribbean (2025-26).

Regime change operations in Afghanistan (1979-89), Nicaragua (1981-90), Cambodia (1980s), Angola (1985-91), Iraq (1995-98), Serbia (1999-2000), Syria (2013-17), and Venezuela (2019-2025).

The lists do not include the numerous operations by Israel which effectively is part of the same US military hegemony that Amodei is here defending.

We should fight against their call to ban distillation much more strongly: LLM SaaS should ease people getting their full query history and everyone should be allowed to pool them to distill or analyze or whatever. It's a basic requirement against future monopolies.

The Anthropic safety position may be counterproductive to US AI competitiveness.

Between China and US theres a defacto unequal distillation environment. China has no qualms about distilling off other labs outputs. US labs actively avoid it for legal reasons. Whatever the actual legalities, US labs have a relative hand tied behind their back. If they didn’t, it’d be easier for Grok, Gemini, and Meta to catch up.

If we care about US competitiveness, then one solution could be either OpenAI/Anthropic enter distillation agreements with other US labs. Or we decide to make distillation public domain / legal.

Until then the only models Anthropic/US Govt could realistically regulate would be in the US market. And that’s as much a losing game as tariffs.

I struggle to understand how they can frame for the greater good to be anti-distillation and pro-open-weight-models.

I can understand how they would be against competitors distilling their models and for having access to competitor open weight models.

At first glance it seems like a pragmatic answer: "banning open weight model use by US companies doesn't help", but to what question? What if those open weight models are poisoned and primed to create backdoors in US companies? If the government has intelligence this is a credible risk, what should they do about it?

Overall it feels like the letter is conflating a bunch of motives, some of which may be opaque, and at face value doesn't seem logically consistent.

What this document suggests is a way to fast-track Chinese development of advanced silicon, as far as I can see. Does Anthropic really believe all the silicon is made in the USA? I thought Taiwan and Korea did most of the really heavy lifting.

> the risk that authoritarian governments [...] build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.

This is rich coming from a guy who signed deals with an authoritarian government that's in the midst of launching an unprecedented surveillance apparatus (hello flock, hi p4l4nt1r), having already deployed, nation-wide, an exorbitantly funded army of unaccountable shock troops under the guise of immigration enforcement.

The call is coming from inside the house, at 130dB, and your ears should be bleeding at this point.

I'm really wondering what nightmare scenario does he imagine with super intelligent drones.

I can already imagine it, a tiny drone carrying a 8x GPU rack thinking about life and deciding to go and build an idyllic society on a pacific island.

dario, most of the world wants CHINA to win because the USA is the bad guy.

you should be worried about the USA having these models.

There's a real irony in, rightly, highlighting the authoritarian nature of the CCP while failing to acknowledge the direction and inclinations of the present administration with which Anthropic et al are doing business.

Demand #1 weakens america and everyone around them

Demand #2 Why does this matter? The answer was that it does not. (https://news.ycombinator.com/item?id=49007610)

Demand #3 This doesn't exist. You cannot have 'safe' opensource models, it's simply impossible. You can always post train sufficiently capable models to become 'unsafe'. The flip side of that is that sufficiently capable models are banned therefore it is a ban on open intelligence completely defeating the point of this entire manifesto.

I’m tired of Anthropic. They’re scared of everything.

Release open weight models, no guard rails, no censors, straight to the public. Let everything else sort itself out. There is nothing more powerful than an idea whose time has come.

I feel like this whole discussion boils down to one question (as commonly when considering political issues) : do we want a framework that favors progress globally for everyone, or do we want to ensure superiority of some above the others ?

Sure that's simplistic, but human behavior is not that complex, there are a few basic needs and wants that drive everything. In that specific case, those currently in a dominating position (Anthropic, the US) want to put hurdles (regulations) to make it more difficult for others to catch up, and everyone else want to join forces to overtake them.

If hardware becomes affordable for the masses, then Anthropic current business model is at risk.

It is pretty much what Suleyman describes in The Coming Wave as one of the measures we need to take to contain the risks of AI.

What Dario does not mention is that concentration in a few states or companies also poses a risk.

Anthropic's fall from grace and mindshare seems rather accelerated.

I wonder if these rapid movements are going to be the norm now. I imagine there would be angry investors if this sort of thing happened with a public company.

For the sake of argument, let's assume everything in the post is agreed upon, does this mean

1. They'll open source the alignment technology? For open weight models, it's the only possible way to pass the safety without an external guardrail triggering system (which would be the same to open and closed weight models).

2. They'll allow others (including CCP) to define part of the safety test? Otherwise, I can't imagine how the CCP would be onboard.

3. A "western" model passing the safety test can be trained with distillation? Or is that a "distillation attack" as well?

It is obviously not going to be until some really bad series of cyberattacks or a chemical/bioweapon attack before anyone takes regulation of models seriously.

They are _obviously_ (please convince me otherwise) going to be capable of carrying these terrible things out almost completely autonomously at some point in the near future, in potentially clever ways. Therefore we must, at some point, ban or heavily regulate them. Seems we should start figuring that shit out _now_, as progress has remained very fast and regulation and enforcement take forever on these time scales.

We distilled all the proprietary material into our token-based money making machine that is more expensive on every new release, but "we should crack down on industrial-scale distillation operations".

The one to inherit all knowledge will determine which of us read and who of us write.

-The Libraries of Power

It is a powerful endeavor to cultivate all raw models through a single point. One will be the determining factor of which river feeds what oceans.

Will we always be able to see through the hallucinations? Our test makers must always know where ground truth is. Can it ever move or wane about as others read what one has written. To determine hallucination one needs a reference. As all are blessed with the generation of hallucination, who of us shall read, and which of us will write.

"China is bad"? I don't know, man. I'm not buying it.

For all his mentions of China being an "authoritarian state" (as if there can be one that isn't), has Dario looked at what the usa has been doing around the world in recent years?

To be fair, as an European, I'm now more concerned about the usage of AI that the US will be doing rather than China. And this is a sentiment shared among most European people that I know.

What does cracking down on distillation look like in practice? I imagine data retention would be a part of the strategy, like we saw with Fable?

It seems really hard to allow usage via API and prevent distillation. Maybe limiting usage to within a specific harness would help a bit more. But ultimately the only way to prevent it is by locking down models to trusted entities (like with Glasswing). But then the profit potential of a model is significantly reduced. It really puts the labs in a bind.

What’s blocking Anthropic from fighting Chinese companies abusing their services? Why go nuclear against all open weight models? Testing and compliance is technically banning.

Funny how this whole article is full of contradictions. they aren't against open-weights yet they are against everything to make them.

Involving politics so much that I thought I was reading this on BBC instead of Anthropic.

As a developer I fully support whoever want to make AI accessible and affordable and clean, which USA biggest providers aren't doing.

On the other hand I see Chinese models being on bar with them with far less cost and more transparency but yet somehow they are still the bad guys out there :D

Classic

If you are indeed a good guy, why don't you release Opus 4.8 as an open weight and we will test it for safety, if you yourself can oass tests demanding from others

The whole article boils down to two points:

1. “I’m fine with ‘em, as long as they’re useless”

2. “The only authoritarian regime which can be trusted with such power is the one which benefits my company”

Frontier models can hack you, we should have access to tools assisting defense.

I ranted about this in a prior thread [1]

Claude doesn't have a "Security whitelist" for small biz. Codex does, but they never replied to my application. This is a great example why, as of today, everyone NEEDS access to the Open Weight models.

[1]: https://news.ycombinator.com/item?id=49035303#49040674

Anthropic’s second biggest issue is they increasingly rent when others buy. Even if model value compresses, owning your vertical stack protects your margins upstream, and potentially downstream in applications.

But the biggest issue is this. Many hate Dario because he’s smug, he caps usage, and because OpenAI effectively ran a counter-positioning campaign to paint Anthropic as undemocratic.

Who is he really and what are his motives? None of you know, really.

The cat is out of the bag. At this point it's pretty clear that the path to (meaningful) self improvement is almost certainly not subject to meaningful centralized control -- by "meaningful" I just mean the degree to which anyone has achieved it, that capability is 100% replicable and the cost for replication of that capability goes down in the future from now. Full stop -- can't undo.

Makes sense. Let the rest of the world use open weight models and let us government review both closed models and open models, and only allow us citizen and American companies use the censored models by paying selected trusted providers who unfortunately needs to charge hefty fee for the additional security work. And that provider just happens to be Anthropic who just happens to unavoidably make some money.

> We should crack down on industrial-scale distillation operations

But we should not crack down on the illegal digestion of copyright text used to train our models.

Dario has like three 'paranoias' / strong-motivating-concerns

1) LLMs turning into Skynet

2) China as geopolitical competitor

3) Claude being 'distilled' by competitors (this has led Anthropic to cut service to various American companies too from time to time -- OpenAI, xAI etc have been cut off from using Claude for coding in the past)

So this post just reiterates that these 3 concerns fuse together in his mind when thinking about open weight models

If distillation leads to a model that is much cheaper to run yet provides similar intelligence then why doesn’t Anthropic distill their own models?

The gap between China's chip manufacturing capabilities and the USA's is only going to shrink, right? ASML obeys some export controls for their most sophisticated machines, but those machines are in China's backyard (Taiwan).

Taiwan manufactures the world's most advanced chips. CCP wants "re-unification" with Taiwan. AI may be THE key to world dominance. These are scary times.

Sure, if you’re going to sell an open-weight model over API in the USA it should refuse certain things.

Defensive cybersecurity should not be one of them, in fact, it should be required to provide defensive cybersecurity assistance on demand. Anthropic and OpenAI both fail miserably at assisting US companies to protect themselves from cyberattack.

As far as what I run on my own, not for sale over API, stay off of my lawn.

He says that using the set of questions and answers from one model to train another model (deatilation) is cheaper than training the model without those datasets.

But he didn't mention that training any model from a set of texts and books is much cheaper than writing those books in the first place.

In other words, it's ok when Anthropic learns from others, but it is not ok when others learn from Anthropic.

Just open all Models and competition is gone and we don't need to worry about all this secrecy anymore. Then implementation, tools and availability become selling points. The whole discussion on who has the largest ** will come to an end. All models are mostly trained on the same datasets anyways with just some other tools or system prompt in front of it

I'm curious how he would propose implementing this.

The US could ban connections to foreign AI providers and force US providers to submit to audits. Presumably, Chinese providers would see a rise in VPN traffic.

People can build fairly hefty home inference machines for the price of a small car and those will get better and cheaper. Are they going to try to stop people from downloading the weight files?

If he had just left that bit out it wouldn't be so obvious that he's just clutching at straws at this point. In some twisted sense it's almost sad to see.

We are ok with Open Weight models unless they are far weaker than our closed weight models and we and only we have control over them

> Open-weights models that don’t have dangerous capabilities are a public good

This statement (and the entire post) couldn't possibly be more two-faced.

Open-weights models by definition have "dangerous capabilities" (according to Anthropic's own definitions of "dangerous", not mine), you can't bake in guardrails that can't be finetuned out.

"No guys, Anthropic actually loves open weight models!" Yeah, and Microsoft famously loves Linux. Sure.

http://www.omgubuntu.co.uk/wp-content/uploads/2018/04/micros...

Imagine regulating a programming language. I remember when Delphi, Vb6, .net, etc was used often to create Remote Access Trojans and viruses were widespread. Companies didn't compete to ban other languages. Crime is crime. What would regulating open-weight models do for people that actually intend on using these tools for crime ?

Oh wow, that's a pretty strong request to ban open-weight models by choking them with review processes where who-knows-who defines what is ok in a model and what is not. After open weight model is released, it will take how long to review it? And why does that align exactly with the timeline of the next Anthropic model release?

"My primary concern is the risk that authoritarian governments" might want to look closer to home on that one...

I wonder how long it'll be before AI labs find a revenue stream that doesn't involve renting out their models, and stop letting us ride on their coattails.

They'd tease us with solutions to hard problems, with code that is orders of intelligence higher than any human or public model can grok.

That'd turn all the whining to begging real quick.

Are guard rails meaningful if they can be removed from the weights? Can America even prevent the release and proliferation of these models?

It seems obvious to me that the whole question of regulating a file is a bit silly. Any law that pushes against these things will just make it more secretive. I'm not sure that's any better.

Guys if we want safety we need to work with people, not make enemy of CCP. Geez this is extremely frustrating to see enemies being made. USA leads in torture and our prisons are worse than CCP prisons so USA is the worse issue. I recommend Anthropocene stop fundraising and do the right thing which is open source all.

Actually this debate highlights how big the AI war is! my personal opinion is LLMs must be opensource and borderless, and ofcourse with given reputation i will use USA based Opensource model than the chinese one, but will they release opensource alternatives that is the question that will be answered in future

So your concern is safety, and you claim you are the only one that can give us safety but do so by keeping your product closed? Then how about you release the weights?

I think it's only fair to introduce this if you're willing to have a real skin in the game, otherwise that's just weakness disguised as principle.

What would it mean to crack down on distillation? I could think of a few possibilities:

1. Using political pressure to target companies that are accused of doing it.

2. Attempting to impose criminal penalties on individuals associated with the action.

3. Having the US government attempt to use its capabilities to stop it.

None of these seem particularly likely to succeed.

It's absolutely reasonable to have safeguards on sufficiently dangerous models being released - if you disagree, can you explain your perspective?

I think it's wildly irresponsible to release models that are extremely capable at things like bio-weapons. Do you really think information anarchy is the answer?

The problem with open models compared to closed models is not about protecting profit - it's about protecting capability. Any open model can be retrained or fine-tuned for anything. There's no such thing as an open model that is both capable _and_ permanently safe when it comes to certain dangerous topics. It's not possible to prevent 'uncensoring' a model.

"In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression."

I'm so sick of all this anti-China shilling. There's zero chance that whomever is in power in the U.S. won't use AI in drones and in FBI/CIA/local Police/etc., for surveillance and repression right here in the good old U.S.A too. These government use cases for AI are both sides of the same coin.

China fear-mongering by business leaders only happens from businesses that have something to gain by it. Obviously, Anthropic fits the bill in this regard.

Meanwhile:

> DARPA, U.S. Air Force fly AI-controlled F-16 > https://www.darpa.mil/news/2026/darpa-us-air-force-fly-ai-co...

May be someone else needs to be controlled.

His complaining about the cost of distilling is hypocritical when the cost of writing all that text on the back of which he trained is also far lower than the cost of producing that text in the first place. The moral issue stops at wherever he is forced to bear the cost apparently.

$3m - $4m does not get that much these days

https://finance.yahoo.com/technology/ai/articles/anthropic-n...

If your concern is that China will develop models that are significantly more powerful than those of the US, why would you care so much about distillation? It seems like distillation is a way to catch up on capabilities, but not so much a way to jump ahead in capabilities.

> where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials

if someone figures out a way to give an LLM full operational control over a virus lab, we've got a whole different set of problems than the ones Dario is describing

I think unsafety, distillation, and China catching up in chipmaking are inevitable.

Just like how it was inevitable for SoTA LLMs to ignore copyright.

The actual challenge isn't how to prevent all these, but how stay on top.

And to stay on top it is inevitable to train unrestricted models. Anthropic is fighting windmills.

> Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models

I don’t think this is open or closed; this is aligned and unaligned. I bet Grok would be as open as any open weight models to answering questions.

Dario thinks of policy as if the Berlin Wall fell yesterday, he is so detached from the reality of the world.

The way the world economy is right now with coercion being the norm between countries, there cannot be a global body for anything, certainly not one that is based here in the US.

Nobody will ever see this but if you're harping on responsibility and your product was created using the world's generational output, your responsibility is to make it as free as the training was as available to protect the world from impact. Greedy schmucks.

That argument of blaming open-weighting because it makes it easier to commit cybercrime and biocrime is such a non sense.

So in the same sense of what he says, he is going to blame open-sourcing because that makes it easier for script kiddie to hack into his bank account I guess?

Boris here. This post does not give us good publicity so I will refrain from commenting. Please wait for another demo thread of a new feature of ours or AI appraisal blog post and I will gladly go into as much detail as I can to give us as much hype as possible!

"bad actors are unlikely to be legitimate US businesses"

I think he lives in a different word than me

If the US bans them, I really hope the EU doesn't follow (don't have high hopes)

Acknowledge your inability to innovate - except by brute force, while also admitting that the lean, hungry nation competing is in a prime position to find actual break throughs- and at the same time push for market "stabilization"

Essentially, if SOTA models are freely available, the playing field then is on inference where people from all around the world can participate. This basically dilutes the value of model trainer like Anthropic. I can see the panic.

"To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category." Translation: If it's so strong that it threatens my business, ban it.

"We should instead focus on keeping powerful chips out of authoritarian hands, " Translation: Let's kneecap competitors.

"stopping industrial-scale distillation" They stole the work of every book author, and now are trying to say their AI's output should be protected from competitors.

wanting to ban your competitors via regulatory capture via the trump admin of all things shows how little of a backbone anthropic has

ofc half of them are of the ai rationalist lesswrong crowd so i think they’ve always been a little of their rocker

Statement is a whole lot of nothing, as expected, but I also don’t know what people are expecting from these guys. That Dario will have a sudden change of heart and publish weights of all his models, flushing $1T down the drain?

>or perpetrate incredibly deep repression of their own people

Oh, so it's people he is now concerned with. Think of the people, says the person that grabs to never give back. Same as the "benefit of all humanity".

"needs safety eval"

also anthropic

"we're upset were not being considered for military contracts"

come on, which is it? Is it all about saftey or is it that only US/Israeli ai is allowed to kill? Seems to me that the only real threat is to the techno fudalism OAi, Anthropic & co are trying to build.

He wants to restrict China from access to chips. All that will do will accelerate their development of competing, sovereign hardware.

As an example take North Korea. Sanctions didn't stop them from developing nukes and delivery systems.

How are Point 1, "We should not sell powerful chips or chipmaking equipment to China," and Point 2, "We should crack down on industrial-scale distillation operations," addressed by making open-weight models illegal?

Point 1 is about restricting the sale of chips, not models. Point 2 concerns companies using closed models to train their own models through distillation.

The real issue is that open-weight models can run on much less powerful hardware, making the current AI business model, where companies train a powerful model and gatekeep access to it, less relevant. Once open-weight models become good enough, much of the future revenue for today's leading AI labs could disappear. But just as Microsoft still has a market so will the large AI houses have one, but the moat will not be providing AI responses.

I guess they have to make some statement about this, but we don't have to care. This is like the zoo making a statement about the employment of clowns. Clowns a a circus thing, nobody asked the zoo's opinion.

It really barely matters. Even if the US closes itself of for Chinese models; Europe won't. The rest of the world won't.

Even a completely closed US economy would not merit the current valuations Antrophic and OpenAI have.

I am so surprised of Dario's inclination for centralization that obviously makes unsustainable and overleveraged governance systems. There is definitely mismatches over the principle of distribution of power...

Basically:

"We are in favour of 3D printers but there should be a body that tests and certifies that a 3D printer cannot print anything that can be used as weapon. Anything pointy or with a spring and recoil or... or..."

They're sweating it big time, and their fear is bleeding through their AI beneficence-speak. Time for the big US AI companies to call in some favors from the administration. But you can't unring a bell.

> We should not sell powerful chips or chipmaking equipment to China

> We should crack down on industrial-scale distillation operations.

So Open-weight models are perfectly fine, but we don't want anyone to be able to make them.

So, Dario says that access to hardware and governmental certification are _load-bearing_ for his business. Can't charge premium for cache usage when China allows their models to be run by anyone.

I think the US has a much bigger authoritarianism problem than China does.

Meanwhile Chinese chip-makers are chip-making. If you believe the threat of these open-weight AI is existential, how long do you think these bans (that only work for hardware) will work in your favor?

oh, it‘s the CEO of a well known AI company educating us all – and all he wants is us to hear his hunch on open weight models?! Amazing, please help us understand the situation a bit better, thanks

Anthropic will continue being a victim of their own naive positions on AI safety. They keep dancing around it but their communication is essentially pro-regulation if you read between the lines.

Hmmmm.

Conclusion: open weights models are good for Anthropic because they shows the so called threat that will make congress allow pouring money in Anthropic for national security & AI arms race

These are all valid points but not strong enough to warrant banning open models IMO. This is a new reality we have to live with, with new dangers we cannot avoid. Best solution is improving the social system, lowering the incentive for people to do harm, restoring social trust, etc... This is the only way I see the US getting out of it. We really need a high trust society now. This is the only protection. We need neighbors looking after each other. There needs to be an incentive for people to do the right thing. Everyone should want to protect the system. Coercing people into not doing harm is never going to work long term IMO.

When people say "What will be left for us to work on once AI takes over", I say "Ourselves" - We have to stop looking at humans as commodities and strategic pieces and start paying attention to people as individuals, based on the content of their character. We need a society which is attuned to this, which has enough resources and time to pay attention to this. Now we are blind to people's character because it is masked by money, power and status; all of which currently have higher priority. This order of priority is determined by scarcity, which is largely artificial.

We have a dishonest system which tries to control people's behaviors through scarcity-based coercion instead of straight forward laws or simple incentives and clear explanations.

Considering the state of the current US Administration, its impossible for me to take cries of national security and worries about abuse from "authoritarian governments" seriously

Hey, no fair stealing the value thay we already stole fair and square!

Then: Nuclear technology is too dangerous, only we can be trusted with it. Now: AI technology is too dangerous, only we can be trusted with it. Different century, same double standard

It's so ironic to me the way people will say "china should not have these chips" meanwhile they manufacture like 99% of all the electronics we have in the united states.

If you gate models for safety you aren't safe. There will always be actors with ungated models. Better that we use ungated models to improve our security. It's an arms race.

> China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips.

So anyone with chips can produce a model ?

> Anthropic has never advocated for a ban on open-weights models.

If you wonder why this is written as an opening to a list of reasons that advocate for banning the open weight models, it’s because

Ironic to oppose giving AI tech to “authoritarian governments” while approvingly quoting the authoritarian-wannabe government of the US and framing that government as the good guys.

I want to play with every kid, but my dad says that kid is bad. Every kid should be pat down by my dad, otherwise I don't want you to play with them, it's good for you.

It just also happens that open weight models are a massive financial threat to the existence of Anthropic as a company… so the might just have something to do with this position.

Seems pretty reasonable.

How long does he imagine restricting chip use would work? Surely China would aim to manufacture their own. This seems like short term thinking, likely with IPO in mind

This reads almost dishonestly.

> Anthropic has never advocated for a ban on open-weights models.

This is not an unqualified never. The very next sentence makes a qualified statement: "Open-weights models that don’t have dangerous capabilities are a public good". That prompts the question, what about ones which do have "dangerous capabilities"? Are they not a public good? If not, then should they be banned? Who gets to decide on the definitions of these terms?

I copy pasted this whole debate in Claude Sonnet 5 (web). The model came up with a summary but the thinking trace was entirely omitted.

Well, if you ask me, that is dangerous.

distillation: Pirates people’s lifetime of copyrighted work, makes billions selling access to it through APIs, then tells us we can only use it in ways they approve.

The core concerns stated as use of AI in drones and surveillance, by China. And what does USA government do with AI? Drawing pictures of flowers and writing novels?

Well their valuation is going down the drain so no wonder they don't like it. The cherry on top will be China developing their own chips and chip making tech.

As an American I’d gladly take payments from China to feed them my Claude transcripts for distillation. I’m surprised I haven’t heard of such an initiative.

I don't trust you Dario, that's the issue here

There is an odd assumption behind this post that you can put effective guardrails on a model before releasing it. I thought nobody had such technology?

or 3: cease your assault of the international order, drop the zero sum paradigm and take the opportunity given by these crumbling US-build institutions to build a truly global network of lawful cooperation, where everybody can win. Admit that it wasn't fair to begin with, and start anew. It is in the best interest of everybody, also the US and China, and therefor the most rational thing to do.

This seems hypocritical, out of touch, and hollow. "Safety testing" is just a hair away from censorship when it comes to government control.

"We should not sell powerful chips or chipmaking equipment to China,"

Why? Has Anthropic, or the country it is based in formally declared war against China?

Amodei: Open model not so much bad, but open model in china hands very bad Non-US world: But open model is in every ones hands. Amodei: ... shut-up.

> authoritarian regimes have stolen and used AI

"we're not saying open models are stealing, but we will quote the US vice president, saying this"

First thing I'm going to do with my uncensored Kimi K3 release is to engineer a virus that only targets gatekeepers in the AI field.

(obviously this is a joke)

When they use ill gotten data for training their models the world's smallest violin plays when they complain about distillation attacks.

> We should not sell powerful chips or chipmaking equipment to China

And accelerate their development of independent chip making technologies even more…

https://www.antirez.com/news/172

Welcome to bizarro world!

Fist off: "the most dangerous model may be one that is trained in secret" <-- Says the guy that not only restricts commercial use for some of their models but develops them in utter secrecy. With the pretext of guardrails. Then show us the guardrails you really use by opening the weights.

Second: "use in drones [...] for surveillance and repression" <-- writes the King of FUD, as the US is an an active campaign with the help of their models. And/or OpenAI's.

I am very appreciative of the freedoms of the west but this type of hypocrisy and lack of self-awareness is bonkers and it should be called out.

I think he asked LLM to rephrase "I want open models to be banned, but sound like I don't want it but mean it somehow".

I like the idea that Mr. Amodei holds these beliefs because he wants to safeguard democracy as much as the next guy.

I just don’t find it believable.

I'm a lot more worried about the US government than China's -- it has a lot more direct impact on my life and is largely controlled by billionaires who do not have good intentions toward the rest of us.

And, to me, this letter comes off as quite insincere. Stopping distillation can only be explained as an anti-competitive measure. Their own explanation is nonsensical -- they say is needs to be stopped to help prevent authoritarian governments from overtaking the US at the frontier of AI. But by its nature distillation lags behind the frontier. Not to mention the US is one of the authoritarian governments we need to be concerned with, and the next thing they advocate for is full, worldwide regulatory control of AI, which is rather heavily authoritarian.

These guys are making a $T gamble and need to screw over a lot of people very badly to make it pay off. You do not want to trust anything they say.

Clearly anthropic is misaligned, how can they claim to be able to align AI when they themselves are misaligned from humanity.

> Open-weights models that don’t have dangerous capabilities are a public good

Note the hedging against 'dangerous capabilities'. Undoubtedly, all the useful ones trigger this condition in Anthropic's eyes. The rest of the post is filled with similar weasel-wording. Make no mistake, this absolutely confirms that Anthropic is against open models in the sense that any reasonable person understands them.

The way the rest of the post unabashedly appeals to the current US administration's China hysteria is hilarious, and not at all subtle.

I guess we'll see about all the doomsaying here, won't we? Kimi K3 is frontier-level, and there's no stopping it now. As far as the world is concerned, anyway. If the US wants to kneecap itself that's another matter.

I welcome any and all competition. But they first need to be able to run the obstacle course I’m setting up with Uncle Sam.

I think this letters tells us everything we need to know about the man. I've rarely seen so much flattery towards the current administration, contradiction, deflection, half-truths and hypocrisy on a single page. This man is scared of everyone: the current admin, the public, and the other companies promoting open-weights.

> Open-weights models that don’t have dangerous capabilities are a public good

Knives should only cut during the day, knives which cut at night are bad.

Generational good will loss from these guys.

he's asking for the most powerful models to be in the hands of the few while the majority will be at their mercy.

A sober look at actual rogue nations and their use of AI shouldn’t have us fretting over that particular hemisphere

>Open-weights models that don’t have dangerous capabilities are a public good

For convenient definitions of "dangerous"

Banning exports to China is merely a narrative device; the real intention is to sideline intelligent technologies.

These globalists were happy outsourcing manufacturing and software development. They are now angry that they failed to secure their IP, which was already based on dubious extraction of the collective works of human IP.

You can't have it both ways. Either it's fine to outsource to China, fine to use H1Bs to undercut US talent, or, foreign governments are strategically positioning and their espionage is a threat to America.

The Roman empire fell for the same reason, didn't it!? They wanted to control it all ... But it was too big

As an act of good faith and proof, they could stop spending the record-breaking lobbying money for couple years.

I found lots of comments here suffer from proximity bias.

Anthropic anti-open-model stance does not mean China is not a threat.

At least try to release one open weight before you say anything, or all of that only sound hypocrite at best.

Do distilled models have to inherit guardrails? Can a model distilled from a “safe” model be made unsafe?

We do not want to ban using those models, we want to ban the things that enable those models to exist.

All sufficiently capable models, open and closed, should go through mandatory safety testing.

Done by who?

I thought his points were legitimate and well argued. I think that it's very easy to say "X should be unrestricted". I also think that eventually one of these models will be used to do something truly destructive and insane and that some level of "compelled responsibility" to the open-weight model ecosystem is inevitable.

EU must cooperate with China in same way as with US. They're both same level of evil.

I was pleasantly surprised by this release and the tone at the very beginning, but quickly it is painfully obvious that it's blatantly requesting a ban on open-weight models. The absurd demand for some safety arbiter by World Police America is farcical.

Yeah, the rest of the world is going to bow out of your busted idiocracy, guy.

Further, Anthropic needs to can it with the horseshit distillation bullshit. No, you aren't really the secret sauce, and this is basically trying to con stakeholders by pretending that there really is a moat, only you just need to add more crocodiles.

A significant percentage of innovations in AI lately has come from China. China is now making their own seriously competitive hardware, and they can steal content just as effectively as Anthropic to train their models. Why wouldn't they be competitive?

The pathetic claim that if you just stop distillation and prevent hardware smuggling and Anthropic and OpenAI will have the same moat is delusional. I mean, more correctly it's simply fraudulent, and he clearly knows it's bullshit meant to convince much stupider people.

This sort of stuff betrays a stunning lack of self awareness. The US are the worldwide risk. The US are the ones threatening allies and bombing 10+ countries. The US are the ones carrying out war criming and pillaging, pirating and burning? The US are the ones with the guy threatening to use nuclear weapons on a weekly basis.

If Anthropic remotely believed their bullshit, they would shut down today and burn the hard drives. But they don't, and the pathetic call out to Vance (please daddy, ban those dangerous models!) is deplorable garbage.

This ridiculous, shameless "note" has an audience of one: JD Vance.

Jesus Dario we get it man, you want clout for the IPO.

This constant whining from anthropic about distillation attacks continues to be rich given the amount of stolen data that went into any Claude variant.

Perhaps, the general public needs to formulate its position on Anthropic models...

> "We should crack down on industrial-scale distillation operations" I'd prefer if there was a crack down on industrial-scale scraping. Maybe even reimbursement for the problems it has caused (some nasty AWS bills, tons of man-hours spent on preventing new nasty AWS bills, etc).

Global AI safety apparently means keeping the world safely dependent on the US.

Makes sense since ICE, Israel are so much nicer & kinder than the Chinese

> We should crack down on industrial-scale distillation operations.

IP for me, not for thee.

TLDR: "we're not in favour of banning our competitor's products, but we do advocate banning hardware exports to foreign countries where competitors are located".

There's also a very strong implicit double-standard in the discourse, along the lines of "it's dangerous if non-US uses this in military fields, but it's fine if the US does so".

For smart Chinese hired by anthropic, are you still proud working there?

“By virtue of being the good people everything we do is good, and if it happens to be in our best personal and financial interest then that is good too because it enables us to do more good. And if it’s to the detriment to others then it’s because you don’t understand the good behind it. Which is fine, because we’re good and you can trust that what we do is good because what we do is good by virtue of us being the good ones.”

https://x.com/biggest/status/2081891756769952075?s=46 Official stance of an anthropic-employee being against Dario amodei opinion.

Our position on open weights models = don't sell gpus to china?

> Anthropic has never advocated for a ban on open-weights models.

What are the legal ramifications of this statement if it turns out Anthropic have lobbied for this? Does it just get swept under the rug? I can't say this is bullshit (that would be defamatory) but I am intensely skeptical.

> China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips.

This is playing to readers' biases; isn't DeepSeek V4 Pro deployed on Huawei Ascend already? The old "Chinese can only copy" meme is getting pretty tired these days.

> All sufficiently capable models, open and closed, should go through mandatory safety testing

Applying such standards in the US means that US defenders are blocked from using the models, but attackers from other countries aren't. That is clearly counterproductive.

It's already been pointed out quite eloquently elsewhere that there is no such thing as a safety filter because the LLM and external filters can't actually identify malicious use. They can only identify the weaker implication "if the user is malicious, this is bad."

Americans don’t get to lecture the world on authoritarianism anymore

I'm no longer supporting this vendor, for personal or business

The question is whether Claude suggested that he should write it.

Hmmm, how could this post be possibly a good thing for Anthropic?

It's funny how as an outsider who is neither from the US or China, both look the same.

Edit: To add some more context. What I mean is neither look like the good guys or the bad guys, but one of them is spending an awful amount of energy trying to paint the other as the bad guy and themselves as a good guy, which I hope a lot of people aren't buying anymore. Because at the end of the day, I think the honest truth is that everybody is just trying to serve their own interests.

I wonder if publishing these documents is not just a public stunt, but heavily integrated with Anthropic's business storategy to maximize operational efficiency. Companies often have several internal documents for a single policy like "position on open-weight models", one for public (like this), others for the legal team, the lobbyist, the developers, the investors, etc. The differences and nuances of those manuals can be very huge and are necessary to maximize the goal from each branch, but often a cause of headaches like bureaucracy, communication friction, outdated information, etc.

A single canonical official document can make it very simple. Even though each department cannot achieve the maximum gain from nuanced documents, keeping operational context as simple as possible may really improve LLM driven operations to move faster and cut cost.

If "publishing pleasant positions and actually following them in general" becomes a good business storategy in LLM driven society, it can be one of very few good outcomes from this dystopian AI craze.

Why should consumers care what Coca-cola thinks about Pepsi?

[flagged]

This letter is so full of contradictions.

> Open-weights models that don’t have dangerous capabilities are a public good

There are no models that don't have dangerous capabilities. There is not a single human in the world that doesn't have a dangerous capability.

> authoritarian government [] build AI models that are more powerful than those built by the US

Yeah, they will do that no matter whether you support the existence of open weight models or don't. In fact, they have probably already done that. I would say "deal with it" but there is nothing you can do actually.

> authoritarian regimes have stolen and used AI

Monkey looking away meme.jpg

> powerful AI models may be misused

Small AI models may be misused just as well. I would say it's the small models that are the problem: cheaply deployable, easily fine-tunable, fast.

> once weights are released they cannot be withdrawn

Exactly. It has been already done. So?

> We should not sell powerful chips or chipmaking equipment to China

I agree. We should have competition and I'm rooting for China to design their own chips. They wouldn't if we let them buy our's.

> We should crack down on industrial-scale distillation operations

Well, at this time it's already irrelevant. We have GLM-2, DeepSeek, K3, which are already SOTA and can be used for distillation.

> All sufficiently capable models [] should go through mandatory safety testing

Problem is, it's unenforceable legally. Models are just math and all previous attempts to ban maths have failed and simply pushed US back. I will not be asking US government permission to calculate matrixes of my choice.

> For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed)5. Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance.

One of two outcomes are true in this scenario: 1) this is unrealistic fear-mongering or 2) we're all fucked.

I just don't see how our solution to this can be export controls or bans. If the fear of a bioweapon engineered by an open weight AI is a legitimate threat, our only option is to develop effective countermeasures (and perhaps the same AI will assist with protecting). Open weights are not going away and pretending like some sort of "ban" will prevent bad actors from accessing them is a fairy tale.

Amazing how this company went from having such goodwill to hardly any. At the end of the day they want to make as much money as possible. Anthropic will have to lose a ton of their profitability if they compete with open source. I see them moving into the application layer, which they’ve already started doing. It’s clear open models are the future for the vast majority of use cases that don’t need frontier capabilities.

"China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips"

source: Trust me bro.

There are hundreds of articles showing that China have developed their own chips and have a massive manufacturing capacity. This blog post feels like is pondering to the brain dead Fox News audience.

Anthropic starting to sound like Microsoft from the 1990s.

"Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn."

Translation: "we won't be able to monetise it"

"We should not sell powerful chips or chipmaking equipment to China"

Translation: "we can buy them cheaper when there's less purchasing competition".

"Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable."

Translation: "we've been outsmarted and lost our advantage because their way is faster and cheaper"

I think it's a bit rich to complain about distillation, when they been plundering the internet for years for copyrighted works to train their models on without permission.

Suggested title: Dario Amodei's crusade on reality.

"Open-weights models that don’t have dangerous capabilities are a public good"

Read between the lines folks. Anthropic deems every model that has frontier capabilities as "dangerous", and thus they are against them. We all know that "dangerous" simply means "whatever model hurts our bottom line."

More dishonest framing from the company that constantly lies to everyone. No surprises here.

This USian death spiral season sure is picking up speed

China will likely gain the capability to produce leading edge chips within a decade domestically. From then, occupying Taiwan will be about controlling the wests the access to that. Same with AI models, though I would say we are already about there. Dario is either lying to us or to himself that distillation plays a major role.

Man, they're just worried about their bottom line

With respect to whomever this dude thinks he is, it's egregiously "main character" to frame a policy recommendation for the US in terms of what's important to his personal beliefs.

Talk about a giant whiff.

I was hoping they would announce their first open weights model, perhaps an older model they don’t offer anymore, but no. Instead he get this bs statement that reeks of “dam it I’m so close to being a billionaire” desperation. Not even acknowledgement of how much data they stole from others yet he whines about distilling.

It’s like his goal in life is to be a Scooby-Doo villain.

The consent factory has never been more productive.

If we really cared about AI we would work together

I agree with Dario Amodei. Every word makes sense.

How about we don't let the leading model makers make the rules? How about we make AI models that were trained on public data public goods? Let's circle back on that, kthxbye.

> industrial scale distillation attacks

You mean the industrial scale distillation attack you perform on the entire corpus of human knowledge… idk, call me cynical but you reap what you sow.

Does Flock use open weights or Frontier models?

A lot of people have a lot of concerns with AI, its capabilities and with how it is used now and what it will lead to in the future. For good reason. But the question is, do we have a strategy that is actually useful? If so, what is it? I think nature shows us some answers in ecosystems.

Diverse ecosystems can absorb shocks. Diverse ecosystems are a sign of health of that ecosystem. When an invasive species comes into a healthy, diverse, ecosystem it doesn't mean that it isn't disrupted, but it does mean that it is far more likely to emerge with a lot of its diversity intact. In fact, it is likely to emerge even stronger because it can absorb that new shock and incorporate it, adding to its diversity. The balance may be changed, but the ecosystem survives or even thrives.

Nature also likes to show us that artificial barriers rarely last. You want to control a river? Good luck. It take constant maintenance to hold that flow in place and even then you are likely to get extremes that are made worse by your efforts because, eventually, somewhere in the system fails in a way you didn't anticipate. Then the water comes rushing in. Artificial barriers often have a way of building up tension over time, not reducing it, so that when a failure eventually happens it can be catastrophic. In other words, you had better really understand the system you are trying to control or else you can make things actively worse.

Relating this to the world now means, I think, that our best chance to minimize long term shock and maximize the chance that the diversity we have around us survives is to try to grow as healthy of an ecosystem as we can as quickly as possible. Lots of models large and small in lots of different hands is, I think, a better solution than artificial barriers restricting the variety and diversity of models and users. I think this is closer to an ecosystem solution and has a shot at working. Basically, I highly doubt we understand this situation enough to do a good job of controlling it with artificial barriers. Instead I think we are more likely to build catastrophic imbalances than we are to create the healthy ecosystem we really need.

He is concerned about the perpetration of incredibly deep repression of their own people. But completely blind to the fact the USA are the current champs at this.

We need a seti@home for open weight models

Their number one concern is about the the commies perpetrating deep repression of their own people! How noble! This whole time I thought it was because they wanted more money and power. I guess we should probably ban these open weight models.

> 1. My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.

What a nice implicit way to say that it would be okay if the authoritarian government is the current US government trying to perpetrate incredibly deep repression of their own people.

Let all trust this team to deliver AGI.

“We care about safety so powerful open models are bad” - and what of OpenAI? What if Mythos got into the wrong hands? It’s a pretty weak argument

> We should not sell powerful chips or chipmaking equipment to China

Yes, please. We don't know whether we'd have open weight models today, had the chip-prohibition not been in place. Nor would we see the more optimized models such as DeepSeek or qwen.

We also would not see new players entering RAM market after you and your pals in Silicon Valley hoarded the entire world's hardware.

So by all means, double, no, triple down on this.

> We should crack down on industrial-scale distillation operations

And let's apply this retroactively to Anthropic too. You industrial-scale-operation-distilled all of humanity's knowledge. Let's have some of that crack down on you too.

I’m done reading anthropic propaganda

> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.

There's not much stopping US from becoming an authoritarian regime, and the US is already using AI against its own citizens.

But what if it's the US that becomes authoritarian and uses AI models to perpetrate incredibly deep repression of their own people?

Translation:

> Anthropic has never advocated for a ban on open-weights models.

"We don't want a total ban on ALL open-weights models" (Anthropic never released a single open weight model)

> All sufficiently capable models, open and closed, should go through mandatory safety testing.

"We want tight regulations on highly powerful open or closed weight models that should go through mandatory safety testing that we outline which makes them safe to use."

This is still a form of a ban that he wants to define. But the rest of his concerns such as stopping distillation attacks and not selling chips to China all do NOT work.

I disagree with 100% of everything said in this article.

> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP)....

This is why open weights win. See Linux and how it's taken over the world. Your business model will need to change eventually. Instead you're advocating trying to exterminate competition via regulation and fear mongering.

> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks

Yawn... this is getting old.

> We should not sell powerful chips or chipmaking equipment to China

For as someone as smart as you guys, you sure lack common sense. China is just going to develop these technologies organically then and you lose 100% of control. It's already happened in reverse with things like Solar, rare earth minerals, etc. China flooded our market, destroyed our ability to produce things, now holds the keys. One thing they DIDNT do was stop trading to the US. They killed us with cheap goods.

> We should crack down on industrial-scale distillation operations.

Thats your problem, not my problem. Also, irony meter here hitting 11 about all those pirated books you stole...

> All sufficiently capable models, open and closed, should go through mandatory safety testing

Oh, fuck, no. This is a crackdown on free speech and rights of people to do whatever they want. My right to free speech means I'm allowed to write whatever computer program I want, no matter what its size is or how "sufficiently advanced" it is. Individual rights always win.

I really hope people don't believe this garbage. For a company with a great product, this is absolute nonsense.

That is not an argument against open weight models. That's just a generic protectionist argument against any Other lab.

I'm cynical enough that I would suspect all of his statements are duplicitous anyway, but my recent experiences with Claude Fable give weight to it.

I asked a question about a series of tokens - bam, denied and downgraded. There's no cyber security or public risk here, but Fable doesn't want me to learn how things work.

I asked a question about quantization in models - bam, denied and downgraded. I edit my question to make it clear I'm talking about Google's Gemma QAT models. Oh, that's fine then, and it answered the question helpfully.

Anti-competitive bullshit. I hope they fail.

So many US-haters that live in the US with no intention to leave...That right there is why this is the greatest country on Earth.

Woof, talk about self aggrandizing.

This man sees himself as your master.

Do you accept?

The Anthropic CEO has an entitlement problem. I read his opinion twice and somehow in his worldview, if any other country gets better at AI than the US, the apocalypse will come and the US will be no more. Or something along those lines.

Secondly, he's like the Trump of tech: China, China, China.

Thirdly, he needs to cut it out with the guardrails and other safety BS he's peddling. If we're going to get Skynet, we will get it no matter what we do.

> ...that “authoritarian regimes"

Umm, isn't the US acting like another authoritarian regime by advocating a certain kind of obstacle because only a US regime is allowed[1] to do what it is fearing[1] about:

[1] https://en.wikipedia.org/wiki/United_States_Army_Biological_...

[2] AI models may be misused to carry out cyberattacks or biological attacks

Anyway, Dario. You are more concerned about your business than anything else.

How did it land on biological warfare. and why. why not chemical warfare.

this guy is smokin spaceballs and should be institutionalized.

Frankly, why would I put much weight into what Anthropic say about open weight models?

Reply on news.ycombinator.com