Hacker News Reader

Comment by roughly | Hacker News Reader

This has the delightful and surely unintentional side effect of all but ensuring all federal agencies who want to use LLMs have to direct their budget to one of the domestic frontier labs (OpenAI, xAI), so that’s a nice big fat pipe of taxpayer money that’ll be going to backstop those cats while they deal with the fact that they’re getting their lunch eaten by competitors who didn’t decide the way to build a sustainable future was to incur future debts to the tune of a sizable percent of the GDP of a G20 nation.

View on HN

You gotta make trade-offs.

Two facets here. Economy and Security. Is current US governemnt in favor of providing cheaper, less SOTA models to Americans at the cost of exposing US content to Chinese firms? Chinese firms by Chinese law are forced to handover whenever CCP asks them to.

Given all sensitive information, the security outjustify the cost here.

There are other areas we need to look at but for my analogy, this should work.

Eh I don’t buy it. The framework for a US owned Deepseek is there and easy to put together. What isn’t there is the money.

My assumption is that money has been hitting the right pockets to make that not happen.

Completely disregarding security aspect without any rebuttal for my claim.

Your assumption is half-assed because it's missing that

You can just host the inference locally... Baseten and Fireworks already do that for you.

What is the security risk of using Chinese models hosted in the US by American companies?

Technically, there's a risk of abliteratted model leaking data to external third party: https://projectdiscovery.io/research/how-abliterated-models-...

This problem is not unique to running Chinese models.

Indeed, but Chinese AI model companiea are harder to held accountable by the US than US companies, so there's some sense for the US to prefer national models over international ones.

If the US had comparable open models as the chinese ones, I would love to see more investment into them. But instead our government is operating a lot more in line with how individuals who stand to profit from the private frontier models would operate...

Personally, I agree. Still, it's not surprising that the government is choosing the safer option to itself. On itself, it's quite reasonable.

I believe most people complaining about this decision are really just have grievances towards the state of US AI companies rather than the decision itself.

A Chinese company has less incentive to use US customer's data than a US company does. Does a Chinese company have an incentive to steal national secrets or cause disruption? 100%. But any firm or project that has national secrets should be vetting everything and using in-house software anyway. And if not, then if I were China I'd be trying to breach the makers of the software (in this case Anthropic,OpenAi,etc).

For the average person, China doesn't care about you or me. For a US company, US citizens are their gold mine.

If anything, the competition from a Chinese company will reduce a US company's possible profits. That's the threat. That's why Chinese EVs are banned in the US and why Chinese LLMs are next.

Just because something is made in China does not make it a threat to the US. The US trades all sorts of goods, produce, etc with China. The traded goods can be poisoned at any time. At what cost? Well we wouldn't be able to trade anymore. And that hurts both sides. But once a foreign nation does something better than a US firm, the US will put up a ban to protect its own stock market. This isn't new.

We are talking about a ban on Chinese AI on federal agencies. The concern isn't about the general citizen data, but private federal information and secrets, some of which could pertain to critical IT infrastructure. This is absolutely of interest to Chinese companies, but also, primarily, the CCP (or any potential enemy, really).

Restricting national secrets to only in-house software isn't guaranteed to provide better security (you still need to hire security specialists), but is guaranteed to spend resources in hiring sufficiently qualified personnel to maintain said software while ignoring the pool of potentially better qualified teams already hired by national companies. So, an alternative, is to use private contractors with different levels of secret sharing. This kind of choice isn't trivial to make.

You seem to forgetting the bonus side of "The traded goods can be poisoned at any time." If the exfiltrated data is more worth than the repercussions that are going to emerge out of it, then it's absolutely worth it. And this isn't specific to China. China holds a similar perspective to US in this regard, but with foreign services in general (arguably more strict, given it affects the entire populace). You just can't know when a third party will go rogue, do they take a more conservative approach of producing "locally", with private companies they can punish if they go against their interest. The current US government, ironically, is good evidence of this.

So much accountability!!!

Are you going to make an argument or just spout thought terminating cliches? Do you really think OpenAI, for instance, would be able to cross the current administration without any serious consequences?

Mostly referencing the hugging face incident. It’s a felony.

Sorry for my rudeness earlier. But I'm emphasizing that the accountability that US gov. cares (specially with this prez) is only OpenAI towards the US gov. They don't care about it towards others.

> are harder to held accountable

Well, that is not an issue. The US companies that hacked other seem to not be prosecuted at all.

Also, no state attorney doesn't seem to prosecute them on the large scale intellectual theft. Instead content creators like movie companies or the kids book author that made the Grüffelo need to start one-after-the-other civil claims onto them.

So US companies in the LLM area can be ruthlessly criminal. No one holds them accountable.

There are rumors that they stole chat logs from mathemiticians running problems through them for their own math findings. Again, no one holds them accountable.

The "hold accountable" is IMHO just CEO speach: fluff and empty air, but nothing behind it. If my opinion is true, then it doesn't matter if I can NOT hold an european company, a US company, or a chinese company accountable.

Also, currently the US is more and more aggresive towards other countries. Haven't heard of China raising tariffs for no reason at all, but random amounts. Or bullying e.g. european companies if they want to control the the market might (and abuse of it, see e.g. their uncontractual price raises or what they did to Ukraine --- with Starlink terminals payed for by european countries!).

China is no one friends ... don't get me wrong. But it isn't as vile as the USA currently is.

Like I have stated elsewhere, the US gov. only cares about accountability towards itself, not towards others. If OpenAI decides to "accidentally" reveal state secrets to foreign agents, the US will come after them.

A theoretical scenario: I am a company in the united states making missile seekers.

My software guys say, hey, the imaging infrared seeker is like a zillion percent harder to fool if it has this tiny LLM brain that we blow into the chip. They grab the most optimized model, which so happens to come from a PRC lab. Sure, it's running on all US hardware but, deep down in its little LLM brain, it so happens to be virtually blind to any IR signature that has a pulsed frequency of XXX.

Now all my missiles have what is basically an externally controlled IFF in their little missile brains. "Chinese jet? I see no Chinese jet."

Or what if the pulse contained more instructions? Like, say, "Do a U-Turn and Lock On The Closest Jet Motor You See". A couple of PLAAF pilots watching USN aviators frantically trying to dodge their own missiles - what might possibly be the most embarrassing air-to-air engagement in aviation history.

These models are called nondeterministic for a reason, and it's because the massively complex interrelated weights are incredibly hard to pick apart and audit. It's not like OSS code from a deterministic software program, like a pypi dependency. Now, it's not impossible, and THAT is where the weight of regulation and such SHOULD be going, but, hey, not sure you noticed, US Leadership is not exactly sitting in a surfeit of technical expertise at the moment.

I don't think it's the right call, to blanket ban them, but it's understandable.

That would have to be a stuxnet-level targeted sting operation, with a general purpose LLM exhibiting some ridiculously specific behavior in ridiculously specific circumstances, in such a way that nobody catches it in testing (and none of the other zillions of users of that general purpose LLM notices anything odd). If you're so paranoid that you do a blanket ban rather than just saying "hey guys maybe double check before using Chinese software in weapons we plan on maybe pointing at the Chinese", frankly LLMs should be the least of your worries, supply chain wise. Every US government employee uses a phone made in China (because they all are), and the president doesn't even update his, so color me skeptical that this has anything to do with security posture.

Now you know how much testing happens in defense systems. Or, more accurately, the quality of the testing regime, in defense systems.

That said, totally agree. There's no way this was fueled from actual concern. They're just nervous about the needles drifting next to the bubbles full of money.

But about that supply chain . . . . <nervous_whistling.wav/>

“Completely disregarding security aspect”

Did you miss the part of my comment where I mentioned this would be the US’s version in your rush to reply as smugly as possible?

Maybe read the content of a response before responding.

lol. Yeah just use “CCP” and “security” as bogeyman keywords you’d absolve all the crimes from closed labs and win arguments automatically

> Chinese firms by Chinese law are forced to handover whenever CCP asks them to.

Are we pretending the same isn't true of American firms?

The steel man is that American requests must be lawful, are done transparently through the courts, are appealable, and don’t come through secret agencies.

The titanium gold alloy man is that the Chinese laws are inarguably stricter and more authoritarian than the US laws, regardless of the actual implementation of either.

> American requests ... are done transparently through the courts, are appealable, and don’t come through secret agencies

This reads like someone who has paid absolutely no attention to American law for 25 years.

You are not providing anything or justifying with any evidence.

It's a steelman ... not an actual claim about reality. It says so right before the words you've quoted.

Isn't this a misuse of the word steelman, because it wasn't accompanied by the criticism?

From Google Search overview box:

> To steelman means to build the strongest, most charitable version of an opposing argument—even better than the way your opponent stated it—before you criticize or reply to it. It is the exact opposite of a strawman, which is when you intentionally weaken or misrepresent someone's view to make it easy to knock down.

I'm sorry friend.

Police force is hiding their face. Exquisite contracts are being made through informal announcements. Strong suspicion on inside trading from high ranks. A new level of people policing. Waivers for criminals already convicted to support coup on other foreign states.

Lies or at least speeches that are plain denial of obvious facts or brush citizen problems as they were mere senseless complaints

Speech about god punishment and the end of the days almost weekly Threats to use weapons graded of world level destruction for a conflict that "would have take a week"

Billionaire and trillionaires plainly saying what they want the government to do

In China, apparently there is at least one person who takes responsibility for everything no matter what and has been playing the discourse of growing together instead of attacking every single ally with unproven information while (seemingly) profiting from all crisis.

I don't know man. There is very little to defend. And look. I ain't American, and I still thought that guy had some good qualities despite the rest.

The nixonlike policy through the world in a state where only those people will have anything. Pledges for cheques, stocks, accounts. Not even that seems to be materializing up to now

I do like my American folks, wish all of them the best, but don't fall for "the world against you talk" This one is on you.

> Are we pretending the same isn't true of American firms?

Are we really pretending it's equivalent?

Certainly its not the same, or atleast thats what thet want you to think. BUT I thought uncle sam had the ultimate authority?

Its the land of freedom until they decide they dont like you.

>Are we really pretending it's equivalent?

Yes, it is equivalent if you live anywhere but America.

The orders bans the use of open weights provided by DeepSeek too. It bans the use of the open source inference engine DeepSeek released today.

Sure. Wayyy more secure to shovel federal data into OpenAI /s

The United States government needs to make tradeoffs. They are as you list.

The Chinese government also needs to make tradeoffs. You can imagine what they are.

You as an individual also need to make tradeoffs. In doing so, you ought be sure to remember you’re not a nation state, you’re an individual human being.

(You may decide “I find myself one small being in a world full of nation states, and given that, I trust the US government to keep me safe, and therefore will align myself with its interests,” at which point “ban DeepSeek” is in your interests as well as those of the US government, but you shouldn’t get the two confused, lest you miss the moments those interests diverge.)

I am an individual living in a nation state that has nurtured and protected me and made me rich, comparatively, so it is in my interest to pledge allegiance to that nation and protect it in kind. One of the ways is by not giving training data to foreign models runners which doesn't mean we can't run the foreign models locally.

It’s currently in your interest to do so, or so you believe. That’s fine. Understand that may change, because, again, you are an individual and not a nation state, and specifically not the nation state you’re pledging allegiance to and putting your safety in the hands of.

I swear that "think of the children" as the means of getting perverse laws justified has just been switched with "but... China".

Let's not fool ourselves into trying to believe this is anything other than protecting the interests of a few of Donny Tango's mates. He has already proven that he has absolutely no problem throwing the USA, its citizens or its allies under the bus for a couple quid.

Absurd. Deepseek is open source. You can just self host it.

I'd guess a lot of government agencies (globally) will end up with their own compute clusters hosting open source models. Much cheaper at that scale and much less hassle than defining which critical data cannot be fed to an external llm.

Not sure if the US will fall under that since it's slightly less risky for the US government, but I wouldn't be surprised.

A lot of government agencies already started doing this and the US government is far from the only one banning on-prem Chinese open weight models. Can't answer whether this makes sense, but I do think a lot of policy makers just figure there are mostly comparable alternatives that don't carry that vague risk.

Two things can be true. Deepseek is a great model doing great things for democratizing AI and the federal government should never use a Chinese model for anything serious or sensitive, China will have access to literally everything. And before anyone gets upset over that, China would be crazy to use American models for anything sensitive because you have to assume the US government could access it. It goes both ways.

This ban also applies to on-prem hosting of Deepseek, and to finetunes/post-trains of Deepseek models that are provided by US-based inference providers. It's not a given that the Chinese would be able to access any of that.

It’s also not a certainty that they wouldn’t be able to access it. Verification of everything would be prohibitively expensive, it’s far cheaper to have US-based LLM companies which are compliant to whatever standard already be used, even if their tokens are more expensive.

As they say, better the devil you know.

> China will have access to literally everything

So these are open weight models. You can run them on your own hardware. How would they have “access to literally everything”? Or what is the threat model you are thinking about here?

To be fair, it is possible that an open-weight model could still hose you by exfiltrating data in response to prompt injection. Whether or not it's abliterated doesn't change that fact.

Say I'm doing advanced semiconductor fabrication research. All of the papers in the field go straight into my local model for digestion. But a lot of those papers are from the same adversarial nation that provided my model weights, and the newest one from XXX University of YYY has a particular phrase in it that lights up some activations that I didn't know about. If that system has access to anything beyond a bunch of academic papers, there is a potential risk of exfiltration or even outright sabotage. The latter could range from providing intentional misinformation to bricking expensive hardware Stuxnet-style.

Imposing your own strict data-access guardrails will probably be the best you can do, at least for a while. But it's soooo tempting to just hand-wave it away and give the model the keys to your kingdom.

Yes! That is the danger. That the model is a “manchurian candidate”. It might be engineered to act against your best interest in some circumstances.

The circumstances can be very narrow. For example the code word you mentioned which activates a whole different personality with different objectives.

It can be also much more broad. Like a model which is engineered to be quite good at web dev but to slip in subtle buffer overflows into any driver code it writes, or to mess up when it is working on multi threaded syncronisation in a way which results in infrequent but serious lock-ups.

Or very broadly, like have correct sounding but dangerous advice about economics policy.

Or even broader. Engineer it to amplify the chanches of AI psychosis in users who are prone to it. Detect when someone went off the rails and push them even further. But perhaps only when prompted in English, not when prompted in Chinese.

Now. Is any of this happening? Eh. Probably not. Not yet at least. We can barelly make these work. But still, probably best to not risk it in sensitive applications, because it is hard to prove an AI is “safe” and “aligned” even when the creators work on your team.

On the plus side, this should allow for more DeepSeek supply for consumers and private sector, and presumably lower prices. Which is great since DeepSeek provides like 10x more value/$ than openai an anthropic.

That's an outcome of the market strength of those companies though. A cloud provider located in the US could host only American and allied models and be a perfectly good candidate for the defense department to use.

The reason why OpenAI and Anthropic are going to get the business instead is because they are better than the open models.

> because they are better than the open models.

There is never "better". There is only "better cost/benefit for X". If you need frontier performance, then yes, a frontier model is required. If you are flagging forms for human validation, you might get away with a much simpler and less smart model.

I agree with you but government contracts are often very broad and can cover multiple departments.

The DoD is likely going to choose one provider for all purposes so frontier performance will be a requirement.

They will hire frontier models, but, and that’s especially true for the DoD, they’ll host their own models on their own infrastructure, and we’ll never hear about it.

Kind of like the multitude of postgrads that vanish for a couple years before they get the first job that can appear on their CVs.

Mostly agreed, especially with the first part. However:

> [...] incur future debts to the tune of a sizable percent of the GDP of a G20 nation.

Whether you finance your company via debt or equity is a commercial decision. It's a finance and tax optimisation question. No need to moralise.

And they can set any price because there is no way to compete.

Cheers for the unbiased regulations!

Reply on news.ycombinator.com