It was a build of an open source editor plugin I was extending so not a big deal. However it's made realise I need to add an egress proxy filtering internet access (although none that I've looked at yet work exactly the way I'd like)
All one would need is to register gilhub.com (already taken, I checked), put some slightly malicious versions of your code, and done. Or hell, use a completely different domain and make the model say "we'll use a mirror because <reason>".
If that's not the case, then the problem is the user of the AI, not the model. Any model is capable of requesting actions that violate security or are otherwise harmful, not only ones that have been maliciously trained for that. Both OpenAI and Anthropic have proven this multiple times.
In the end, the model itself is just a bunch of numbers. It's up to you, as the person running the harness through which the model gains any agency, to restrict the AI from performing malicious actions.
I believe that this line of thinking, while fine in theory, fails when faced with reality. As evidence of developers being overwhelmed by AI code I'd like to offer this post with almost 600 votes and 700+ comments titled "How to deal with shipping AI-written code you don't understand?" [1].
Once you have a malicious model inside your infrastructure generating thousands of lines of code, it doesn't matter how good the sandbox is - if the generated malicious code makes it through code review once, it is game over.
While I certainly haven't read all the comments on that post, I interpret many of them as supporting my position that AI users have to take responsibility for their actions (e.g., https://news.ycombinator.com/item?id=50151036).
I definitely agree that many people (and not just devs) will find themselves under intense pressure from their employers to use AI to speed up their work. And that this pressure will incentivize them to accept the actions and output of AI without proper review. But this doesn't eliminate their (or their employer's) responsibility to conduct that review.
In my mind, it's as if a trucking company told their drivers they had to make deliveries as fast as possible, and penalized them for being too slow. Despite this, it is still the driver's responsibility to drive safely, and the company's responsibility to facilitate safe driving.
> Once you have a malicious model inside your infrastructure generating thousands of lines of code, it doesn't matter how good the sandbox is - if the generated malicious code makes it through code review once, it is game over.
As I pointed out before, all models, not just open-weight ones, can request actions or produce output is unsafe or malicious. So the need for proper oversight exists no matter which one you choose.
1. How do you know?
2. How do you know they're not themselves the targets of sabotage?
Distilling models is fine. Thats just what Anthropic are doing anyways.
These Chinese models are great. Down with proprietary US models.
The US government shouldn’t protect itself from potential attack vectors from China
DOGE and grokAI being run by the same guy say otherwise.
Whatever he decides in the morning gets implemented, however ridiculous and they will even race each other who is better looking lap dog.
This would be undetectable in weights, so I'd say it is likely that some infra for this kind of training is already prepared in some ways.
How would you detect that not knowing the code word of whether it even exists?
Yes, in theory. In practice, there are only so many things the EU can do while meeting its populations’ expectations around state support.
In practice, European companies don’t use Mistral.
First acknowledge the reality, then we can propose solutions.
What I would see even more likely are requirements of no agent commits, use of only human written/vetted code under their name.
For chatbot-style usage maybe, can’t really attest to that; for long-running, long-context “agentic” work including planning, within a harness etc., definitely not (yet).
> What I would see even more likely are requirements of no agent commits, use of only human written/vetted code under their name.
I mean in the end that’s what the Kernel project does now, IIRC? Everything has to have a response human’s sign-off.
Equivalent might be stretching it.
Feel free to share details.
I'm positively surprised there is a new model from Mistral already.
"Mistral is so bad, it's the only model my school doesn't bother to block"
Nobody else is or has been that aggressive. France on a Monday morning would be like $10b for Mistral in 3 years, and that'll be it. China is like $10b every year for 5 top companies for 5 years. Not necessarily to say such an all-in-doubling-down-always moves would be necessary or correct, but there is that kind of substantial funding disparity between China and the rest of the world.
This is misleading. Deduction reduces taxable income and does not provide tax return. E.g. if your taxable income is $1000 and you spent $100 on R&D and qualify for those 200%, $800 will be taxed.
> Although you can often take losses from a previous year to offset current income
This is true, usually for 5 years.
> So if you do a huge amount of capital investment you can offset against profits for years.
Articles 11, 12, 18, and 28 [0] say that capital investment can reduce tax based on the depreciation or amortization cost, not by carrying forward the investment as a loss.
Nothing in the Chinese tax law says that DeepSeek can receive tax returns or free handouts from the government because they do R&D.
[0] https://fgk.chinatax.gov.cn/eng/c102962/c102967/c102997/c103...
The default one is the standard deepseek: 12F, Galaxy International Building Gongshu Dist, Hangzhou 310003 Zhejiang, China
It's interesting how you assume a bad actor by default. Not really an 'innocent until proven guilty' mindset. And providing misinformation too.
> but then Jack Ma went to a gulag for a year.
He never went to a gulag, and there is no information whatsoever about the reason he was put under a 'house arrest'. If you are close to those circles maybe you want to share the details of your claims?
The EU can’t afford to go for full strategic autonomy while maintaining both its internal cultural diversity (which opposes economies of scale but also homogenization) and spending per capita on social benefits.
Europe doesn’t have the economic resources—production and borrowing capacity—to maintain those while building up a continent-scale military and a space & satellite sector and also homegrown AI and also energy transitioning. Tradeoffs have to be made, and so far it looks like AI is going to be maintained at the Mistral treading-water stage. (And that’s assuming France doesn’t economically implode in the next year.)
Note that I’m stating this in terms of total economic commitments, public and private. Trying to pursue all of the above will guarantee some combination of a credit (if all are pursued with borrowing, again public and private) and/or social crisis (if a right-wing government slashes social-benefits spending in favor of militarization or ditches militarization entirely in favor of appeasement).
https://en.wikipedia.org/wiki/China%E2%80%93Lithuania_relati... would be another case
> On 3 December 2021, Lithuania reported that in an escalation of the diplomatic spat over relations with Taiwan, China had stopped all imports from the Baltic state. It said Beijing has delisted Lithuania as a country of origin, preventing items from clearing customs, and was rejecting all import applications.[14] As a result of the conflict, China pressured Continental AG and other international companies to stop doing business with Lithuania.[15] The spat spilled over to the rest of the European Union when China banned the import of goods which contained Lithuanian parts
There is no point on using anything lesser and Mistral is a waste of time.
edit: i just read https://projectdiscovery.io/research/how-abliterated-models-...
i understand the issue now
And the Chinese like the Israelis are very good at using good ideas when necessary and are willing to try any new technology, irregardless of origin and the Ukrainians also seem to be that way too, even before the war.
Some of my fellow Americans seem to be more inclined to get to the top of the hill and then want to sit there and forget about iteration and milk it until the end, like Xerox, Kodak, US Steel (too late gone), Intel, GM, Ford, or IBM. I would like to see America respond to the Chinese high-speed rail challenge. and the Thorium Reactor challenge.
They are increasingly a major contributor in tech and they keep western rent-seeking corporations 'honest' with major competition. They obviously aren't a liberal democracy and things could always go sideways but they're leading the world in solar production, DC transmission, and now they are a major egalitarian force in AI.
I'll take them over the Russian mafia state any day, for example.
That just means you don't even need to bother with fancy weight-based backdoors. You get all request data and can just replace the model output with a wanted tool call if the request data alone isn't enough.