Hacker News Reader

Comment by jacquesm | Hacker News Reader

The models may well contain malware or security vulnerabilities. But they are at least open to inspection because the weights are available, the same can not be said about US providers of inference such as OpenAI and Anthropic.

View on HN

I don't know if inspection would be practical. How do you inspect the weights for an intentionally hidden trigger that writes malware?

I don't have the answer to that, my simple solution to this problem is not to trust anything a model ever writes and so far that strategy has worked very well.

Why write that comment in the first place when you know that inspection of model weights is not a thing though?

Inspection of model weights is a thing. But that thing is out of my wheelhouse so I do the next best which is to judge the output.

It is absolutely not a thing in any practical sense that relates to detecting post-trained backdoors, no.

Or provide a source showing how to do this (and not just a circuit paper showing which activations light up with the word dog)

> It is absolutely not a thing in any practical sense that relates to detecting post-trained backdoors, no.

> Or provide a source showing how to do this (and not just a circuit paper showing which activations light up with the word dog)

The fact that you're not informed does not place a burden on me.

https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...

This is all just a sub-class of supply chain attacks.

You don't have to be able to explain every aspect of a model to be able to inspect for undesirable traits.

Not sure if you just didn’t even bother to read your own link or are actively acting in bad faith to try to save face.

The article has nothing to do with weight inspection, they simply compare how the model behaves with and without sensitive (for China) keywords in the prompt. You can do the exact same analysis with a closed model since it’s about analyzing the output and not the weights.

You can't because there the model could be switched out from under you and with open weight models that's not the case so the degree to which you can inspect the model is much higher.

So nothing to do with weight inspection at all, got it. Have a good day.

You don't. You handle it the same way you handle security holes which every LLM (including "safe" American ones) will put in your software sooner or later -- by understanding and reviewing the output.

Glad someone is pointing this out here on this community. I keep reading about people "solving" the problem of LLMs outputting too much code that's hard to review by not doing human review at all!

You can do that for non-critical code. But code has a habit of being repurposed and sooner or later your non-critical original is used in a very critical application.

They're about as indictable as binary blobs found in your BSPs or the 3rd party drivers you get through Windows Update

Still better than a Windows SaaS edition

It's also odd for them to single out DeepSeek in particular, and not just any model from the PRC?

DeepSeek is the one that is the most visible, and probably the most cited in the departure messages to US providers.

Because it is DeepSeek in China, they must have open weight models to be even partially acceptable to the world. I am sure many would balk at using a Chinese company if its products were not open source

Ridiculous statement.

Best greetings from my Oppo Find N6

- Sent from my device containing a bunch of closed source products from a Chinese company

Look around you.

You'll have a hard time finding electronic devices not made in China. And all of them have software which is harder to monitor and debug than LLMs.

So I don't think distrust for Chinese LLMs will hold.

I think Chinese models deserve the same level of trust as US models: none.

Not only open to inspection, you can also post train it to get exactly what you need from it, both in terms of capabilities and alignment.

Model weights being open to inspection means absolutely nothing. What are you going to inspect in a bunch of numbers?

Reply on news.ycombinator.com