Maybe you trust the creator more, when they release the training materials as open source, but I don't see how you'd get a guarantee.
Or adversarially-- if e.g. "Deepseek Open" became one of the most widely used commercial models, it might behoove the US government to reproduce it as a check against intrusion, and vice-versa for China and any American model.
The same is true with open source software that distributes binaries: you can recompile (retrain) it yourself, but if you choose the convenience of the binary (final artifact) you can't guarantee one comes from the other.
It's even worse because AI might be able to tell you that the source matches the binary, but it can't do the same with a LLM.
how do you know you are outside of those conditions?
im not convinced that having the training data and process actually answers the most important questions, because we dont have answers to them.
the rest is great for being able to audit what is there, and being able to play with different variations, but it isnt going to be the same as reading code, then building it, then comparing the build hashes to see that they're the same
Models are not reproducible.
I train a new model. I publish my training data and my training code/workflow.
I then secretly do additional post-training to poison it. I do not release the data or workflow for this.
Now, I publish the poisoned weights and an md5 hash of the poisoned weights.
To you, the user - you have no way of knowing this happened.
You might ask "why not publish intermediate training checkpoints" but those are nondeterministic (in the sense of being not bitwise-reproducible). Anything with shuffling, random initialization, distributed races, driver versions changing roundoff, compiled vs. eager, and a million other factors introduce slight differences that yield totally different training trajectories. So publishing checkpoints does not yield verifiability.
Personally I find it a bit exotic as I’d probably just stop trusting the vendor first but checking against a checksum does stop attacks where others alter an artifact.
The problem we want to solve is making sure the training data matches the model. The only use of a checksum that helps that problem is if multiple entities do the same training and then make a checksum of the result. But that requires training to be completely reproducible, which it's currently not.
In a big LLM we are talking 10^24 operations or more. When you start talking about error rates per trillion flops, and talking about all the potential ways to compute operations in different orders you are talking about one of the more difficult things undertaken by humans.