Wouldn't a good and robust configuration management plan overcome this problem?
Wouldn't a good and robust configuration management plan overcome this problem?
In some of the plans I have seen, some deviations are tolerated but those deviations are spelled out in excruciating detail.
How have you dealt with this issue?
If I'm debugging a production issue and want to create a debug build from the same source to step through things, it would be nice if I could just build both debug and release, check the hash on the release to confirm it matches production, then start debugging, without reference to any external records or other systems.
Reproducible builds reduce the number of links in the chain needed to verify what you're really doing.
...but what's really surprising is that it's surprising, since everybody is experienced with details in some context.
I mean, my response isn't entirely a rhetorical question. If there's a reason it wouldn't work I'd be curious what makes his situation different from mine.
This should have been the first question to ask yourself before trying to give advice.
I hope we agree that justifying these deviations can be a lot of work and having reproducible builds would be easier.
High-assurance systems is my thing btw. There's a few ways we approach this issue. One is a certifying compiler (eg CompCert C) that's verified and tested to basically never screw up. CompCert got SIL qualification recently with Solid Sands partnering to validate them, too.
Another I like is equivalence testing across multiple binaries from multiple compilers with a large, automatically-generated test suit and fuzzers. Such test generators are already great for bug hunting. You just rerun them through the same app compiled with other compilers. Helps catch app and compiler bugs. If you have CompCert, I have another trick: re-compile with CompCert, re-test each case that failed, and any that suddenly pass were likely compiler bugs. The multiple compilers approach should catch them, though.
Now, I should point out that what reproducible builds is addressing are often changes in hashes that aren't changes in functionality. Just stuff like timestamps or symbols. You should be able to show the regulators that the only thing that changed was that kind of stuff. Maybe we need tools that automate that, too, showing diffs annotated by what does or doesn't affect execution.
Anyway, what exactly do they require of you for source-to-object correspondence? And is this U.S. F.D.A. or a non-U.S. regulator?
In my case this is not really clear. You have quite a bit of leeway but you never know when you will get rejected. We had things that are approved a few years ago being rejected now because some rules have been tightened.
With reproducible builds you could save a lot of time validating build chains. It would be a huge timesaver and probably also allow us to be more creative with the build chain because if the binary is the same you know it worked.
If I have deterministic builds then the user can verify that the build matches the source. Otherwise it's potentially compromised (or the wrong version).