Hacker News Reader

Comment by vitiral | Hacker News Reader

> SQLite’s test suite is proprietary

This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in this way, but that's a whole other can of worms.

View on HN

Well, Java during its initial life was controlled to a degree through the control of the tests: https://en.wikipedia.org/wiki/Technology_Compatibility_Kit .

Huh, I wasn't aware that Java was initially open source

I think Java/JDK was closed source initially, then went open source in 2006/2007 (?), but without the TCK. The TCK was never open sourced but the JCK is now kind of "open": https://openjdk.org/groups/conformance/JckAccess/

They have a test suite that is part of SQLite3 then public domain product, and they have a much bigger and better test suite that is proprietary.

they do not fully own said proprietary sql test suite. they've licensed it. that's why they can _run_ it but not publish it or share it. That's at least how I remember Richard Hick describing the situation at a talk.

It could be simply to prevent forks, but if it really is 100% branch coverage, why do they still have memory safety related CVE coming out? With asan turned on, and full static analysis, that should make such errors exceedingly rare. Part of the benefit of rust is that it makes coverage both easier to get due to its type system, and less necessary because of the guarantees it makes. But if they really went all the way to 100% branch coverage that should be almost as good if all the samitizers are running.

They claim 100% on https://en.wikipedia.org/wiki/Modified_condition/decision_co...

However, unless you can guarantee that every branch tested has been covered for all possibly relevant application states, that does not preclude CVEs.

Large chunks of the test suite are open source, committed to the repo and easy to run with a `make test`.

Everytime a bug is reported in the forums, the open source tests are updated as part of the bug fix for everyone to see.

There's a separate test suite that offers 100% coverage, that is proprietary, and which was created for certification for use in safety critical environments.

HN loves to discuss business models for open source, but apparently has a problem with this one. Why?

Reply on news.ycombinator.com