Go Assembly Mutation Testing

words.filippo.io

40 points·Metalnem··3 comments

Code coverage not being meaningful because all instructions are always executed in order to make the crypto function constant-time is certainly a niche issue! It’s great to see such rigor going into Go.

Not assembly, but one of the side benefits of Go’s infamously verbose error handling is that code coverage generally tells you when you’ve never tested error cases. It gives a more fine-grained picture than languages using exception handling (unless you try/catch at a similarly fine-grained level).

The features of Go's error handling that people don't like aren't necessary to achieve that. In any language that requires a function to declare in its signature whether it can err (either through result types or through checked exceptions), code coverage tooling should be able to detect and report all untested error cases.

That said, I'm not sure whether actually existing coverage tools for those languages do this. I'll check and see if I can find out.

Both branches are executed and the result merged, in single basic block fashion, which the code coverage tracker doesn't understand.

OK, sure. That could be fixed at the code coverage level - you're looking for something along the lines of store the select mask (as opposed to the bit branched on).

Could report it in similar fashion to the mcdc coverage, would look a bit like check each bit in the word is set or clear on some execution. Probably a reasonable extension to implement.

Reply on news.ycombinator.com