math.pow(2, 4) vs math.pow(4, 2)
math.pow(2, 4) vs math.pow(4, 2)
Properly input space partitioned tests have something like 90 to 95 accuracy if properly written.
The issues always come from people not wanting to add sufficient tests.
Continous Integration is mostly about regression, so the new tests have less value than running old tests.
We tend to fix issues in production (bugs) by mandating a unit test with the failing issue to be written as part of the fix.
Properly, if your code has an ‘if’, you need two tests, for the two branches. Same with every place the outcome may diverge. With this approach, it's basically impossible to botch the code unless something slips your mind while writing both the code and the tests. Otherwise, it's pretty much ‘deploy and go home.’
if (x % 3 == 0)
println "Fizz"
if (x % 5 == 0)
println "Buzz"
There, solved it! And it works fine in all cases you told me to test (e.g. 3, 4, and 5), which means it's impossible I botched anything! Surely I nailed this interview?Seriously though, the criterion you mentioned is only one of many of increasing strictness (see e.g. https://en.wikipedia.org/wiki/Code_coverage#Basic_coverage_c...), namely branch coverage. Having branch coverage still says very little - the interaction between different branches can be trivially wrong. And desiring full path coverage immediately leads to combinatorial explosion (and the halting problem, once loops are involved).
> unless something slips your mind while writing both the code and the tests.
That is true for any choice of coverage metric and target.
Well, if you don't see other points where the input→output mapping can diverge (and rely on the client's requirements for that?), then no, you didn't nail it.
You normally cannot prove general correctness with unit tests. You can try to probe interesting points in the input space, and different coverage metrics encourage different levels of rigour and effort with this, leading to different fractions of bugs caught, but you'll never have a guarantee to catch all bugs (short of formal proof or exhaustive input coverage).
And if the same unit has multiple branches, you need double the tests to cover all paths.
And that doesn't guarantee correctness, and it's only unit tests.
But the 50% was a number I vaguely recall from Code Complete.
Yeah, I should've probably clarified this, seeing as we're on the topic of correctness and pedantry.
Personally I'm doing this in TDD style, before the code is written. And then while it's written, too. And for code that didn't have tests—the approach really helps catch bugs previously unknown.
I don't think a unit test will be able to distinguish 16 from 16.