Falsifiability and Unit Testing
codingismycraft.com
codingismycraft.com
Given enough parameters (or complications), it's possible to fit any set of known data points.
What's missing, to be falsifiable, is to pose a set of questions/tests which you expect to succeed if the theory/software is working. These might include new edge cases, scalability testing, and performance testing that fit better outside of TDD, which focuses on fast, short tests, and especially outside of red-green-refactor, since these validations tests are supposed to be green-green-green and never red.
I, for example, have a set of tests which require loading >4GB of data, to test that I didn't use a 32 bit offset in places where I should have had 64 bit offset. These tests take minutes to run, and are implemented as system tests through the command-line API, not unit tests.
> Following this way of delivering software, TDD is one of the practices that is usually overlooked, as inexperienced and non technical PMs, like to believe that writing comprehensive tests is very time consuming and it does not really add lots of business value in the final solution!
To address that point more directly, "testing" is much more than TDD. I believe after-the-fact tests are more akin to tests for falsifiability than TDD is.
The ultimate test for many programs is, do users use it? (Or "do people buy it?") That certain is one way to judge the validity of the program, with a way which is outside the TDD framework.
The software development process starts with the definition of the problem and its desired answer. The code is a finite state box that claims that it meets the expectation of receiving the problem and return the answer while the purpose of testing is to verify this claim.
>> To address that point more directly, "testing" is much more than TDD. I >> believe after-the-fact tests are more akin to tests for falsifiability than >> TDD is.
I agree. Still, these after-the-fact tests belong in the infinite universe of tests that they were not identified originaly and can now become part of the testing suite. What distincts a good from a bad tester, is his ability to minimize the need for these "after-the-fact tests".
That's precisely where the metaphor breaks down. For a physical theory, we don't have a desired answer.
The desired answer that motion in the universe is made of circles does not somehow turn ellipses into a set of epicycles.