Comment by nunuvit | Hacker News Reader

Comment by nunuvit | Hacker News Reader

One of the most important lessons I've learned in my career is that if a common problem has existed for a long time, a simple solution has probably been hiding in plain sight for a long time somewhere you haven't thought to look. I don't know anything about productivity research, but the fundamentals of defect density were figured out decades ago [1].

> (a) there’s no difference between test-first and test-after but (b) interleaving short bursts of coding and testing is more effective than working in longer iterations

I'm glad you quoted this study because it's a perfect example of a conclusion that should have been the starting point of a more interesting experiment. Interleaving code and test is known to be the most impactful factor. TDD and TRL don't differ in any of the ways that account for the vast majority of defects. Therefore of course the difference should be small, and of course shorter iterations should be better.

[1] https://www.slideshare.net/AnnMarieNeufelder/the-top-ten-thi...

Replying to my own comment to explain how it directly relates to the asked question - what to do with empirical research?

The key takeaway is to evaluate everything in terms of sensitivity because it gives you design insight. The above quoted study identifies one insensitive factor (order) and one sensitive factor (iteration length) of interleaving test and code. Now you can choose, modify, or design any test method as long as you do short interleaving. The order is something you don't have to worry about. If the research doesn't reveal anything about the sensitivity, don't worry about it until someone figures it out, your beliefs are probably wrong and inconsequential anyway.

See, for me this is an example of how the hyper-empirical, sciencify-everything mindset is great for getting at truth but less great at giving advice.

The default, intuitive approach is to write a whole bunch of code and then test it. When people start doing TDD, they struggle with only making enough changes to the code to pass the test. But it helps a lot that the test suite tells them when to switch from writing code (which they prefer) back to writing tests (which they don't prefer) by passing.

Then we discover that the main reason TDD works is that it gets you to interleave coding and testing in small batches, and it works just as well to reverse the order of the batches but keep them small.

And then, somehow, knowing why TDD works (better than what people do by default) gets translated into "TDD doesn't work" (better than something that's carefully controlled to be exactly the same, except for the part that gets people to do the rest of it). And most of the people who hear that go back to writing a whole bunch of code and then testing it.

Because the real world doesn't control all the variables, so we often have to think about factors that the rigorous research is silent on because they can help or hinder us in achieving what the research says matters.

As with any research, there are many possible audiences and it's not an article targeted at you. It's your job to already be educated in a way that's relevant to yourself.

The link I posted shows that the research has pretty comprehensively determined the relative impact of various factors, and I gave an example of how to interpret new research providing single data points within that framework. So now you should have the background that you need to read the literature with an eye for your application.

Reply on news.ycombinator.com