Far as defects, problems people report are definitely a start. Fagan Software Inspection Process had the nice idea of standardizing in a list the kind of issues in code that would be treated as defects and fixed by priority. They could be lack of a bounds check, code style issue, whatever team thought was important. So, my definition for defect in such a scheme was "unacceptable code."
"Low defect potential." Interesting metric. Often a function of complexity, coupling, amount of state being thrown around, and handling of interface assumptions. Scientists should come up with ways of assessing stuff like that then see what effect it had on defects during changes among many codebases. Of course, the minimizing f_n stuff should help here.
"When you find a defect in production a week after deploying the code, it is probably still rather cheap to fix: All the original developers are still there, they still remember what they did, the documentation is still accurate."
That's actually a really good point. I don't remember seeing anyone else say it. I'll have to remember that.
"Well crafted code is self documenting. When I read the code together with it's tests, I want to be able to understand what is going on. Without reading some external documentation (wikis, ...)"
Well-crafted code section is overall good but I'm not sure about this. I've seen code that can work like this. Yet, I've also seen high assurance work where there were abstract specs of what it does w/ pre and post conditions that were easier to understand for the use rather than modification of a module. Could also be fed into analysis tools. Some modern languages include that stuff in the code itself and can export interface documentation.
That's not even getting into issues like consistently using a filesystem where all kinds of extra commands are needed for non-obvious reasons that need a whole article to describe. A 1 line operation suddenly takes more like 10. So, I'd say make the code self-documenting where possible but there's potentially good reasons to have the other things. If they're needed or useful.
"Good software design is, in my opinion, less subjective than some other terms I described above."
I agree. You did alright explaining that.
"It's Not Much Slower Anyway The Microsoft study on TDD"
You're conflating quality and testing here. They're not the same. There's plenty of low-quality software with lots of testing. Likewise, Cleanroom (extreme example) normally resulted in high quality from app/code structure and verification via code revies before testing. Same with Fagan inspections and OpenBSD's auditing process. All of those had vastly higher quality than TDD projects far as I've seen. And that's without arguing against testing, as you know I'm for it.
Maybe renaming anything like that in your articles to specifically focus on benefit or cost of testing. So, this would be "TDD/Testing is not much slower anyway." The arguments you made for it are good and actually apply to all quality techniques. I think you intended that but set it up wrt to testing. So, maybe it's not the title and just something about that section in particular. Sounds nitpicking but my reaction is more that we need to always convey quality as a combo of prevention, code review, and testing. Code review & testing at the least. Gotta keep putting it into their heads until they can't forget. ;)
"Let's look back to "Minimizing F_n (For Large n)":"
"Executable specifications (a part of the test suite) make sure everybody understands the current functionality that is implemented. And that "documentation" cannot be outdated, because otherwise the tests would fail."
All good advice but I counter that this long-running problem is a management and SCM/VCS problem. One can implement a policy and rule that any change to code requires an update of the documentation. Then enforce that. A test suite is also a good idea but not a requirement for dealing with this problem. Not even enough as other verification methods (eg DbyC, static analysis) were needed to catch problems in this area that testing misses.
"At some point, your users will recognize that it takes longer and longer until they get software that contains the new features they requested. And they have to pay more and more for it."
A really, good point that few mention or argue satisfactorily. Should be one of the top points. Need more ways to illustrate this in numbers or visually so less technical people get it. Maybe just delivery time and cost themselves as a graph going up over time. ;)
"putting it all together"
Strong, sound conclusions. The science supports it up to a certain quality point where ROI diminishes. You're advocating a realistic one, though.
"A former colleague once said TDD would not make sense for them "because our iOS app only consists of a user interface and some server calls, and you cannot really test those. There is no logic in between. And it's really easy to test manually." Well, maybe you can get away with it when you have an app like that."
You can counter that software is rarely that simple. There's assumptions built in to various component, failure modes to account for (esp network), and effects of state build-up if it's isn't fully stateful. Hell, tell them a non-bloated web server is "simple" with an "interface and some server calls." Then show them the release notes detailing all the bug fixes. Murphy's Law says the "Simple thing are always hard." Not always, but often enough to justify verification activities.
Just popped in my head that it also helps when platform (eg iOS) breaks something with a change and you didn't know about it. Proper tests can isolate that immediately.
"Also, your actions now will come back and bite you later."
Very important. Most "short-term" jobs end up lasting a long time and even being reused in some way. Besides, getting in habit of doing it right makes doing it right more effortless. The delay between short-term and long-term should get smaller as verification activities become natural. I also like "unlimited downside" concept haha.
"And in many organizations, you will have a hard time to argue throwing away "perfectly working code" when the time pressure gets bad..."
Always worth remembering, too.
Overall, when I'm more thorough in the review, you have a good write-up with only a few things I take issue with. As you predicted. :)