Why Developers don’t TDD: a podcast series | Hacker News Reader

Why Developers don’t TDD: a podcast series

agilenoir.biz

43 pointslancerkind106 comments

Starting at episode 14 : https://agilenoir.biz/series/agile-thoughts/

I've been a developer for many years, so most of the time I kind of know where I am going and have a map in my head guiding my development. I like test cases to help me with refactoring, but full TDD seems to slow me down when I try it.

I have found that TDD helps quite a bit when coaching a new developer. It sets up intermediate goals that are not as daunting to get to. Sometimes thinking about a big project can just seem overwhelming to someone that hasn't done it before.

I like your points about when working with a new developer and how it creates intermediate goals. Are you Pair programming when doing this?

Sometimes, but more often just looking over other people's work or progress when I'm asked for help or advice.

Because I find that I do not fully understand a problem and it’s potential solutions until I am working through it in reality.

When Kent Beck (credited with helping create TDD) talks about TDD in his book, that is the primary reason for following TDD. He mentions, that if a developer already knows the solution, just do it. TDD is not helping at that point.

I used to have that view of the world as well, that a lot of the programming I was doing was simultaneously exploring the problem space as well as working towards a solution and thus writing tests (let alone full blown TDD) wouldn't work.

I think that what changed my mind, other than the dubious joy of maintaining my own code a couple of years down the line, was the realisation that while I don't know the solution to the problem yet I do know what each function I'm writing is supposed to do and thats what I should be testing. As a side effect it also meant that the functions I was writing became smaller (easier to test) and it became easier to see what the end solution might look like because I could understand the intermediary steps more easily.

Or to put it another way, I might not know how to write the compiler, say, but I do know that I'll need to start off by reading a line of input and I can test that I'm doing that properly, etc.

Good example: writing a compiler. I started (but never finished) writing a WASM interpreter in JavaScript. (https://github.com/bgard6977/wasm-int)

My test suite had simple programs in C, and used emcc to compile and run them, then compare the results from a known-good implementation to the one I was writing.

As I went, I would add one new instruction to the program at a time.

That's a perfect example of outside-in testing, and evolving the requirements, tests, and code simultaneously.

The test case programs/expressions input into a compiler are external regression test cases. By itself, that is not TDD.

External compiler tests may seem TDD-like because they exist first, and make the compiler fail until the required features are working.

TDD is about bogging down the code with internal tests that treat every aspect of the code as a committed feature.

You cannot do a single thing that doesn't first fail some newly added test, which then goes green.

I call this "unit test driven development". I find it pretty horrible unless the code is largely algorithmic.

ATDD works though.

>As a side effect it also meant that the functions I was writing became smaller (easier to test) and it became easier to see what the end solution might look like because I could understand the intermediary steps more easily.

I definitely use "it's hard to test this code" as a smell indicating refactoring is in order.

I've never tried TDD, but I suspect most of the value is that it forces people into practicing separation of concerns and other basic anti-ball-of-mud practices.

This is true, but you still go through a painful process of learning what that actually means.

My early sins included happily relying on field injection in Spring (don't), happily blasting stuff into Ruby objects (don't), mocking my software's world into an unrecognisable comic book universe of gravity-defying nonsense (don't) and so on.

A common criticism of TDD is that poor tests make things harder to change, not easier, and I believe it, because I have absolutely done that.

One thing I have found helpful in this sort of situation is to start by writing the exploratory code as a self-contained test.

So a test that has no production code it is testing.

Preferably in a language with really fast turnaround, a "scripting" language.

After a while, I have a self-contained test that does something, and does it reliably.

I can then refactor the test by extracting the production code.

I do this too in cases where I’ve got an “extreme” amount of learning to do. (Learning a new language, a library, or something I’m trying to make work just simply won’t do I break it down into a tiny test).

This isn’t my usual TDD workflow but I use it when my understanding of the mangauge or language feature are blocking me.

The best way to explore solution space I've found is to make the simplest thing that works end to end then add addional cases. The reason that TDD doesn't work well is that it tends to be done bottom up which often has you building things you don't need, use, or worse use but with a poor fit.

So skip TDD if you know exactly what you're doing or have little clue. Use it if you want dopamine hits to motivate continuous progress on average complexity tasks. If it comes out wrong, commit a checkpoint locally, but attempt a refractor now rather than repeatedly paying the debt.

Having just written this, I can recall a number of times I've pushed the debt rather than follow my own advice. It's due to having written tests that must also be refactored (i. e. debt) and pairing where I don't apply as high a standard to keep moving forward.

The ideologies that developers publicly claim are often described as religions. One way they resemble religions is that people who espouse them don't live the way you would expect them to if they really believed in them.

XP (and by inference TDD) is often described this way.

But after one has practiced it for a while, it's hard to go back.

The way I describe it is this: When most people start as programmers, they write a big 1000 line file that should completely solve the problem. Then they run it, realize it was very wrong, and spend hours debugging it.

As programmers learn, they tend to practice "error driven development": they get an idea what they want to happen, they run the program (usually "hello world" to start), and slowly run, edit, run, edit until it evolves into what they want to see.

If you practice 2nd process above, you are one step away from TDD. You are still testing, you are just using manual tests. Once you learn to automate those tests easily, why wouldn't you TDD?

I've done TDD for years now, and see how my peers do development (non-TDD), and I would definitely agree.

Is it at all possible that different people like different things, or maybe think differently, or perhaps find different workflows to be productive? If you could buy any of those as anything less than totally preposterous, then you may be able to make the next jump to considering why people like a development practice that differs from the one your prefer.

I like tests as more of a 'checkpoint' thing, myself. To each their own. Also, I spend most of my time in a REPL, which I suppose is pretty close to what people tend to get from a TDD kind of loop.

> Is it at all possible that different people like different things, or maybe think differently, or perhaps find different workflows to be productive?

Yes. But I don't think that's entirely down to innate or inborn characteristics, I think it mostly comes down to experiences. I tried to learn TDD from Beck's book and walked away thinking it was nonsense.

Then I went to work at Pivotal Labs and I got a chance to learn it with people whose job consists largely of teaching it.

Neither experience is about my largely inherited characteristics. They were experiences. Both were valid in themselves.

I have experienced TDD in the good and the bad. I prefer the good, but I know why most folks consider it to be bullshit: that's how they've experienced it.

My experience has been that TDD tends to do a poor job of testing complex systems with complex interactions. If any of the assumptions the tests are based on change (or are incorrect/brittle) then you will silently experience failures.

Integration testing is the more valuable side of testing imo but much harder and slower. Also not amiable to TDD in my experience.

I don't think of TDD as being confined to unit tests and I was taught to test from outside in, meaning I'll have acceptance and integration testing as a matter of course.

It’s best to have both micro tests (TDD) and macro tests (Acceptance Tests, which if you write acceptance tests first, is referred to as ATDD). TDD refers to micro tests. ATDD is what you should say to communicate clearly what your doing. Saying TDD when you mean ATDD will cause confusion with those who have mature test automation practices. Macro versus micro are important distinctions as they need to be managed differently. Mix those two test inventories together actually causes damage as all your test infrastructure will be slow rather than having a micro test suite that’s faster and run more frequently. Check out Agile Thoughts episodes 1 through 8 for background on this. https://agilenoir.biz/series/agile-thoughts/

None of those tests will silently fail. That’s their point for existing. They complain when the code stops operating as the developer designed it to work.

Integration testing is valuable. It just isn’t scalable to the whole code base (because, as you said, it’s slow), that’s why macro tests should be targeted to the parts that can’t be micro tested (10-15%), leaving the bulk of the code to be tested with blazing fast micro tests. Agile Thoughts podcast goes over this during a series on the Testing Pramid, episodes 1 through 8. https://agilenoir.biz/series/agile-thoughts/

It’s not that the tests fail, so much as they become misleading. They still pass, but they pass because their assumptions are the programs assumptions, but their assumptions are wrong.

Because the focus should be on correct software and there's a sunk cost fallacy in arguing that you're almost there anyways so why not just buy in to TDD for some purported value.

It might help some developers, but it is not a panacea and should not be promoted as such.

Sometimes I like to rely on gut-feelings while coding.

Ever filled in that user-signup form for the thousandth time?

Ever start with nice input: John Doe, 555 fake St

Then after the 500th time you are just inputing:

aaa abc wsdfg

And clicking "sign up"?

That is the point you should know in your heart: "I missed writing an automated test somewhere".

If you can't relate to the above experience (and have never had your code broken by a coworker), then maybe for some reason you are in a position where you're fine without TDD.

There's also the cost of building the automation to bear in mind https://xkcd.com/1319/

This implies that a) you can't rely on "gut feeling" without TDD and b) that you don't write tests without TDD. Both of which are incorrect.

> When most people start as programmers, they write a big 1000 line file that should completely solve the problem. Then they run it, realize it was very wrong, and spend hours debugging it.

No, I start line by line, and tend to be thorough. When I start out there is zero complexity, everything I add to it I don't add blindly, with a "rough idea" of what it does. I know exactly what it does. I know where user input and other code ends, and my code begins, at the least, and how I normalize things that cross the threshold.

Yes, I'm talking about a solo dev making little things for their own needs, sure. I have the luxury of being as thorough and slow as I want, too.

It's not lazyness, the idea of tests actually excited me when I first heard about it. But for example, when I write a particle system, how would I test that? Do I test all combinations of parameters and then compare them with images, which takes a million years? How do I generate the images, with another particle system? How do I test that, in turn? It really does strike me as a "now you have two problems" kind of deal in that situation. In others, the code is just too trivial and unchanging to write a test.

What's a "real" application that has good and complete tests? Something like the GIMP, or Blender, or an audio editor, or a complex game? I wouldn't know where to look, and the tests I saw when browsing random github repos often seemed too simple to be useful, often just covering some "token input".

But what if your "escape string" function has a bug that kicks in at exactly 15 characters? What if your "get user ID from username" has a bug that ONLY happens when the string "whoops" is in the username? Fuzz everything? Who really does that for applications without serious security implications?

I'd love to know how to test graphics and audio programs, frankly.

TDD works very well for things with a defined input and output, including: 1. compilers 2. databases 3. all web apps (output is the DOM)

In regards to the escape string, if there was a bug at exactly 15 characters, there must be a branch somewhere that isn't being hit, and it should be pretty easy using code coverage reports to find it, and generate the correct input sequence to exercise it.

When you get to fuzzing, you're hitting the edge of state-of-the-art. There is good research going on at automating some of that: http://lcamtuf.coredump.cx/afl/

What I do for graphics (never had to test audio programs) is to make the actual graphics code as simple as possible - no logic, very few lines: just take this thing and put it on screen. This way, I can inspect it visually, test it manually once and be reasonably certain that it will work. (Which can be incorrect, of course, but it's relatively rare. Lost a contract because of it once, so it can also be painful :D)

Put all the logic in a class that can be tested and that returns something that can be used in the "get this, put it on screen" part mentioned above.

> But for example, when I write a particle system, how would I test that? Do I test all combinations of parameters and then compare them with images, which takes a million years?

Look up pairwise-independent combinatorial testing: https://github.com/Microsoft/pict

> But what if your "escape string" function has a bug that kicks in at exactly 15 characters?

Equivalence classes: Analyze which groups of inputs all exercise the same codepath, and test each group once. Boundary value testing: Anywhere there's a boundary between two different types of behavior, test the boundary, one less than the boundary, and one more than the boundary.

> No, I start line by line, and tend to be thorough. When I start out there is zero complexity, everything I add to it I don't add blindly, with a "rough idea" of what it does. I know exactly what it does. I know where user input and other code ends, and my code begins, at the least, and how I normalize things that cross the threshold.

TDD isn't explosively incompatible with being careful.

> What's a "real" application that has good and complete tests?

Cloud Foundry has 4 or 5 million lines of Golang, about a million lines or Ruby and I think half a million lines of Java. Pretty much all test-driven. It's been a while since I checked the SLOC figures.

Kubernetes is of similar (now greater and definitely faster-growing) magnitude and the community has tooling to track coverage (I think the target is > 90%), though TDD is not the norm. I've personally worked on maybe two dozen systems where TDD was the ordinary practice, both in Cloud Foundry and for clients.

But it's a rare practice. Just on the numbers, almost nobody practices TDD. I'm aware that this argument is a kissing cousin to No True Scotsman, but there it is.

> But for example, when I write a particle system, how would I test that? Do I test all combinations of parameters and then compare them with images, which takes a million years?

A couple of assumptions:

- your particle system is not one monolithic function, right ? - you also didn't just write it in a single pass and it had all features.

So:

(i am applying the test pyramid here ( https://martinfowler.com/bliki/TestPyramid.html )

- you start with a very simple (system) test that describes the minimal particle system. something like: given a particle system with particle count 1 and an emitter with probability 1 and velocity (1,0,0) and random seed 0; then after 1 step, there is a exactly one particle with velocity 1,0,0 at position 0.

- and you make this test compile and run and eventually pass.

- eventually you split out different functions; you develop also a test first. and then write the new function. and then change the original large function of the particle system to use the smaller functions. :)

- when it comes to rendering, you probably never want to write a system test that renders a full blown particle system with random paratemers ( because the test will likely be unstable unless you have a good idea about to keep it stable ).

- what is easier is to take the rendering functions and let them render aspects separately, i.e it renders a single particle at a known position. and you check the rendered image if it is correct; this because pretty simple with the image resolution is for example is just a pixel or 9 and that you can compare with a single static value. color animation can be tested that way as well. etc.

- thus in general even a particle system can be tested; break the system into small functions and test them separate and fully.

- then integration test the function in concert. use stable test scenarios.

- test manually; and when you find a bug (and you will). write first a test for the simple buggy function to expose the issue on unittest level. and then you decide if you want to add an extra scenario on the high levels.

Brilliantly said!

It’s a false equivalence to equate TDD to religions as the effects are observable and measurable with TDD. In contrast, none of my loved ones have contacted me from the afterlife nor has any diety ever responded to my exhortations.

TDD (I lump together lots of extended practices here) is hard.

It's hard to learn from a book or blog post.

It's often hard to solve the many problems of "how do we test <behaviour X>?".

It's hard to know when to stop spiking and start test-driving.

It's hard to stick to it when you solo and you think "oh I can just skip a few steps here and come back afterwards".

It's hard to learn mockist style and statist style. Hard to go all-in on one style, then all-in on the other style, and then afterwards often hard to strike the right balance between them.

It's hard to deal with the vast universe of technologies which, having not been developed in a TDD fashion, are hard to test.

It's also hard being told your years of practicing amongst practitioners aren't real experiences. That you didn't see the remarkable speed and confidence it granted on massive codebases with hundreds of engineers in dozens of teams working for multiple companies on several continents for years on end.

Tdd won't take off until developers realise they need to start thinking more.

If you don't understand the problem you're trying to solve well enough to write tests, you shouldn't start coding.

But too many developers just want to dive in and hack code together and try to figure it out as they go. The resultant code from this is almost always a tire fire.

Someone on hn a whole ago posted the quote "coding should be to software development what moving the pieces is to playing chess"

I couldn't agree more. I wish more developers would get this.

I don't do TDD because for the types of code I generally write, it is much easier to test the general cases back running the code directly. It's not that I don't have tests in mind when writing the code, it's that the tests I am working against are behavior driven, so it is easiest to just perform the actions myself and see where it falls down. Particularly in UI code, TDD seems to lead to code that is "technically correct", but produces a UI that is an absolute pain in the ass to do anything with.

Tests have their place, but the proper time in the dev process to write them is very much a personal preference thing and nothing more.

my experience is the exact opposite. ui code is where tdd shines the most because your spec is almost always 'when a user presses this button/fills this form/navigates to this page of the app do this thing'. there's zero uncertainty in the expected behavior so you can start out by writing exactly that test

Which is what I mean by leading you to "technically correct" code. At the end of it, you have everything wired up in a way that exposes the functionality, but you haven't necessarily spent any time testing out the ergonomics of actually using the UI.

Could just be the people I've worked with aren't good at UI design.

i mean your tests should cover the ergonomics too. that should be the first thing you test, in fact. if you can't describe how it should work, you shouldn't even be opening an editor

I do a lot of ui stuff and the sort of test I'd write for say a form and a save button would be:

1) populate the form with invalid data, click save. Verify that I showed an error message and it was structured how I want it. Verify I didn't call the database save.

2) populate the form with valid data, click save. Verify no error was shown, verify we called save on the dB layer and that the model we passed to save was as expected. Verify we showed the user a confirmation.

3) if we have required and non required fields, a test to verify each required field is actually required. If its not required verify the correct defaults are used.

4) if we are passing messages on save yo update other ui forms, then verify the message was sent and the underlying object or collection on the other ui element is updated with expected values.

I would expect these tests to run on every build, and on your dev machine on every save.

Otherwise you are just hoping devs arent breaking stuff... And hoping they test each case. Which for even a trivial form can be numerous. I'd definitly want this automated!

I also write those sorts of tests. I just write them after I am happy with the way things are working. Like I said, tests have their place. When to write them is just personal preference.

> Particularly in UI code, TDD seems to lead to code that is "technically correct", but produces a UI that is an absolute pain in the ass to do anything with.

TDD (or ATDD or BDD; unit testing tends to be removed from the level of UI concern here) is not a substitute for UI design, at most it validates that the UI correctly implements the specifications given. This is not a problem with TDD/ATDD/BDD for UI-related components, it is a problem of neglecting UI design.

(The same true, mutatis mutandis, of every other part of a system, too; development driven by building executable tests and the coding to pass those tests, in any of it's forms, is not a substitute for analysis and design but a tool to assure that implementation matches the design and does not do work unnecessary to fulfilling the design.)

Maybe that is part of the disconnect then. If some people either aren't responsible for the design portion, or don't include it in the "development" process, then I see how TDD could work pretty well. High level design has to come before specifics, and I don't think writing tests is the way to do that (unless you are counting things like acceptance tests, which seem like a different beast to me)

If you don't understand the problem you're trying to solve well enough to write tests, you shouldn't start coding.

Sometimes there are problems that are only really possible to appreciate after one starts coding. In many instances, I've found that attempting to build a prototype is the best way to understand the problem. In those instances, TDD is a huge mistake. It ties you to architectural decisions that you know you're going to change when you better understand the problem.

Contrast Ron Jeffries' attempt to build a sudoku solver using TDD with Peter Norvig's attempt [1].

[1]: http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s...

Peter Norvig's approach seems like the textbook way to TDD:

1. Find a large body of known good input/output pairs 2. create tests around that known good data 3. create an implementation that produces the desired output from the given input

There's no reason Peter couldn't or shouldn't have started with the tests.

What you're describing is closer to what Ron Jeffries attempted than to what Peter Norvig did. It's Jeffries who started with input and output and then faffed about for five blog posts while attempting to figure out a way to get input to match output. What Norvig did was start by thinking about the representation of the problem and the constraints that the problem imposed. He then found a way to elegantly represent those constraints in code. And, only then did he write tests.

Moreover, the process you describe is quite similar to the parody of "how to draw an owl" [1]. It completely glosses over the fact that "create an implementation that produces the desired output from the given input" is where 99.5% of the complexity lies.

[1]: https://i.kym-cdn.com/photos/images/newsfeed/000/572/078/d6d...

That a practice is not universally valuable is different from whether a practice is universally worthless.

TDD has a sweet spot and by itself is not enough. "Reinventing computer science, one test at a time" is a joke I've heard at Pivotal about Pivotal. And it's sometimes true. There is a lot of valuable work in the TDD sweetspot. There is also a lot of work to be done on a whiteboard.

Neither is a perfect substitute for the other. You profit from having both.

I'm not the one arguing that the practice is universally valuable. I'm responding to the grandparent's assertion that if you don't understand the problem well enough to write tests, you don't understand the problem well enough to write code. I'm suggesting that that's not always the case: oftentimes when encountering a new problem domain, it's easier to hammer out a crappy solution and watch it fail than it is to deliberately think through test case after test case. Moreover, writing test cases, in my experience, is often a form of fake productivity. It feels good. You're writing code. Hundreds and hundreds of lines of code. But you're not really making any headway on solving the problem.

Sure, I think this is reasonable. Spiking and prototypes are legitimate. I've done both and I expect I'll do both again.

I don't necessarily agree about the fake productivity. I don't think of SLOC as the metric of success. In fact, I rarely think about SLOC at all. What matters to me is proving to my own satisfaction that the software works, now and in the future.

I think we agree more than we disagree. I'm totally in favor of having a comprehensive test suite that ensures that further changes to the application won't break already built features. I'm just of the opinion that unless you already know pretty much what the solution is going to look like it's not a good idea to build the test suite for the solution before you've built the solution itself. Building the test suite first locks you in to a particular way of organizing your code, and changing that organization is painful, because every time one of the externally facing APIs to one of your modules changes, you also end up having to change a bunch of tests.

TDD works well for CRUD applications because there really isn't much new that's going on there. You're reading some data, validating it, manipulating it according to a fixed set of rules that you get from business analysis and saving it to a database. It's fairly straightforward to put tests around each of those stages.

However, if you're making a program to solve a problem which is novel to you, it makes more sense to write the tests after you have a solution, since it's likely that your first few attempts to solve the problem will have to be heavily refactored before you have something that works in the way that you want.

> I think we agree more than we disagree.

I think so.

> However, if you're making a program to solve a problem which is novel to you, it makes more sense to write the tests after you have a solution, since it's likely that your first few attempts to solve the problem will have to be heavily refactored before you have something that works in the way that you want.

I slightly disagree. I am happy to put down my purity hammer and spike away happily for a while until I feel I understand the problem well enough. Sometimes long enough to call it a prototype. Then I go back and TDD my way back.

The mix, as you note, varies by project. I work in R&D. Some work is more Research, some is more Development. But ultimately some of my work is meant to improve someone's life. One of the ways I satisfy myself that I did a workable job is the tests, and for me at least, writing the tests first achieves that end more effectively.

Incremental test-last development works about as well in the small studies as test-first, IIRC. My suspicion is that a non-zero part of the benefit of TDD is that it disciplines you to write a lot of tests. It's a second-order effect that delivers the first-order value.

There are classes of problems for which TDD is quite difficult. My favourite example is fizz buzz. Asking someone to TDD fizz buzz in a job interview is cruel and unusual punishment because it is super hard to do. The reason it's hard for fizz buzz is because it is an infinite sequence. You end up with this choice of either testing some random examples, which may or may not help you with the actual problem (i.e., we can test fizz buzz of 1, 3, 5, and 15, but have we actually said anything about the other ones), or we end up writing tests that are exactly the same as the production code.

A sudoku solver falls into this same category because constraints on the rules are global and the number of combinations are near infinite. So you either write some mostly useless smoke tests or you figure out all the constraints and compare the production code to the test code using the same implementation -- which is useless.

This doesn't mean it is impossible to test drive these kinds of problems. It's not. But you have to take a very different approach than you would with a more normal problem. Jeffries's mistake was in not realising that his approach would not work. It is entirely possible that he did not realise that there were some problems for which this approach would not work until he tried it. While these kinds of problems show up frequently in some domains, then don't show up at all in others. He may have never run into it before.

I highly recommend taking a look at this talk by Kevlin Henney (the TDD discussion starts at 29:15 but the whole talk is worth watching. If you are really pressed for time, his approach for testing fizz buzz starts at 37:40).

https://www.youtube.com/watch?v=_M4o0ExLQCs

I think the thing that's important to realise is that TDD is an approach that has benefits. The benefit of "I can turn my mind off and suddenly the answer will appear" is not one of the benefits. I don't think TDD makes coding easier per se. In fact, I find that junior developers have a lot harder time doing TDD than not doing TDD. It's an added constraint, so it shouldn't be surprising that it's harder. The question is whether or not the constraint leads to other benefits.

One of the things I will quibble with you a bit is on whether or not TDD ties you into architectural decision. I do not agree that it should, but I do agree that a very large number of people who believe they are doing good TDD are doing exactly what you say. I have even seen people go off TDD for exactly this reason. However, I will argue that if you see this problem, it's the first clue that you are not doing TDD well. Should you avoid TDD if you aren't doing it well? That's a good question.

Because tests have to match requirements and those are usually ambiguous or close to non-existent.

Usually it's because we let TDD lapse, and suddenly the rebuilding the scaffolding to properly do TDD for a feature is more work than the feature itself.

I know when I'm coding for me I don't because to me, tests don't solve the problem I'm out to solve at the time. I want this thing, over there. A test does not get it there.

Now, once I'm done with getting that over there, then I look at it and evaluate my level of testing I'm going to commit to it.

Some code, I can keep straightforward and narrative enough where even coming back a year later, I can pick it up and quickly move with it. Part of my secret to doing that is forcing myself to sit down with a saintly non-programmer, and talk them through it.

If I can get them to follow, I've got it nailed down. If I can't, I haven't whittled it down to it's simplest incarnation.

For all the other code... Let the test suites be written!

Because I test everything thoroughly at the REPL as I am developing, and record my REPL tests.

Because I have been doing this for 35+ years and generally have acceptably low error rates without it.

Because I would rather invest test resources in runners that actually exercise the application at the UI layers and the API layers directly in a fully configured and deployed environment rather than trust mocked up unit tests. (Which CI can also do.)

Because tests can have as many bugs as the underlying code.

But, for teams of junior developers, I am all for it.

(I also will write some test harnesses/unit tests for aggressive or major refactorings.)

Using a directory watcher to rerun ruby unit tests on file changes was something that helped me do more TDD. Having a terminal open and seeing it turn from red to green as tests start to pass on every save is a very satisfying feeling. Saving and manually re-running your tests after changing what should make the next test pass is not.

Edit: It was guard for those curious: https://github.com/guard/guard-rspec

TDD wasn't a thing when I started coding, there's no software that I use or that has been influential to me that's developed using TDD. For my personal projects, I'll never use TDD, it's a crutch, I don't even jump into tests, I add assertions and then only add a test if I find a bug that wasn't obvious.

TDD is great when you have a solid set of requirements. But when you are doing "Agile" and the requirements change every day, your tests can be invalidated almost instantly. If I was doing a reliability focused project TDD would be my first step. But that isn't the kind of project I work on.

My fun test for TDD advocates is picking random GUI framework X and ask the presenter how to TDD a native GUI.

After all I am not supposed to show anything on the screen, send a shader to the GPU, or change any widget property without an existing test.

AFAIU TDD, it's beneficial to have an automated test than can check what I'd check manually if I didn't have automated tests (since automated checking is quicker, and is consistent). One benefit of writing a test before fixing code is it avoids the problem of writing a test which passes but doesn't actually check the broken thing. ("You have to write the test before or else you're doing it wrong" seems overzealous or cargo-culting to me).

Of course, as with anything, "it depends" and so if you're confident that adding more tests would hinder your development more than benefit it, sure.

Some programming tasks are difficult to get the system setup for test input, or for checking test output. But it's not like you're not going to check whether the program works, and so it seems beneficial to make an automated test if you can.

I fully support automatic testing as far as possible.

Now TDD cargo culting of writing tests before working code, or even actual design of data structures just seems nonsense to me.

It only works for CLI demos of simple tools, or data processing pipelines.

Anything else seems convoluted, without a sound architecture design, and just impossible in some scenarios, e.g. GUI code, UI/UX.

Tests should be a mix of unit, module and integration tests, written after the architecture design, overall UI/UX design process, performance analysis if the chosen data structures are the best ones for the case at hand.

Given that there are plenty of test harnesses that allow for testing of GUIs, what about GUIs do you think proves impossible for the TDD process?

Large majority only work for browser based GUIs,

Then overall they are only able to validate the UI position or widget properties, which are like 25% of the overall UI/UX of a GUI and HIG compliance behaviors.

There are test frameworks for every tier of any application on any platform. People are getting it done.

Testing pixel presentation is a problem. Today, Automated macro tests can’t describe in a program how the UI should look before the UI is built. The UI is simply too dynamic and subject to the whims of fashion. But writing tests first for the UI and smoke testing that the UI elements are there and basically function is doable.

Most devs struggle keeping their micro testable code seprate from presentation code. TDD is a great way to incur this requirement. Otherwise it needs to be tested in a slow macro UI test. If you’re not familiar with the three tiers of the Test Automation pyramid, give Agile Thoughts episodes 1 through 4 a listen. It takes you through each stage of the pyramid and describes how macro (cutaneous and subcutaneous) and micro complement each other. https://agilenoir.biz/series/agile-thoughts/

> One benefit of writing a test before fixing code is it avoids the problem of writing a test which passes but doesn't actually check the broken thing.

In fairness, this isn't an ironclad guarantee. I have written tests that misled me. That was educational.

These days I do some quick mutations of the test and the code I wrote before moving on. My anecdata is that it catches errors in my tests maybe 5% of the time.

Nice idea to use defect injection to test the automated tests.

> One benefit of writing a test before fixing code is it avoids the problem of writing a test which passes but doesn't actually check the broken thing

Luckily code is version controlled so I can apply the test written afterwards to the old code with relative ease most of the time.

Tdd is writing tests and getting a side effect of production code.

Tdd feels like you are always after the next hit. Turn that red into green by hardcoding. All that matters is green.

This is a very good article on the same topic by DHH. A great read if you haven't read it before:

> Test-first fundamentalism is like abstinence-only sex ed: An unrealistic, ineffective morality campaign for self-loathing and shaming.

https://dhh.dk//2014/tdd-is-dead-long-live-testing.html

To me this is confusing the level of abstraction tests sit with when the tests are written. You can write system tests first and you can write unit tests last, they are orthogonal.

I practice double-loop TDD, which involves writing a system level test to drive out some behaviour, then writing other integration or unit tests down through the layers until the system test passes.

This is something I need to try. And concisely stated. Thanks.

I've heard this called "outside-in" testing.

I thinks this is more something like ATDD (acceptance test driven development) consisting in capturing specifications in acceptance tests and use them in a second loop to drive traditional TDD. Outside-in TDD or mockist TDD following Martin Fowler’s vocabulary is just a TDD technique extensively using test doubles in order to define how actors collaborate or interact with each other to achieve the specifications.

Agreed: * Step 1) (macro) ATDD one or more acceptance criteria * Step 2) (micro) TDD your code to get a macro test to pass.

I played a bit with mock driven development (I think this is the same as mockist). I so for haven’t found that valuable other than as a kata for learning a new mocking framework.

@walligaotr Have you found mockist useful?

Nice! That’s is the best workflow for me too. Your basically working from big picture downward or said another way: outside in.

To do so otherwise creates more risk. I feel the bottom up approach leaves me exposed to discover bad news later in the game and I have to go back and refactor or toss a bunch of code.

This actually says nothing, when you get down to it. The key word is “fundamentalism”, and probably any $foo fundamentalism will satisfy the premise. Take away “fundamentalism” and you’ll see how the “abstinence only” comparison is weak, and the rest does not follow. TDD (w/o the fundamentalism) can be practiced in degrees, and on-and-off, and is not nearly as dogmatic as an abstinence-only sex campaign.

Agreed. Well said and good points.

Lol. Absitinence only sex Ed has nothing to do with TDD. Nor is it fundamentalism. It’s a waste of time to ponder as no real conclusions can be made from slapping the fundamentalism label on TDD. Better to just have an open mind and give it a go. And pay attention to the real benefits than to poison your mind against trying something that could help you deliver more features.

What about languages with strong typing? Is it as useful as with, let’s say, Ruby?

It's 100% as useful, but much more difficult to mock and inject.

> much more difficult to mock and inject

Isn't it just the opposite?

The whole reason of Mokito ( https://site.mockito.org/ ) is to make Java mocks easier by adding dynamic typing.

What you write in your tests, how you test, how you design the software and so on are altered by the language you work in. This is as it should be.

If I have a type system that prevents certain classes of defects, I gleefully accept that bounty.

If I have a type system that simplifies certain classes of tests, I gleefully accept that bounty.

I will happily rant and complain about all of them, but I will also try to program to the strengths of what I'm using. Wishing one language was a different language is a waste of everyone's time.

The static typing makes mocking harder (Java, C#). Working in Javascript makes mocking so easy you don’t need a mocking framework. TDD is an awesome practice in all languages, strongly types or not.

This is a nice series, but the background music is too loud.

There was a back-and-forth between DHH and some of the original TDD advocates a while ago, which I thought was pretty interesting.

Test-Induced Design Damage: https://dhh.dk//2014/test-induced-design-damage.html

And a resulting conversation between Kent Beck, Martin Fowler, and David Heinemeier Hansson: https://martinfowler.com/articles/is-tdd-dead/

I think a lot can be learned from the exchange.

Can someone recommend a podcast on TDD that is to the point?

I couldn't get into these because they seem more entertainment focused than technically focused.

This is tough to do in a listening medium. Episode 12 covers a simple example that expresses the workflow and give you the general idea. https://agilenoir.biz/podcast/012-an-example-of-doing-tdd/

Getting a copy of “TDD by Example” or buying a video course will get you further.

Hey. I do.

Here's a question I find interesting, though: Do you TDD or do you write tests? Lots of people write tests for their code. I don't know many people who actively embrace the idea of TDD -- where the testing activity is driving the other aspects of their coding. Even among the people who do, I know of very few people who agree on what the word "driven" means. I suppose I should be more inclusive and say that "driven" might just mean, "I write tests". I've had numerous people tell me that it is improper when doing TDD to modify production code to suit your tests (which, for me is a really weird definition of the word "driven" ;-) ).

If you do TDD, how do your tests drive your development and do you find that it is a concept that you are able to communicate to others easily?

Disclaimer: I have not listened to the podcast, but the combination of the title and this comment was too thought provoking to let slip by :-) Now I'm quite excited to find some time to listen to the podcast!

What I've found most people do wrong is start with unit tests, which test implementation not deliverable, then realize after writing a bunch of code they did it wrong.

I tend to write the acceptance test first.

e.g. #1 If I'm writing a web app I write a selenium test "as a user, when I click the button, I see X".

e.g. #2 If I'm writing a distributed database, I write a test "state should converge".

These acceptance tests (aka end-to-end, UAT, UI test) ensure the desired end state is tested for, then I work outside-in to create implementation, writing unit tests as needed along the way.

When I find people fail at TDD, it's usually due to the lack of knowledge, ability or infrastructure to approach the problem this way.

Small stories in the correct order are also critical. I agree with the "risk first" approach an article on HN described yesterday.

Source: I taught a course on TDD to some big enterprise clients.

Regarding your first paragraph. It sounds like your mixing up the TDD workflow with ATDD. ATDD uses a requirements level view (macro test) and those concepts are usually well known before starting development. So you can write those tests upfront before development.

TDD is about writing tests as you build the design. You’re not supposed to write a bunch of micro tests before writing any code. Rather you want to write a single micro test, then write a little code, when the test goes from red to green, you then enhance your existing test or add another test, then add a little more product code. So the effect is growing your tests as you grow your product code. Episode 12 illustrates this workflow for a simple example: https://agilenoir.biz/podcast/012-an-example-of-doing-tdd/

The TDD series starts with episode 9 and is still continuing as of episode 15. https://agilenoir.biz/series/agile-thoughts/

The problem I find is that it takes quite a long time after having a sketch solution to realize that it’s a dead end.

You start with “as a user when I press the X button...” but you realize there is no way a button will do. The premise is flawed. It has to be a multiselect list. And this can’t be a user thing, it has to be a config thing” and so on. That is: you can’t (no one could) even specify what is to be implemented without implementing one or more sketch solutions. You can specify the user problem but not the solution. “as a user I need to do X”. But a problem specification alone doesn’t allow easily creating a test on any level.

If I were to do “TDD” I’d perhaps accept tests as being not the first step but the step before the final solution. Sketch solution(s). Throw away sketch. Tests. Final solution.

> Sketch solution(s). Throw away sketch. Tests. Final solution.

I think it's worth pointing out that this has been my interpretation of the general approach that XP originally took many years ago. If you know the general design that your story should take, you do a test first approach using your knowledge of what you are doing as a guide. However, if you don't have a good idea of the best approach to take, you "spike" a solution, throw away the code and then use test first to develop the "final" solution (hard to say "final solution" in XP, so I hope people understand what I'm saying here -- the solution that will be refactored over time as opposed to the solution that will be rewritten ;-) ). As the sibling thread pointed out, though, "test first" in the TDD sense is an iterative process and usually you write a test, satisfy it, write another test, etc, etc. You wouldn't want to write all your tests up front because it would constrain your design too much -- and then it wouldn't be "test driven", even if it is good ;-).

I find it especially interesting that you use the word "sketch", which is exactly the same word I use. For me the activity is based around trying to get my head around the basic composition in the design. But like you say, if you are unsure about the UX workflow, just banging out some code and looking at it can make you realise, "That work flow is just not going to cut it because it doesn't use/generate the information I need".

Because it's tedious and not as fun as just going for it.

:-)

I actually enjoy writing well crafted code that does test and production functionality. Still, I admire your spirit. Just don’t join my team. ;-)

Good idea. I'd take your code coverage down to single digits in one afternoon bam!

TDD requires you to manually predict every possible failure mode and edge case before starting work. It is obviously complete nonsense. Pure snake oil designed to sell training and consulting, not to produce working software.

While I don't do TDD, I know this isn't correct. The idea isn't to prove every single possible way the code can go wrong. It's supposed to be an iterative process, this should do that small thing, write tests to prove it, then refactor updating your tests.

Saying it's just designed to sell training and consulting is ridiculous.

Nicely said @voodootrucker. And I agree with @Tashoecraft. Lots of people get value from TDD. Unfortunately, some don’t and assume it’s the TDD process that’s broke. Its not the tool’s fault. It’s also understandably difficult (though not rational) that groups of people have trouble adopting new practices like TDD. Agile Thoughts podcast exposes some of the social blockages starting with episode 14: https://agilenoir.biz/series/agile-thoughts/

It sounds like you have experienced a project where all the tests are defined up front? This generally should not be the case.

The process should look more like: 1. write a test 2. make it pass 3. refactor 4. repeat

Generally, one would start with "the happy path" tests, not spending too much time testing for all the failure cases until the correct functionality of the overall feature works as intended.

Reply on news.ycombinator.com