Hacker News Reader

Comment by mannykannot | Hacker News Reader

Do you mean the series of posts linked to from the one you give, beginning with [1]? They are an excellent demonstration of two of the pitfalls of naive TDD:

1) The things you can test first (before you have figured out the central problem) are not necessarily the things that matter most. The author should have addressed the algorithm for solving the puzzle first; everything else is secondary.

2) This is an example of those cases where having tests for what you are going to write does not provide much guidance into how to solve the problem.

[1] https://ronjeffries.com/xprog/articles/oksudoku/

View on HN

When making this comparison, it's worth noting that Jeffries is showing his work process. Norvig is showing a cleaned up version of his final result after throwing away some earlier attempts. (Source: I spoke with Peter about Sudoku solvers shortly after he'd published.)

That is a fair point, but if you just consider the TDD attempt in isolation, it demonstrates some ineffective ways in approaching the problem.

I think that's understood. But what's telling is that no single step of Jeffries' series of articles seems to be approaching any kind of sensible solution. It reads like an amateur -- e.g. me -- trying to write a game and getting stuck in the low hanging but inconsequential fruit.

Because Jeffries' series is about TDD, and not really Sudoku, the process is important. And the process failed him spectacularly.

> They are an excellent demonstration of two of the pitfalls of naive TDD

I think it's particularly interesting because Ron Jeffries is not a novice but one of the creators of Extreme Programming, an expert TDD practitioner and evangelist. And to my knowledge, he never acknowledged clearly that he failed in this exercise because he approached it the wrong way, or that he was naive about TDD.

I think he acknowledged it pretty clearly; he claimed it as one of his most public failures, didn't he? But also, where's the TDD naivete? The problem isn't TDD, the problem is knowing how to apply constraint propagation. You could TDD constraint propagation and it'd work fine.

> You could TDD constraint propagation and it'd work fine.

That's exactly why I call it naive TDD, and not a failing of TDD in general. Jeffries deserves credit for making it public.

He deserves some credit, but he would have deserved full credits had he acknowledged the limitations of TDD. Which are the ones you mentioned in your first post.

> I think he acknowledged it pretty clearly; he claimed it as one of his most public failures, didn't he?

He did? All I see is a series of articles where he randomly flails towards no clear goal, with no clear understanding of Sudoku and gaining no insight either. At the end it just peters out. The last article claims "Sudoku will not ship", and that's it for a post-mortem.

Unless he wrote another article elsewhere explaining how TDD failed him, or how he employed it wrong, or owning up to any mistakes, I think he acknowledged nothing.

Watching Norving vs Jeffries writing a Sudoku solver is watching someone who understands the field vs a novice who doesn't know what he doesn't know. Except this novice is a self-proclaimed expert. It's embarrassing, really.

> But also, where's the TDD naivete? The problem isn't TDD, the problem is knowing how to apply constraint propagation. You could TDD constraint propagation and it'd work fine.

Jeffries "sold" TDD as a design and problem exploration technique (aka "TDD is not about testing"), which it really is not suitable for, or at least not for algorithms (I've no doubt you can TDD a CRUD application, but how uninteresting is that?). It's a lot less sexy if he had said "yeah, all the TDD in the world won't save you researching the problem, maybe actually reading the literature on the subject or learning about specific techniques (like constraint propagation) will help". TDD as a design/discovery method for algorithms doesn't work -- you won't learn constraint propagation by doing TDD. Did Jeffries acknowledge anything of the sort?

You know why TDD is often demonstrated in toy examples with the Fibonacci sequence or similar algorithms? It's because everyone already understands them. It's easy to "arrive" at a solution you already know beforehand.

Be wary of anyone who proposes some procedure X, and when it fails claims "yeah, well, this is not a failure of X because <reasons>".

Reply on news.ycombinator.com