Pledging for unit tests
weblog.madebymonsieur.com
weblog.madebymonsieur.com
TDD on the other hand is both an unprofessional waste of time (write code that you know isn't going to work just to make a test that you know isn't comprehensive enough pass? Really?), and an unprofessional way to structure your design - instead of a bunch of well behaved classes that abstracts and neatly encapsulate their behavior, you have to insert hooks so that you can inject your mocks.
At the end of the day, you end up testing software and find some, but far from all, of your bugs.
If you are concerned about software quality, great - write better code, do root cause analysis, but most of all: use a language with a good static type system.
Cheers!
Unit testable code becomes ugly and unreadable very fast. The main goal becomes to make it testable instead of making it readable. It pained me every time I had to do this in the past, and I did it because some company wanted it no matter what. I don't do this in my personal projects and they are almost bug free anyway (no unit testing does not mean no testing at all).
In TDD you spend time making your code ugly and you probably introduce bugs in doing this, and then you spend other time to find and fix those bugs. And when you need to go through that code again for fixing or refactoring it's harder to understand because of all those unit testing hooks.
For very big systems or for applications with a certain structure it probably makes sense to have unit tests. In functional languages they are a lot better because you usually don't have to change the code to adapt to it. But for a lot of applications they make no sense at all.
So please, stop selling them like a panacea for everything. Even better, please stop selling anything like it is a magic medicine for every illness.
Unit tests are useful for all these ugly things that come from developers not paying attention to something. I've found them extremely useful as a sanity check when I get a particular specification (might it be a design document, an API or a specific RFC).
Then again, the testing I usually found the most useful had to be integration testing. I would just sit with the testers, see what they expect the program to be and basically translate the test cases they would do (or that I would do in a REPL) into a module and run it. Lots of time saved for everyone in the process.
IMO, the basic principle of TDD is solid: write the tests, then write the code. Like most ideologies, this takes an acorn of wisdom and grows it into an oak tree of folly (or some other really pompous metaphor -- that sentence rather got away from me).
The problem comes when people think the tests are something valuable in and of themselves. They aren't. They are only valuable insofar as they show a developer (a) if something is broken and (b) what it is.
I generally agree that Mocks are more trouble than they are worth -- they are an entire mini-codebase that doesn't contribute business value and must be maintained. But usually you can get most of the benefit without spending the effort: e.g. instead of a Mock::DB, have a separate instance of your prod DB running a snapshot; instead of a Mock::HTTPConection, just do a real HTTP request.
In short, I relate to your philosophy and share it to a certain extent, but I encourage you not to let the TDD zealots drive you away from what can be a useful tool.
This is where we disagree: the value of the code is secondary to the tests (this shouldn't be that big a point of disagreement: would you rather have no tests and the code of a functional program, or great tests and no program?), but if you design the tests first, you are going to design the program such that it is easy to write tests for.
Write the program first, then write tests for it in the places where it is reasonably worth your time to do so.
IMO, the "killer app" for unit tests is regression testing. If I make a change and the test suite still passes, I can be reasonably sure that I didn't break anything. All the other benefits are "nice to haves" in my mind.
The common complaints he lists at the start and acknowledge actually all boil down to the fact that unit tests do not always make work more efficient. It's a tool in the toolbox. Sometimes they are great for your project. Sometimes they are a waste of time. It's still not magic pixie dust.
In my current project we have a test suite for our public webservice APIs and various clients that interface against those. Those tests are invaluable. For other parts of the project unittests don't make sense, so we don't have them, since making them would be a colossal waste of time.
So yes, you still need to defend unit tests, just like you need to defend every other choice you make during development. No specific tech gets carte blanche.
Generally I am for brevity: the program should be short. So I prefer that the 'checksum' also should be short. Unit testing is a really elegant and good tool if you can test a quite big chunk of functionality with a short test. If the test is for example longer than the tested functionality then its value is more controversial. Ok, for extremely important code, like the control of an airplane, put there as much redundancy in the checksum as you think is needed. But otherwise, it can be questionable.
The issue is quite similar with static typing. I am a static typing fan, but dynamic language guys always complain that you have to type too much in static languages. And they are right to some extent, although better static languages have good type inference, so it is not that big of an issue. In fact I don't really understand dynamic language guys. They always state that they like terse code, but with modern statically typed languages like Scala you really don't have to type much more than in dynamic languages. The difference is negligible compared to that those same guys are usually unit testing fans and their unit tests sometimes really make the code base significantly bigger.
TL;DR;: I value terseness, and I prefer to use both static typing and unit tests only if it does not increase the code size significantly.
Wanting to find bugs seems to be an important factor in writing quality tests. It's easy to write soft tests when the same person writes the code and does the test. Maybe that's why pair programming and TDD seem to go well together, as the other person is more motivated to try and break your code?
1) How many projects have you worked on that were 100% test driven and written outside-in?
2) Do you have the knowledge to test any code, regardless of web APIs, filesystem etc.?
3) When you deploy an app without 100% coverage how can you be confident that there are no regressions?
4) When you add new functionality are you comfortable changing the existing code or do you try to leave it alone and build on top of it?
It's perfectly possible to develop and maintain large, complex systems without 100% code coverage. Clearly some level of testing coverage between 0% and 100% is optimal. The case for it being 100% is, to my mind, extremely weak.
edit: In response to child post - That was my interpretation of this "It will make our work so much more efficient, motivating and a bit more professional."
It seems that I definitely need to improve my English. :)
Somethings need/should be unit tested, but most "glue" code benefits far more from investing time in robust error handling rather than testing.
There in lies the problem. It isn't unit test vs full up build an installer, and run the server. Those are just the two ends of the spectrum.
The only way to call it unit testing is if you make unit mean something completely different to what most people mean by unit. Yet it is automated testing that covers real functionality.
- Create integration test/stories - For each step in integration test - Add unit tests to make it pass, starting at the view layer and working backwards.
The level of unit testing required for each step depends on the complexity of the code. So what does unit testing get you over just performing integration tests? 2 things:
1) Documents and enforces expected behaviour for edge cases that you haven't written integration tests for - e.g. unusual error conditions
2) Isolates points of inflexion in your system - your stories/use cases will often rely on several components that have multiple logic paths. When you combine the branching logic from several components the combinatorial range of behaviour can be vast. By isolating units at the points of inflexion you are able to test a far smaller range of inputs/outputs so that you can achieve effective test coverage. You're quite correct that the units you tests are not necessarily aligned 1 to 1 with your classes. However, given that a well designed class will have a single responsibility this is often the case.
I often see stories that are not accepted multiple times simply because the product owner has changed the intent of the original story. Tests really help to define what it is that the product owner wants before a line of code is even written.
In the bigger picture, how much "more time" does it really take?
Dumb story, but my dad taught me how to do word working: making European Cabinets. So many times he would say "Measure twice, cut once." This, in some convoluted way, could apply to software engineering.
Measuring software is expensive, so you are better of not doing it.
Untrue; tell that to the person who's paying the QA teams who have to validate your code. Or the patch teams that work 48 hours straight to get a hotfix out because something broke after deployment.
Software is anything but free.
It's about as hard as convincing people of the advantages of static typing. Not to pick on Ruby particularly, but those guys go crazy for writing their own tests for things a compiler could tell them "for free".
Compare that to writing code. Fixing something you did wrong is so trivial that I'll often find myself writing 3 variants of a bit of code and testing rather than pulling out a sheet of paper and doing a bit of math to make sure I code it correctly.
"Should this be cos(theta) or -cos(theta)?"... code, compile, check, whoops, delete, compile, check, done.
That's waaay better than "134 inches, I think", cut, fit, whoops, phone Jerry: Hey, could you pick me up a new piece of #4 crown moulding on the way in?
I guess the point you're responding to meant that "eyeballing" whether something actually works is the equivalent of "estimating" a length, a couple of easy tests is the equivalent of "measuring once" and "measuring twice" is taking the extra effort to build a small suite of tests with corner cases, a small sign of having learned from bitter experience.
Of course with automated tests you are literally measuring twice (indeed, measuring many times) even as the tools and materials change out from under you.
I hereby declare that anyone who doesn't follow Jackson Structured Programming is a two-bit script kiddy!
As a pure developer working on an established codebase, the benefits of automated testing are quite clear, especially if you have paying customers and the cost of failure is high.
But as a startup, every extra line of test code you write slows you down from launching, and the most important thing at this point is launching and figuring out whether you actually have a market for your product.
After all, what's the point of having 100% test coverage for a product that no one wants to use?
It's also entirely likely that you're going to end up throwing large sections of your codebase out as you pivot towards something else.
Here's the way I see it:
It's rather pointless to have 100% test coverage of a product that no one is using, but it's also dangerous to have 0% coverage of a product with live, paying customers.
Intelligent developers will have the find the right balance between the two (as it shifts over time).
I'm the author of the blog post. I'm very glad that it has gained already so much opinions and feedback.
As user ollysb, I'm quite surprised to notice that there seems to be more against the post than for it. And that was one of the reasons why I actually wrote it: You are smart guys, please proof me wrong about unit tests. I can only find good things in unit testing, so it seems that I have not fully thought it through.
ps. Did the post have "inflammatory tone"? It was definitely not my intention. Sorry if it did. I have done computer programming for living only just about 10 years and I am definitely not a professional in it yet. Which parts made it sound cocky? I would love to hear feedback for that too improve my writing.
Then you are simply not very experienced using it. You're in the love-phase where everything about some new technology seems wonderful, and since it's solved so many previous problems for you, you are blind to the warts and problems that come with it.
We understand your enthusiasm for it, unit testing and TDD are good tools, but they're just that. Tools. That you can choose to use if they fit the project.
Your blog post (and the previous one) both said that unit tests and TDD are always awesome for all projects. This is not true, and this is what people here are reacting to.
Yes, most likely the truth is that I'm not experienced enough of using it and the type of systems I build are limiting my view.
I also understand unit testing is a tool with it's limitation.
However, for me, unit testing is a good tool that helps me to proof that the code I did works. And I hope I would see more of unit tests on the field.
But as you said, probably the way to find the dark side of unit testing is just doing it and finding the problem. So I'll keep using the tool until I do.
I think I also really need to learn to write better. :) Where did I do such a claim that the TDD and unit tests are always awesome for all projects? I think I even did the opposite.
On the systems side (in this case, a PHP framework), I've found that once test coverage passes 70% you can make optimizations and refactor the code without having to actually run it - the tests will tell whether the changes you did still maintain API stability.
Pledging for unit tests (madebymonsieur.com) NaN points by koski 1 hour ago | 11 comments