The first part of your comment sounds really odd to me. I definitely write tests to check if my code works, among other things.
Why do you think that is the wrong approach?
Why do you think that is the wrong approach?
Automated testing only shows its real value when you go back to change code that was working before. With the manual approach you'd have to retest everything to have any real confidence. With the automated approach you just run a command.
I'm a big fan of automated testing, but if I didn't expect to have to ever change code, I wouldn't bother with it.
> With the manual approach you'd have to retest everything to have any real confidence.
This kind of implies that software development before "~tdd" was a complete disorganized gong show of quality especially where refactoring, but in fact that was not the case. There are ways of coding that are more conducive to quality than others.
> With the automated approach you just run a command.
Once you've written all that code, yes.
> if I didn't expect to have to ever change code, I wouldn't bother with it.
Usually you don't, in which case the extra effort on testing is wasted, usually.
The typical claim is that code maintenance is at least 10x as long as the initial write.
The question isn't "do we need to test." Testing can just take the form of running the code manually and making sure the output makes sense, but you do need to test.
The question is "do we test automatically or manually." It's the same question regardless of how good your practices are. Note that this is totally distinct from the question of whether or not to use TDD.
> Once you've written all that code, yes.
I've found that writing tests often doesn't take much longer than testing manually, and rarely takes longer than testing manually twice. Sure, if you obsessively try to test every possible case and input, you'll waste time, but well targeted testing doesn't have to be slow.
> Usually you don't, in which case the extra effort on testing is wasted, usually.
For non-trivial projects I have an average number of revisions per line of code much closer to 2 than to 1. Sure, some of the code only gets written once and never touched, but other code gets revised multiple times. If you're good at writing tests, the tests will focus on those often revised lines of code.
And, again, you have to compare the effort against the effort of manual testing, not against the effort of writing code you've never run and shipping it.
end to end tests, integration tests, regression tests, etc, yeah.
Unit tests though...usually no. Often if you're making any kind of significant change, the entire code paths may get refactored away or change too much and the test will get nuked anyway.
And it's that kind of test that usually confuses people, so it's worth understanding.
A unit test's goals are many:
It proves at authoring time that the code works. It saves you the time right away of having to go through the UI or spinning up a server just to validate a function is working. It proves that you thought about specific edge cases (and if you do testing consistently, the lack of test is your evidence of unconsidered edge cases. It's documentation of all of the things you considered when writing the code. It is an example of how to use the code with all it's use case.
And when you nuke a piece of code away, the failing tests are now a guide of all the cases you have to make sure are truly no longer necessary.
If, in the future, you do a refactor of an implementation detail (so the existing tests are still valid), then that's bonus as you get green/red validation. But in practice, that is less common than all of the other reasons for testing. That's why the "I don't expect this to change" thing isn't really a reason for or against writing tests.
That pretty accurately describes my experience with unit tests. I've been part of several projects where we had pretty comprehensive unit tests (I'd say small to medium sized projects) and I never managed to get as much use out of unit tests as I liked. After one or two big refactors most of the tests needed to be, as you said, nuked anyway. While seeing all the pretty green lights is reassuring, they are rarely working when you most need them - during large refactors which blows away big sections of code.
I picture unit tests as a row of black boxes sitting on a table in certain positions. Unit tests are great when you don't move the black boxes but do change the mysterious processes are running inside them. But refactoring is rarely ever that isolated in programming since you tend to move some of the black boxes around, remove some entirely, add some new ones, change the contents. To then expect the unit tests to give you back useful information on whats broken is rarely possible.
I've had more success with e2e testing using things like Selenium, but it's still frustrating as a developer to read articles about how great unit testing is (like this root comment) and never able to actually get a decent working version of it in your projects (because of the reasons mentioned by this parent comment).
Building up small oases of dumber/more verbose code that has unit tests seems like the best way to wrangle legacy code into something that anyone else on the team can understand and not mess up 6 months from now when it's their turn to have to touch it for the first time. Of course no one wants to touch that 200-line, 10-levels-of-indentation monster method, but bringing just a little bit more and more of it under unit test over time will help a lot. Every other benefit of unit testing that you listed besides the time saving aspect (e.g. why launch a big e2e test if you can test the same thing in an xunit-like context? even if it's not strictly a "unit" test) pales in comparison of the benefit of making crap code nicer to work with. A corollary is that if your code is already nice to work with, and you have a system to keep it that way, unit tests won't be very valuable. (Though other tests, which may or may not be in an xunit-like context, may still be quite valuable.)
1. Write tests to prove your code works, which is sometimes referred to as "test-driven development"
2. Write tests to catch any side effects or regressions when altering code
Funny thing is that if you write good tests, then the results are pretty much the same regardless of your motivation.
1. Have an overall idea of what your software will do before writing the first line of code.
2. Challenge and change any touched assumption from #1 during development when you refine that idea.
3. Test that the program satisfies your refined idea after it's written.
4. Create some assurance you'll keep #3 correct while you write any further code later.
However you fulfill those needs, if you got them all, you are good.
When something doesn't work as expected, I now check my design and not the code.
Or you're tired. Or you're just not as focused as you could be. Or you have a deadline. Or you're trying something new. Or you're not fluent in the language yet. Or you're fluent in the language but not the framework. Or you're fluent in the language and framework but not the design pattern (if you use those).
Or a whole host of other things.
I'm not saying spot-checking the design isn't a good idea, but saying that it's the design more often than the code just doesn't match up with my experience.
Based on a true story
Out of curiosity, did you use any debugging on paper for your code?
"Because you should write code that works in the first place!" -- your boss