Comment by z3t4 | Hacker News Reader

Comment by z3t4 | Hacker News Reader

I like to use something often called "regression tests", that whenever I find a bug, before fixing it, I write a test that would detect that bug. Confirms that the test catches the bug, then write the fix, and re-run the test to confirm that it's fixed. All tests are then run automatically before deployment. Also if you already know those 2% of the code that can go wrong, also write tests for those! The only disadvantage with tests are that writing the tests can be more difficult that writing the actual code. Another strategy I use is defensive programming; "throw an error if the invariant wasn’t properly maintained" used together with code coverage and testing. So when the tests run the code, (or during manual testing) those defenses would trigger. I leave the defense on even in production, and log all errors (if on server) or generates a bug report, that users can submit with the click of a button, if in a user interface. The problem with defensive programming is that even if you have a call stack and state, it can still be hard to figure out the steps that lead up to the bad state. So you need some levels of logging.

I do not consider testing (TDD) or defensive programming a band-aid for bad language design, they catch many sorts of issues, not only type errors. Even if your code is very beautiful, you still want to verify that it works. Although demoing the app, or putting the app in-front of thousands of users, is probably the most effective way of finding issues.

I used to make this argument, but I think I was wrong. I sort of hate working in javascript projects I’ve written with a million tests, because if I want to refactor something, it takes 3x as much work because of all the tests I need to rewrite.

Lately I’ve been really enjoying typescript and rust. The ability to write code and if it compiles run it and have it be correct is like a cool breeze on a summer day. In typescript I need to write fewer tests (because the compiler catches bugs that I would need a unit test for in JS). But also, when I refactor something the compiler gives me a list of all the places in my code which I need to update. In JS I have learned to worry about lingering bugs that I just forgot to write unit tests for or something. With a typed language, that fear is (mostly) gone. It’s a lovely quality of life improvement, and it noticeably improves my velocity.

If you're re-writing tests you're not refactoring you're just rewriting.

If tests need to be rewritten to support a refactor those aren't tests, they're at best just testing the implementation rather than requirements and at worst just a second implementation.

I do not like traditional unit testing as the tests are glued to the code, then I would rather write assertions in-line together with the function! Instead I take a black-box approach, for example a HTTP server is tested by making actual HTTP requests to it. That way you do not have to rewrite the tests when you change the code - just add additional tests.

That's another checkmark for keeping functions pure so you can easily do substitution. Problems will arise if your language doesn't have managed IO because you can never be sure.

That is another useful way of testing. The problem with it is eventually you write some low level code and realize that sometime in the future someone is likely to use your new code in a way that will expose a bug - but right now something higher level prevents that bug, and you have no idea how to prevent it with assertions inline and the outside black box tests can't cover this case yet.

For large complex systems all levels of testing are needed to get the best quality.

I use defensive programming, as in throwing errors when something is used the wrong way. Assuming that the developer always run the code at least once before shipping. But I try to avoid thinking into the future. I'm already too good at finding edge cases, that if I would also think about possible issues that might arise in the future, I would get nothing done. I instead wait for it, and then fix it only when it has become an issue. Especially in optimization, I often fall into though traps like "what about when this table has millions of rows, this query would take a long time", I have to remind myself of the problem and what the priority is right now. While there is code debt, there is also opportunity cost.

Unit tests and type checking solve very different problems. In practice there is a lot of overlap, but there are gaps in both that the other covers, thus you need both if you want the best quality (if you want the best quality you will also look at formal proofs which is a super set of type checking and can catch even more problems that the above two won't - but even that will not catch everything)

Unit tests prove that everything you thought of works correctly. Formal proofs show some ares where you didn't think of something still work - but they can fail because the assumptions are wrong.

I agree that you need both. But in my experience you need far fewer unit tests in a typed language in practice to be reasonably confident of correctness. Like, maybe 1/3 of the number of tests. The reason you need so many tests in JS is that its just so easy to accidentally refer to a variable that doesn’t exist anymore or something like that. And the only real way to be safe from that sort of mistake is to go overboard with unit tests and aim for 100% coverage, which destroys velocity.

As you say, unit tests and type systems are good for different things. Unit tests can be used as a poor mans type system, but that seems like an amateurish deal now we have typescript & friends. Well, at least for code you want to maintain long term.

Defensive programming leads to hell. The next one looking at the code won't be able to tell the actual business logic from defensive checks, or what may happen from what should never happen. I've been badly burned by this, especially while trying to reconstruct flows from badly documented code.

Reply on news.ycombinator.com