Hacker News Reader

Comment by arnsholt | Hacker News Reader

Semi-relatedly, this is why I'm quite liberal about throwing errors while developing programs. During development it makes sure I'm reminded of my slipups when I inevitably forget some invariant that needed to be upheld, and then towards the end of the process any throws remaining are easily found and vetted. And in production, the closer to the source the error occurs the easier it is to fix; if you're especially unlucky, the error doesn't occur until after state has been persisted and read back, in which case you not only need to fix the bug but also have to figure out how to handle and/or remediate the bogus state written to your database.

View on HN

I used to code this way, then I started to write tests, as I was writing my code. I could ensure the behavior was expected with the test. If the code wasn't easily testable, I would refactor it right way and that would generally leave me with easier to read and follow code as well. The end benefit being that if I wanted to change the behavior later, now I knew if I was breaking anything. With everything tested, very little made it to production that wasn't working.

It's not about being easily testable, it's about being in an invalid state for an unexpected reason, you can't test that trivially.

You can try to avoid it by architecture or abstractions, but it only works to some extent.

Testing isn't trivial, it can be downright difficult. But if you test for those invalid states, the reasons are expected.

No, this is a pipe dream. The only valid state for a sorted list is sorted. You can not possibly "test for invalid state" here because there's no way you can handle a sorted list that is not sorted in any meaningful way.

Mind you this example is the one of the most trivial data structures in existence. Real software has insanely more complicated valid/invalid state, the size of the state space growing exponentially with the number of components.

If I have a method wrapped around adding items to a list, then I'd call that method with random data. Then, I'd call the method that gets the items from the list and I'd check that the order is consistent. Now, you're testing that the underlying implementation is a sorted list.

I'm sure you can object in many specific cases, but the point is there's no general way to do that. Even your description could be unable to find the famous Java bug with integer overflow in binary search had it only happen on extreme cases. I'm not even talking about race conditions.

And these are still single components, so the exponential growth of state combinations point is left unaddressed. In fact, it directly supersedes your point in this comment, as the exponentiality eats your "random data" idea before you even wrote it.

We're not testing Java, we're testing your code which is a method to add items to a list. If it passes the tests, great. If there is a bug in the JVM, maybe it picks it up or doesn't. That doesn't negate the need for the test.

The way I look at testing is less about the current point in time. It is about changes over time. If someone else comes along and changes the underlying implementation to a list where sort order is not guaranteed (maybe thinking their version is faster or something), then the tests will fail and that's exactly what you want. You're testing the expected behavior over time and right now, the expected behavior is a sorted list.

Regardless, I don't understand what you're trying to argue or prove here. Yes, testing is hard and not easily won, I already said that.

> But if you test for those invalid states, the reasons are expected

I am arguing against your claim that this particular suggestion is feasible for anything but toy projects.

Suggesting that tests are only for toy projects is some weird kind of ego troll thing you've got going. Look inward.

Exactly what I'm talking about:

https://erikras.com/blog/final-form-to-typescript

"Of course, it helped that long ago I had put in all of the legwork to have near 100% test coverage, so I could be certain that the migration did not introduce any bugs."

Reply on news.ycombinator.com