Hacker News Reader

Comment by juliangregorian | Hacker News Reader

Doesn't sound like you're unit testing well then. I'd take good unit tests over integration tests any day. Integration tests are inherently flaky.

View on HN

Your sentiment is strange to me - I've only ever seen unit tests as absolutely worthless. The reason integration tests are flaky is exactly the same reason they are valuable.

Someone who checks in broken code will also take flaky tests as an excuse not to look at the build results, or to convince themselves that the failure is caused by flakiness and not their changes.

If I had a dollar for every time I asked someone why they weren't fixing their shit, and they answered "Oh, I thought the build was failing randomly again." I'd have a lot of dollars.

The fact that your tests are flaky at all means that the integration tests have revealed a bug that you need to fix. The tests have done their job. The fact that people aren't looking at the results speaks to a larger cultural issue.

Not if it's a timing error in the test harness. That's usually what people mean when they say flaky tests. Selenium, especially three or four years ago, would simply fail occasionally no matter how good your tests are.

Yes, there are bugs that look like bad luck, but are really flaws in your logic. I have fixed a lot of bugs that look Random, but eventually you're left with, for instance, selenium just refusing to click a button.

Unit testing well takes 3-4x as long as integration testing well. Sometimes it pays off; other times it doesn't.

Reply on news.ycombinator.com