Hacker News Reader

Comment by jamesgreenleaf | Hacker News Reader

I've felt the same way - as though not being able to shoehorn everything into pure unit tests somehow means that I'm not doing testing right. In most cases though, what we really care about is verifying the full functionality of an app. Maybe integration & e2e tests should be considered as the primary form of testing, with unit tests applying only in the specific cases where they make sense, like you mentioned.

Maybe the ulimate form of testing is something along the lines of Netflix's chaos monkey (now apparently an entire "simian army" suite of tools). Continuously test the real application, live in production, where it's not allowed to fail.

View on HN

I believe pretty strongly in the "testing pyramid" - that you should have a broad base of unit tests, with fewer integration tests and even fewer e2e tests. Source (also from Google): https://testing.googleblog.com/2015/04/just-say-no-to-more-e...

I think good unit test coverage is essential for good maintenance, but you can't cover _everything_ with unit tests. You definitely need integration and e2e tests to ensure bigger picture correctness. I have found that value one gets out of the different kinds of tests as you proceed up the pyramid get higher the closer to release you are, and are most valuable post release. During development they become obsolete too quickly due to changes in how things work. But locking in an already-released version to avoid regressions is great.

I think the real question for me is, suppose you come into maintainership of a legacy application with 100k lines of untested code, where do you start? I think the classical TDD view is "every new change, you add a test for, and so test coverage grows over time."

And that's fine, but I think it's much more valuable before making any functional changes at all, to impose some high level smoke tests that run the entire thing, and do a handful of common, end to end user flows.

Yeah, I'm a big fan of the Outside-In approach (I think it's from Michael Feathers's work).

The idea is that you test something at the very end of whatever you're working on, and ensure that it's identical to the original. You can then make larger changes to actually test the code properly while confirming that the overall system still works.

Super, super useful with ML things where running stuff can take a very looooonnnnnngggg time.

Yes, that gels with what I was saying - use e2e tests for "big picture" assurance that the system is correct. Use unit tests for super fain grained assurance of individual business rules, etc. All of the legacy projects I've ever inherited have lacked unit tests, and are theoretically functioning correctly, so the e2e tests make sense there.

Reply on news.ycombinator.com