Hacker News Reader

Comment by cousin_it | Hacker News Reader

Yeah, I mostly prefer end-to-end tests as well. Though to be fair, they are often slower than unit tests, because you need to start up the whole system. And they are worse at pinpointing problems, though that doesn't seem to be a big deal in practice.

View on HN

I like to test the whole system via end-to-end tests, as they're the best bang for the buck. And then I'll create unit tests for more algorithmic code, like a parser, sort algorithm, shortest path calculator, financial calculations. Those also tend to require the least amount of test context setup, making them less painful to write.

I have almost always worked for customers who enjoyed changing their minds arbitrarily and often in ways they swore they would never do.

In the face of grossly changing requirements, I've never had much luck keeping E2E tests up and functioning properly. And people have a bad habit of investing more time and energy than a particular test is worth in trying to keep it working or porting it to the new requirements.

Unit tests are cheap. If the requirements change invalidates twenty of them, you just delete them and write new ones. Easy.

I in part agree with the pinpointing, though in my experience this really boils down to an issue of scope and how much of the data you want to test in each of your tests. E.g. an API returning user data, do you have one test for the whole set of data or one test per field.

For our use case we have some pretty "fancy" deep-equals logic for nested structures and allow to specific fields as "to be ignored" in our test expectations.

When it comes to speed, the most important part is to cut out everything you don't need, for us this means that our testing framework completely throws away the internals of Node.js HTTP layer. We never create a single TCP socket in our tests, which gives an incredible speed up. As a bonus this also allows to test timeouts and low level HTTP errors in < 1ms, since we can just return the timeout directly from the low level APIs and Node.js will invoke the timeout event on the http client handle.

> Running "noir:mock" (noir) task > ...................................................... > 1048 passing (10s)

And yes, that really reads 10 seconds and yes I always get a bit bored when I have to run the tests for one of our Django backends, feels like an eternity... :)

Count yourself lucky that your test takes 10 seconds. A partial build on my project takes ~3 minutes to compile + link. A full build is about 3 hours for everything.

I recently cut some link times from 3 minutes to 14 seconds by switching linkers (from bfd to gold doing Android dev.)

...sadly that's still only for a single project x config x arch combination. I still need to play around with incremental linking options that appear off by default...

I had a build that took 25 hours to run once. Never again. The only reason we got any work done is that we ran a 1, 3 and 19 hour part of it in parallel. But I really just wanted to throw out the long part (bad E2E tests) and start over.

> though that doesn't seem to be a big deal in practice

Gotta be careful there chief. In practice is usually refers to "your experience" (but might not be mine).

My experience is on the flip side: end-to-end tests are super slow due various reasons...

Reply on news.ycombinator.com