You don't test the http.Client as part of your business logic, even (especially) if your business logic makes HTTP requests. Instead, you trust that package http is already well-tested, mock out http.Client as an abstract HTTP Request Do-er, and you downscope your tests to your business logic exclusively.
I'm not so worried about http.Client behaving incorrectly, but I am worried about the server I'm targeting behaving in a way that's different than my expectations. I'm also worried about getting cookie, post-data, url, etc. formats wrong. Even if they match what I put in my mock, they won't necessarily match what's out there in production. I'm worried that I'll handle download resumption wrong, for example.
If I have the capability to bring up the target server locally, that's what I will do. If I can't, but it's straightforward to create a work-alike, I'll bring that up in process. Only in the most dire of circumstances will I use mocks to test my code.
Think about it this way. What harm is done by using the real object, under the circumstances I described? Sure, if the dependency code breaks, my tests will too. Is that bad? Is there any point in my tests passing when my code won't actually work in production? OK, so maybe the tests will run slower? This matters in the limit, but often times you can use the production dependency without materially increasing test time. OK but what about the CI system using more resources? That will only be a problem of significant economic import if you're running at a scale like Google or Facebook.
My opinion is there are a few main valid objections to my approach. One is an argument about the size of the test code. In some cases, bringing up the production dependency will require more code than a mock. Sometimes a lot more code. My response is that we should make it easier to bring up the production dependency! But if we can't, I can see the case for a mock or a fake. The other counterargument is for when we really are interested in testing the sequence of interactions the code under test has with the production system. In that event, I guess I'm ok with using a mock, but I think most of the time we're more interested in testing state. We can also use things like testvalue injection to test behavior sequences, which is normally a lighter-weight approach, and in some cases can allow you to reproduce paths that you'd otherwise have trouble hitting, even with a mock.
This is the responsibility of system or integration tests, not unit tests. (Unit tests are what's under discussion in the article.)
> What harm is done by using the real object, under the circumstances I described?
Time, energy, and effort, all of which inevitably lead to longer (slower) dev/test cycles.
My contention is that using real objects normally does not take materially more of any of these, and thus that most code is served best by being tested only by integration and regression tests. Where my contention is untrue, apply your best judgment, of course. :)
full-system testing, with an honest attempt to do fault injection, by definition exposes all the issues you might have in production. if you have a decent test suite you can honestly claim you can expect it to take client code correctly.
i don't know how thats true if you've only tested A against some stubbed out version of B that always returns success.
integrated testing is harder and requires more framework. its also kinda necessary, and if you only have the resources to do one or the other, the choice seems pretty clear.
i also heartily agree with your comment that since we're supposed to have frictionless and scalable deployment. there is no reason not to test this as part of integration testing and also exploit this function to make integration testing easier.
I work on search and indexing systems at Google, and have done for more than a decade. I'm by no means the best, nor an authority. But I have spent some time living in this problem space, and this is my conclusion from my years of experience.
FWIW a single run of the tests for the system I work on now takes . . . I don't know. I would guess hours or maybe even a day of computer time? But, computers are cheap, and defects are expensive. So you split the test suite into bits small enough that the whole thing runs in ten minutes, and run them all in parallel. And, yes, there have been cases where I have had to debug the slow startup of a dependency so that I can make it fast enough to run in my test. But, that's what they pay me for, I enjoy that type of work, and it has ancillary benefits to the dependency when it needs to start up in production.
OK, well I guess this would be another caveat for my advice. If bringing up a production simulacra would take more resources than you have, then you might have to lean more on various types of doubles. This may be more often in environments that have insane prioritization of dev time vs. machine resources.
If I may ask, what was the bottleneck in your test suite? Did it saturate the cpu, ram, disk?
I'm sorry, I'm making an argument about how people should write tests, not what terms they'll use to describe it. :) I promise you that there is no disingenuity going on in my use of certain terminology to describe how I think it's best to write tests.
There are a few general ways to do this. The best way is to use the real underlying dependency. This should be the default.
If the dependency is too slow to use in the test (e.g. it has a long startup time, or is hosted remotely), the next best thing is to use a fake -- a local implementation of the dep that has the same behavior, but operates in memory on ephemeral data. The best libraries will provide these, and if they don't, you might be able to implement one and contribute it back. Even if you have to maintain it yourself, it often takes less code than describing mock behavior. The downside of fakes vs. the real implementation is that their behavior too can diverge. But this is not a net downside compared to mocks.
If you can't use a fake, the next best thing after that is a stub. See below.
Only if none of these options are viable should you use mocks.
(The position I'm describing is the one referred to as "classicist" in this post: https://martinfowler.com/articles/mocksArentStubs.html. I'm not by any means its originator.)