Hacker News Reader

Comment by paranoidrobot | Hacker News Reader

Perhaps I misunderstand you.

Say I'm writing tests for lets say ShoppingCart.

ShoppingCart depends on an implementation of ICartDiscountCalculator (CartDiscountCalculator)

CartDiscountCalculator depends on a number of implementations of ICartDiscountStrategy (ChristmasCartDiscountStrategy, VIPCustomerDiscountStrategy, BulkPurchaseDiscountStrategy, etc)

Each of those, in turn has their own dependencies, perhaps webservice calls, db calls, accessing caches, etc.

The way I would test ShoppingCart would be to mock out ICartDiscountCalculator, and provide known responses.

Are you suggesting instead that you build up the dependency graph - i.e the actual implementation of the discount calculator, the strategies, etc - and only mock when it goes to external code (eg the DB calls)?

This would seem to make testing very very difficult - since each test needs to know implementation details of everything it depends on.

View on HN

Um, no. The test needs to know the expected system behaviour, given that dependencies are properly wired & behave as expected. You can argue that "it no longer tests just the ShoppingCart in isolation" and that would be true; but whether that's a downside or an upside is up for debate IMO (and more often than not it really is an upside).

[edit] In other words, you don't test that "if discount is 10%, adding item to shopping cart increases cart total by 0.9×<price>"; you do test that "if user X adds item Y to shopping cart, and X has a 10% discount code, then shopping cart price increase by 0.9×<price>". This does seem like a headache, but it has benefits - in that it tests actual product scenarios, and you can now easily add variations like "if discount code only applies for books, and item I added isn't a book, then shopping cart total increases by <price>".

I would argue then that this isn't Unit Testing, it's some other kind of testing.

One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.

If what you "spin up" is internal classes/functionality, not external system, I don't understand the problem? The article doesn't argue "this is how you do unit testing", it argues "this is how you do better testing". It's highly debatable whether unit tests are preferable to larger, "component tests".

The problem is that it makes testing every upstream component more difficult to write for, and more fragile.

To take the shopping cart example I gave, if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations.

For instance, I've written tests for CartDiscountCalculator and ShoppingCart that have 10 different items in the cart from Acme Corp Manufacturing.

Some time later, I implement a new AcmeCorpDiscountStrategy which results in a different discount being returned. Now my tests for CartDiscountCalculator and ShoppingCart need to be modified.

I also need to potentially modify everything else up the dependency graph from CartDiscountCalculator - so my internal API that is used by customer service, an API used by mobiles, and the front end website.

Using Mocks/Fakes to isolate the ShoppingCart, it doesn't matter what the strategies are, we only care about how it interacts with ICartDiscountCalculator.

That interaction might change, maybe ShoppingCart uses the CartDiscountCalculator in a different way yeah, that'll break ShoppingCart Unit Tests - but it's that method only, and you're already in there modifying ShoppingCart anyway.

> if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations.

No, you would need to consider only the business requirements.

> Some time later, I implement a new AcmeCorpDiscountStrategy which results in a different discount being returned.

Why did you implement that? Is it a business requirement change, that "existing customers should get this different discount"? If yes, then your tests break - but for a good reason. If it's not about existing customers, but new sort of customers, then your tests should be unaffected and you just need to write new tests that target these new sorts of customers and validate that the new discount strategy is applied.

It depends on how you define your 'unit'; a unit can be as large or as small as you require.

But, I'd suggest if you can't test a single class in isolation easily (and you might need fakes/mocks) then the class probably needs redesigning.

I say that it depends on the "unit" you're trying to test. Is it large and complex enough in itself that you need to isolate it from other components (dependencies)?

For your use case, if the shopping cart is simple enough, I'll just choose a simple or frequently used discount calculator and make some unit tests to test the shopping cart functionalities. Then for the rest of discount calculator cases there will be separated unit tests.

The problem with testing almost everything in isolation makes refactoring some small-inner components costly, since every change may break some of unit tests.

You're right, it's not unit testing. I call it integration testing, some people call it functional testing.

The question is why you want to test that one class? We do have some modules in our (elixir) system that are 'thoroughly' unit tested. These are modules that are algorithmically complicated so testing them in isolation is useful. Most of the code is tested in logical chunks though.

I think this is what OP is saying, you need _both_ unit and integration tests.

In unit specs you should use mocks/fakes for external behaviour. Then you can simply verify that it's calling the dependency correctly.

Isolation specs are useful because when integration tests fail, they usually give little feedback as to what exactly went wrong in the system.

Also integration specs are expensive and slow to run since they execute everything.

Finally I think mocks/fakes are useful because if you find yourself mocking 20 objects to get a single unit test working, then it's a sign that you need to redesign the code.

It shouldn't need to know implementation details, but it should need to know expected behaviour. The use of ICartDiscountStrategy and BulkPurchaseDiscountStrategy is an implementation detail, the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for.

I would refer to these tests as "integration tests" although the language is poor here. Test a load of module together. This is beneficial in a number of ways: It tests whether modules are integrated together correctly - you can check that module A works and calls your fake of module B correctly, but you don't have a good way to check that your fake of module B matches module B behaviour properly (in practice it's usually when there are multiple modules involved that the problems occur). Unit tests only test stuff in isolation. It also means that you can refactor all that code, and as long as you don't change the top interface the tests will remain valid. If you move some responsibility around and change interfaces this can result in a lot of time consuming unit test changes, and that usually means that people don't actually structurally refactor as much as they should do.

Integration tests do not replace unit tests, they should just be taking a lot of the load. This at least is my interpretation of the well known "Write tests. Not too many. Mostly integration" tweet (https://twitter.com/rauchg/status/807626710350839808).

> the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for.

But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger. Maybe it varies based on time of day or for certain customers.

> I would refer to these tests as "integration tests" although the language is poor here.

Agreed, it's a discussion I'm having at my job now, I havn't found particularly great wording. "Functional" testing still doesn't quite work right.

> Integration tests do not replace unit tests, they should just be taking a lot of the load.

Agreed on the first part. Integration tests are valuable, but I think they're more fragile.

Writing them is, IMO/IME, slower and more difficult.

As for the linked tweet, it's the "mostly integration" part that I disagree with.

> But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger.

Language is poor, but I think there is consensus on what "implementation detail" means, and I don't think that's it. Whether it's 9 or 10 is customer facing behaviour. It's behaviour that might come from your marketing team or product owner or whatever, but it is outward facing. It is something that someone could write in a word document and call part of your specification.

The implementation detail would be whether that logic lives in BulkPurchaseDiscountStrategy or as one case in a big ShoppingCart class. A user pressing buttons on your UI can see the result of the price break, but they can't see the difference in code organisation.

You can say "price breaks should occur at 9 items" without ever saying "BulkPurchaseDiscountStrategy". In one ideal, all tests would test user facing requirements and not know anything about your code. You could refactor your code to your heart's content and know you the result was still what you wanted. There are downsides to that though because those end to end or system tests are fragile to write and expensive to run. It's much easier to cover all possible code paths with more focused testing. Hence in this case you could test the top interface of ShoppingCart, with all it's assembled dependencies. Not the same as pushing button in a Web Browser, but a lot more convenient to write and run.

But the price break is a requirement that I expect to change. Today it is 10. Tomorrow someone looks at the production process the machines give groups of 11, so the bulk discount should be multiples of 11 (if you order 12 you get 11 at the bulk rate and 1 and the regular price). Then next week management says small orders are too hard to serve, and so the minimum bulk order is 100, but bigger discounts if the total order is a multiple off 11. Then the week after...

Point is that your implementation of the discounts is expected to change. I should not have to change my shopping cart tests at all just because the discount requirement changed - it makes the costs of those tests too high.

Of course this is domain specific. I expect discounts to change all the time. Likewise I expect taxes to change often. In both cases they need to be mocked/faked just to keep my test code sane. there are many other domain specific things that won't change. I'm always going to store the shipping mass in KG, and for some customers convert it to lbs. It is reasonable to hardcode the weight in lbs as that won't change. (future me who has to ship to mars will disagree, but I don't know what the requirements are)

If your requirement changes so that the price break is at 11 rather than 10, then I expect the process to go:

* Change the (or shared test code) that has the price break specification from 10 to 11 * Run it, see the failures * Update the source code appropriately

The updated source code is the same in both our cases. In my case the changed test code is testing the ShoppingCart interface and in yours it's a unit test of something lower down. There's not extra cost, it's just in a different place.

Or are you suggesting that your tests don't know about 10 vs 11 at all? There's a reasonable architecture where such data comes from outside your system, and the spec for your system is 'price breaks should reflect this data input' and then you mock that data input and test it, but that's a different thing.

I'm saying the discount code tests have the change from 10 to 11. this entire thread isn't about the discount code tests though it is about the shopping cart. There is a lot more to calculating price than just discount codes, tax rates for example. The cart needs to show regular price, the discount, the tax, shipping costs, and then total. Each of those is tested for multiple edge cases, so if we do it all in the shopping carts there are potentially hundreds of unrelated tests that need to change when the discount factor changes just because they happen to check the total.

What it does is show up something that is true anyway for unit tests: automated tests are software and they should be designed like it.

It might affect multiple tests, but all you should have to change is one or two places in some shared code pulled in by those tests. Too many tests have too much repeated code, and so become expensive to maintain, exactly like normal code. Refactor, think about shared functions etc. exactly as you would do with your production code.

You have found one exception: if the item has multiple different implementations - as your discount strategy does - then you should mock (or better yet fake) the implementation. In this case testing with a real doesn't gain you anything.

Reply on news.ycombinator.com