Test Doubles: When To (Not) Use Them

I'm currently introducing somewhat 'advanced' testing techniques such as Dependency Injection and using Mocks/Stubs to my team members. They're getting the hang of it pretty fast, but the typical "do i need to mock this object for this test?" question came up pretty frequently at first, and sometimes it still does. Which is only normal when you're starting out with these techniques. So i figured it would be useful to list some guidelines about it in a post.

First of all, i'll use the term test double instead of mock or stub or whatever for the remainder of this post to (hopefully) avoid any confusion. Gerard Meszaros (author of the excellent book xUnit Test Patterns) provides the following definition of a test double:

A Test Double is any object or component that we install in place of the real component for the express purpose of running a test.

One thing that's not clear from just that sentence (it is very clear in the book, but not when you merely look at this sentence in isolation) is the fact that it usually only makes sense to use a double for what Gerard calls Dependent-On Components (more commonly referred to as dependencies) of the code you are testing.

So, when does it make sense to use a test double, and when are you better off just using the real component? A lot of people think differently about this, but i have found the following guidelines to be the most efficient with regards to keeping the tests focused, fast, easy to set up, and most importantly: meaningful.

If a component uses an external resource (a database, a file, a remote service, ...) either directly or indirectly, then i always replace the component with a double when i'm testing code that is dependent on it. An external resource usually means that the component is either slow (keep in mind that you want to run a couple hundred tests per second), hard to set up (required data or files or whatever), or a possible source of unpredictability (tests should never fail and then work again without changing code). Using doubles allows you to avoid the slowness, makes it easier to set up any required state, and they are always predictable.

Components that do not use external resources yet still require a lot of work to set up (for complex in-memory calculations for example) are also good candidates to be replaced with test doubles. Even if the real component is lightning fast, if you need to write a lot of set up code just to get the real component to behave the way you want it to for a specific test, then you're better off using a double for it. Tests should be focused, so you don't want a lot of code in them (either in the test method itself or in the setup/teardown methods) if that code doesn't really pertain to the functionality that you are testing in that fixture. Performing a bunch of set up for dependencies is simply a waste of time while writing it, while maintaining it, and it's also a distraction while reading it. So, always avoid that, even if the components don't use external resources.

That's pretty much all for when you should use doubles. Now, when should you not use doubles? My answer to that is very simple: when there is no clear-cut benefit to it. If you're using classes that do not have external resources and hardly require any set-up, then by all means, use the real components in your tests. Using test doubles is a great technique, but it is only a means to an end. Using doubles merely for the sake of using doubles will not lead to better tests. It's very important to keep that in mind.

I often hear developers say something like "but if you're not using the real components in your tests, how do you know for sure that they work?". Well, the answer to that is also pretty simple. Each component should have its own dedicated tests. Yes, even components that use the database, or the file system or even remote services or whatever else you can think of. These components will be used at runtime, therefore they need to be tested. Test them as thoroughly as you can, but no more than that. Test them in isolation, perhaps even in a different test suite that you don't need to run every time. Perhaps you don't even need to run these tests automatically? But that would be a nice subject for a different post

Anyways, the guidelines (or rules of thumb) are just that... they're only guidelines, not strict rules. As we all know, when it comes to coding, there are always exceptions (no pun intended). If you follow these guidelines and some things still feel like it requires too much work or are causing too many issues, then there's probably a better solution for that specific problem still waiting to be found. Don't be afraid to spend a bit of extra time searching for it. It might save you a hell of a lot of time later on.