Testing with mocks vs testing behaviour

In testing implementation we’ve been talking about coupling our tests to code structure.

One of the test’s technique that increases coupling are mocks.

“but wait, you need somehow to mock your dependencies right?” - you might ask.

Mocking is just one of the strategies possible to setup your dependencies in tests. There are others that can give same or even better results of “checking your code together with it’s dependencies”.

First lets see an example how mocks influence our testing when the implementation changes, but the actual behaviour of the system doesn’t.

Let’s continue on previous example - reviewing a movie in an online movie database with 3 modules involved: Users, Movies and Reviews.

Client communicates with Reviews only, which saves the review if both user and movie actually exists.

One obvious way to write a test that checks if a review is saved in user’s reviewed movies is to use a mock for Movies and Users modules:

    @Test
    fun `when user leaves a review, it's returned in that user's reviewed movies(using 'getMovie')`() {
        // given
        val usersFacade = mock<UsersFacade>()
        // mock "getUser" call
        whenever(usersFacade.getUser(userId)).thenReturn(User(userId, "some Name", "myemail@here.com"))
        val moviesFacade = mock<MoviesFacade>()
        // mock "getMovie" call
        whenever(moviesFacade.getMovie(movieId)).thenReturn(Movie(movieId, MovieDetails("title", "description")))
   
        val reviewsFacade = ReviewFacadeImpl(moviesFacade, usersFacade)

        // when
        reviewsFacade.addReview(userId, movieId, AddReviewCommand(5, "Great movie!"))

        // then
        val reviewedMovies = reviewsFacade.listReviews(userId)
        assert(reviewedMovies != null)
        assert(reviewedMovies!!.reviews.any { it.movieId == movieId && it.userId == userId && it.review.rating == 5 })
    }

implementation change required

Now imagine an implementation change is required - in Reviews module instead of using getMovie method to check if a movie exists, we want to use a newly introduced method checkMovieExists in MoviesFacade (performance benefits).

interface MoviesFacade {
    fun getMovie(movieId: String): Movie?
    fun checkMovieExists(movieId: String): Boolean // new method that we want to use
}

The change is easy, so we implement it, we run the test and… It goes 🔴 - test fails because we should mock checkMovieExists instead of getMovie now!

We need to change test’s implementation and mock new method instead of the old one:

@Test
fun `when user leaves a review, it's returned in that user's reviewed movies`() {
    ...
    val moviesFacade = mock<MoviesFacade>()
    whenever(moviesFacade.checkMovieExists(movieId)).thenReturn(true)
    ...

After the fix test is green again 🟢.

But there is one big problem with this approach:

Using mocks results in coupling our tests code with actual implementation, because code under tests must know the exact methods/objects we’re using in the implementation

We’ve tackled this problem partially by agreeing that we’re using modules Facade pattern in tests - both for tests setup and verification. Facade API rarely changes, so also mocks of other modules won’t change often - instead of having tests based on classes/functions that are only a part of our domain - where contract between code structure can change very often.
Still, let’s see if we can push it even further!

Alternatives

In our scenario - an implementation change, without behaviour change made our tests go 🔴 - and that breaks the promise of tests, that they shouldn’t break without behaviour changes 😮‍💨

Mocking is often a default strategy when dependencies set up is hard, but is it always? There are multiple strategies to set up dependencies in tests. Let’s see how we can implement other strategies and how can we benefit from them, instead of using mocks every time.

Modules fakes in tests

One of the underused patterns is the Fake pattern.

Fake is a pattern of using some simplified version of a code just for tests. It provides the same behaviour of the “real” system, but in a simplified way:

can use In-memory repository pattern instead of a real database implementation
it can have a small stub-like setup required, instead of making external calls (performed in production environment)

In our solution, each module provides their fake implementation for other modules to use in their tests.

An example Users module fake can look like this:

object UsersFacadeFactory {
    fun testFacade(): UsersFacade {
        return UsersFacadeImpl(InMemoryRepository())
    }
}

Where InMemoryRepository is a simplified implementation of UsersRepository pattern:

class InMemoryRepository : Repository {
    private val users: MutableMap<String, User> = mutableMapOf()
    override fun saveUser(user: User) {
        this.users[user.userId] = user
    }
    override fun findUser(userId: String): User? {
        return this.users[userId]
    }
}

Now, when we want to test Reviews module, we can use UsersFacadeFactory.testFacadeWithUsers() to get a simplified version of Users module, that we can use in our tests. It provides a ready-to-use facade with data already generated, so that we don’t have to setup them ourselves.

@Test
fun `when user leaves a review, it's returned in that user's reviewed movies`() {
  // given
  val (usersFacade, users) = UsersFacadeFactory.testFacadeWithUsers(usersCount = 1)
  val (moviesFacade, movies) = MoviesFacadeFactory.testFacadeWithMovies(moviesCount = 1)
  val userId = users.first().userId
  val movieId = movies.first().movieId

  val reviewsFacade = ReviewFacadeImpl(moviesFacade, usersFacade)

  // when
  reviewsFacade.addReview(
    userId,
    movieId,
    AddReviewCommand(5, "Great movie!")
  )

  // then
  val reviewedMovies = reviewsFacade.listReviews(userId)
  assert(reviewedMovies != null)
  assert(reviewedMovies!!.reviews.any { it.movieId == movieId && it.userId == userId && it.review.rating == 5 })
}

If we use fakes instead of mocks in our tests, when we implement the change in Reviews module, our test will keep working. The only requirement is that Movies test implementation will provide a working implementation for checkMovieExists method (which it will and we’ll later cover how to test those modules depending if those are local, in-memory calls or remote clients like gRPC/HTTP)

Summary

What summarize benefits of using fakes instead of mocks in our tests - especially at the level of testing modules.

🟢 Let’s start with some pluses:

we don’t test other modules at all! We use provided API to create a working Users and Movies modules.
out tests are not coupled to the implementation details of other modules - we can change the implementation of Movies module and our tests will still work as they are.
we end up with a reusable TestFacades to use in all other modules - and it always behave’s the same way

🟡 There are of course some minuses:

somebody has to provide the Fake implementations
- true, yet the team providing the Fake implementation actually uses it partially in their own tests (we’ll see how to test your modules in details later on)

FAQ

Do we always have to work with in-memory implementations of other modules?
- No, as the Movies module team we can provide a configuration to connect to any DB that’s required (for example Postgres) - if we’re at the level of integration testing.
isn’t the cost of having to provide fakes for other modules too high compared to the benefits?
- It’s not that high, as the team providing the Fake implementation can actually reuse parts of it in their own tests. You maximize re-usability if you’re running unit tests focused on your domain
What if Movies module is a remote client, like gRPC or HTTP?
- Well, for most of the cases - it doesn’t really matter. We’re testing the behaviour of our module, not the implementation of the Movies module, and Movies module shall execute the same logic whether it’s a local or remote call.
  - where it differs is some additional use cases required - like test case when Movies module is not available or returns an error.
- when Movies module is always used as Http client, responsible team can provide a Fake implementation that uses a local server that provides the same API as the real Movies module.
  - you can provide it using tools like Wiremock or MockServer for example (as you probably will not like to run real HTTP calls in your tests - you’d like to run your tests at the airplane, right?)

Wrap-up

We’ve seen how using fakes instead of mocks in our tests can benefit us in the long run. We’ve seen how it can help us to decouple our tests from the implementation details of other modules, and how it can help us to reuse the same test setup in all other modules. Of course remember to be pragmatic in terms of choosing the test double you’re using, but keep in mind great benefits of using fakes in your tests compared to using mocks

Additional reading

https://martinfowler.com/articles/mocksArentStubs.html#CouplingTestsToImplementations