Even More Asserts in a Single Unit Test Method

In my last post I stated that sometimes it is OK to have multiple asserts in a single unit test method and I devised a helper class MultiAssert for that.

Please do not get me wrong – I am a great believer in keeping unit tests readable and maintainable, and restricting a unit test to have exactly one assert is one way to achieve that.

But despite that I am now going to argue that sometimes I find it to be OK to have multiple asserts in a unit test method, even without packing all the asserts up using MultiAssert.Aggregate.

Remember that “The arguments against multiple asserts are multiple, a main one being that if one assert fails the rest will not be executed” (from my last post).

However, what if I in this case explicitly want the rest of the unit test not to execute – is it then OK to have one assert ensure that another is not executed? I think so. Read on and I will explain.

The other day, I looked into a unit test that failed randomly. I knew that a person whose unit testing skills I do not usually question wrote it. Still, it turned out that it was surprisingly tricky for me to figure out why it failed.

The following is a simplification of the original unit test,

[TestMethod]
public void DoThis_MustCallDoThatOnFooWithExpectedParameters_WhenCalled()
{
    _target.Initialize(_foo, "first", "second");
    _target.DoThis();
    _foo.AssertWasCalled(
        f => f.DoThat(
            Arg<string>.Is.Equal("first"), Arg<string>.Is.Equal("second")));
}

In order to figure out why this unit test failed, I started making assumptions.

My first assumption was that DoThat was called with at least one of the two parameters having an unexpected value, so I added a line to the unit test to let Rhino tell me what the actual parameter values for the DoThat call was,

var actualArgs = _foo.GetArgumentsForCallsMadeOn(f => f.DoThat("", ""));
MultiAssert.Aggregate(
    () => Assert.AreEqual("first", actualArgs[0][0]),
    () => Assert.AreEqual("second", actualArgs[0][1]));

This did not help me, as inspecting actualArgs only caused an index was out of range exception to be thrown.

My second assumption was that DoThat was not called at all. To test this hypothesis I tried the following,

_foo.AssertWasCalled(f => f.DoThat(Arg<string>.Is.Anything, Arg<string>.Is.Anything));

Bingo! The unit test still failed (randomly) with this assert, which showed me that DoThat was not called.

So the single assert of the original unit test was actually multiple asserts behind the scene – one assert to state that DoThat was called and two asserts to state that each of the two parameters were as expected. This is one case of having multiple asserts that I do not like!

With this knowledge, it was quite easy to track down the root cause of the failure. Somebody had checked in a parallel implementation of _target.Initialize so that the initialization of _target randomly made it to completion before _target.DoThis was called. While I realize that it is important to figure out what this new parallel code will do in production code, for now I will keep focus on the correct implementation of this unit test method.

Since we have three asserts in this unit test, some would say that it is obvious to split it into three separate unit tests? Well, I would go for one or, perhaps, two. Read on and I will explain.

If we want to split into three unit tests, that would be

• One to check if DoThat was called at all,
• One to check that firstParam had the expected value, and
• One to check that secondParam had the expected value

It could be argued that this is a lot of repeated setup that should be avoided. I do not agree – common setup can be put in auxiliary methods and execution will be very fast because I have mocked out external dependencies to the code under test.

My problem with the tree unit test approach is rather that the first unit test is a prerequisite to the two other unit tests; if the first fails, the two other will certainly also fail (barring random behaviour). I believe that if you have an assert that, if failed, will make another assert fail with certainty, then these two asserts can, and often must, be packed together into a single unit test method.

So does that mean that the correct implementation of this unit test is to split it into two unit tests?

• One that checks if DoThat was called, and also checks that firstParam had the expected value, and
• One that checks if DoThat was called, and also checks that secondParam had the expected value.

I believe that the most maintainable way to implement this is to have a single unit test.

• One that checks if DoThat was called, and also uses MultiAssert to verify that both firstParam and SecondParam had the expected values.

All in all, this means that we have changed the original – hidden – multiple asserts into two explicit asserts. This is better than the original, as the first assert is a prerequisite of the second, which means that it makes no sense to execute the second if the first fails.

Here is the resulting implementation,

[TestMethod]
public void DoThis_MustCallDoThatOnFooWithExpectedParameters_WhenCalled()
{
    _target.Initialize(_foo, "first", "second");

    _target.DoThis();

    _foo.AssertWasCalled(f => f.DoThat(Arg<string>.Is.Anything, Arg<string>.Is.Anything));
    var actualArgs = _foo.GetArgumentsForCallsMadeOn(f => f.DoThat("", ""));
    MultiAssert.Aggregate(
        () => Assert.AreEqual("first", actualArgs[0][0]),
        () => Assert.AreEqual("second", actualArgs[0][1]));
}