Practical Testing: 3 - Test 2, Enter The Mocks

I’m writing some blog entries that show how to write tests for non trivial pieces of code. This is part 3.

Last time we wrote the first test. It doesn’t test much, but it proves we can test. Now we’ll write a real test for real functionality and along the way we’ll start to deal with some of the issues that come up when you’re trying to test multi-threaded code.

To test the timer functionality we need to write a test that sets a timer and then waits for it to go off. This first test should be able to verify that the timer was triggered and that the callback was executed. The function we will be testing is SetTimer() this has the following signature:

void SetTimer(
   const Handle &hnd,
   DWORD millisecondTimeout,
   DWORD userData = 0);

The Handle type is what the CCallbackTimer uses to identify an instance of a timer. There are two constructors for a Handle but we’ll ignore the second one for now as that’s part of the unfortunate complexity that we mentioned earlier. The constructor we’re interested in for the purposes of this test is this one:

explicit Handle(Callback &callback);

So, to call SetTimer() we need to create a Handle and to do that we need to create a Callback… The Callback is just an interface that the timer queue uses to alert us to the fact that our timer has expired. The interface looks like this:

      class Callback
      {
         public :
   
            virtual void OnTimer(
               Handle &hnd,
               DWORD userData) = 0;
   
            virtual ~Callback() {}
      };

To be able to call SetTimer() we will need something that implements the Callback interface; that something will be a mock object with the sole purpose of helping us to write tests.

As I’ve said before, I like to keep my mock objects in a library project that’s in a subdirectory of the project that the mocks provide functionality for. So, if the interface lives in Win32Tools the mock for that interface lives in Win32ToolsMock… This is a useful way to structure the code because you will quite likely need to use the Win32ToolsMocks when writing tests for libraries that depend on interfaces from the Win32Tools library. When I started with unit testing I kept my mocks in the test harness project but this made them hard to reuse in other test harnesses. The new way is better.

Our first mock only has to implement a single function, OnTimer(), but what does it do once the function is called? Most of my mocks inherit from a base class from the TestTools library; CTestLog. The test log allows derived classes to log messages to it as they are used. At various points during a test we can query the mock object’s logs to make sure that the correct functions have been called in the correct order and with the correct arguments.

Our implementation of the mock’s OnTimer() could be as simple as this:

void CLoggingCallbackTimerHandleCallback::OnTimer(
   CCallbackTimer::Handle &hnd,
   DWORD userData)
{
   LogMessage(_T("OnTimer: ") + ToString(userData));
}

But to be able to test reliably we need more control. Suppose our test looks something like this:

void CCallbackTimerTest::TestTimer()
{
   const _tstring functionName = _T("CCallbackTimerTest::TestTimer");
   
   Output(functionName + _T(" - start"));
   
   CCallbackTimer timer;
   
   CLoggingCallbackTimerHandleCallback callback;
   
   CCallbackTimer::Handle handle(callback);
   
   timer.SetTimer(handle, 100, 1);
   
// what to do here?
   
   callback.CheckResult(_T("|OnTimer: 1|"));
   
   Output(functionName + _T(" - stop"));
}

We set a timer using our mock object for the callback, we then do something and then check the test log contains the expected result. The question is, what should we do in between. We need to wait for the timer to expire; we could stick a Sleep() in there, but that adds an element of uncertainty to our test. For the delay to be effective in all situations it needs to be long enough to always be long enough, no matter what machine we’re running the test on or what the CPU loading is at the time. A test that only works sometimes is worse than no test at all. If developers start wasting time debugging bugs in the unit tests then they’ll soon lose interest in testing.

Unfortunately, since we’re dealing with time, we can’t solve this problem completely; at least not this time. We can make sure that the test executes as fast as possible though, and still keep the delay long enough to make the test reliable. The trick is to add some code to the mock so that we can wait for the timer to go off. A manual reset event will do. If we change OnTimer() to this:

void CLoggingCallbackTimerHandleCallback::OnTimer(
   CCallbackTimer::Handle &hnd,
   DWORD userData)
{
   LogMessage(_T("OnTimer: ") + ToString(userData));
   
   m_event.Set();
}

Then we can add a function to the mock so that we can wait for the event to be set:

bool CLoggingCallbackTimerHandleCallback::WaitForTimer(
   DWORD timeoutMillis)
{
   return m_event.Wait(timeoutMillis);
}

Our event class returns true if the event is signaled within the time limit and false if it isn’t. Since the wait will end as soon as the event is signaled and the event is signaled when the timer callback is called we can delay the test until the timer expires and provide a timeout that is reliable but that doesn’t slow down the test run.

The test code becomes this:

   timer.SetTimer(handle, 100, 1);
   
   THROW_ON_FAILURE(functionName, true == callback.WaitForTimer(200));
   
   callback.CheckResult(_T("|OnTimer: 1|"));

So now we have a test that’s reasonably reliable. We could increase the delay to something larger and improve the reliability at the expense of slowing the reporting of failure situations. Unfortunately we can’t make the test any better than this until we remove our reliance on time; and that will have to wait until the next posting…

Code is here. Same rules as before.