Recently I finished developing a high performance ISO-8583 financial transaction authorisation server for a client using The Server Framework and whilst I was running the final black-box tests against the server I realised that these particular tests were dependant on the system date of the machine running the server. The server uses the current time and date to make some decisions on whether it can authorise a particular transaction or not. The date is used to look up some information from a database table. The table contains details of whether a particular competition type is available for entry or not; it has start and stop date/times and the transaction is only valid if it’s possible to enter the required competition at the time the transaction is processed.
The black-box server tests load the database with test data, start the server, generate lots of valid and invalid message sequences and their expected responses and pump the messages into the server from multiple concurrent connections. The test works well and thrashes the server nicely.
My problem is that the database data needs to relate to the date and time that the test is run to make sure the test harness and the server are in agreement about what can happen when the test runs. I don’t want to have to change the date on the test machine each time the test runs but, of course, since I load the data before running the test there’s a relatively simple way to fix this problem, and another way, I chose the other way…
I discarded the idea of changing the date on the test machine pretty early on. It’s fine if you only ever run the tests on a test machine but it becomes a pain if you want to run the tests on a normal developer machine. Messing with the system date has all kinds of unintended side effects and so that’s not an option.
The simple way to avoid the problem is to generate the data that we will load into the database based on the date the test runs. Right now I have a fixed script that sets the database to the state that I want it and this is run as part of the setup to the black-box tests. It’s also run when I run the database connectivity unit tests. It simply empties the database and loads a particular data set into the database before the tests are started.
For the database connectivity unit tests there are no date related problems as we pass in the ‘current date’ into the functions that talk to the database; following the SRP means that the database access code is responsible for “accessing the database” and not for “accessing the database and working out what the current time and date are”. As we rise up through the layers of code in the server the unit tests mock out the database and, once we get high enough, also mock out the code that obtains the time and date. All of our unit tests are freed from date dependency by judicious use of dependency injection techniques.
The black-box server tests are more problematic. Somewhere in the server there’s a line of code that does actually obtain the current date and time and whilst in the unit tests for this code we pass in a mock date/time provider, when the server is actually running it uses the real date/time provider. It takes the date/time that it obtains and eventually passes it into the code that calls into the database.
Generating the data would mean that we either need to have two sets of test data, one that’s a simple script and one that’s date dependant or we need to make our database connectivity unit tests date dependant. I’d prefer not to do this and I could live with the duplication of having date dependant data for the black-box test but the actual problem is something that I’ve come up against several times before and in most of the previous situations we couldn’t simply change the data we were using.
The work I’ve been doing on other testing tools has generated a nice library of useful code and I figured that it would be a useful test of the flexibility of that code to see if I could use it to write a tool that would help me in this situation. It took me about three days to put together a tool that solved my problem. The result is a program that can be given a program to run (the target) and can then manipulate the results that the target gets from
GetSystemTime() etc. The result is a tool that allows me to run a program on my development PC that has its own view of the date and time on the machine. In the case of my server test I simply run the server as the target process and set the date to be the appropriate date for the database data. The tool can change the date or time for a particular target and also freeze the time and/or date for the target. Whilst working on the tool I decided that I could also use it to help with testing
GetTickCount() usage. By adding functionality to the tool so that it could affect the result of
GetTickCount() I can use it to set the tick count to, say, 10 seconds before rollover and see how the target program responds when the tick count wraps to zero.
Next on my list of things to do for this tool is to add a simple GUI so that the user can control the date and time interactively rather than simply setting values at the start of the program; this would have been especially useful when I was working on “The Refactoring Project” as we had lots of date related issues and it was always a pain to change the system date to test them.