What would I do??

There’s an entry over on the Dr. Dobbs blog about testing and how you make sure that your tests are testing the right thing; effectively, who tests the test. There’s a question at the end “What do you do?” and I think my rather pithy, I’ve had some wine, answer is, “I think harder”.

The poster laments the fact that if you’re doing TDD then the test fails first and then you write the code and then it works and therefore you know the test is testing the correct thing but if you have existing code then, well, it doesn’t work that way. It only doesn’t work that way if you’re being lazy.

The example given is that an already developed component that has tests is now made multi-threaded. The poster decided that simply running all the existing tests in parallel would test the component for thread-safety. And was then surprised to find that it didn’t due to how the tests all tested their own, isolated, instance of the component…

Hmm. Personally I feel that if you’re trying to write a test to prove that something is thread safe then you need to write a test that deliberately puts the thing under test in a situation where the lack of thread safety shows itself. You don’t just assume that running existsing tests together and having them work will somehow prove something… Writing tests for multi-threaded code is hard. You need to think about it. Abdicating thinking and then somehow pushing this failure back onto the tests themselves is, er, rather crap.

As I’ve said before, the tests act as scaffolding for the code and the code acts as tests for the tests. If either is wrong or is changed so that previously held beliefs are no longer true then the tests fail. You don’t write tests for tests you write tests for code and if either disagrees with what should happen then the test fails. It’s like aircraft having multiple redundant systems, they should agree, or there’s a fault.

The original poster’s problem is that he didn’t actually bother to write a test for the situation that he wanted to test. I think he should be asking “who tests the tester”…