In response to @dhanji on unit testing

Dhanji over at Rethrick Construction has written an interesting piece on the value of unit testing.

I agree with his conclusion; “So the next time someone comes to you saying let’s write the tests first, or that we should aim for 80% code coverage, take it with a healthy dose of scepticism.” But then I tend to take everything with a dose of scepticism…

I also agree with the fact that sometimes the tests get in the way of refactorings that you’d like to do and sometimes the tests give you more code that needs to be maintained and that they often appear to slow down your development time. I’m probably more affected than many by these problems because I happen to like rigid tests with hand rolled mocks which log and validate all of their interaction with the objects under test. Yes, you need to maintain the tests as well as the code and, yes, sometimes you’ll find that a simple change to the code results in lots of changes to your tests. This is, often, a chance to refactor your tests - something that I don’t do often enough. It’s also often a sign than you need to refactor your code too, if seemingly unrelated tests are failing then you possibly have some unexpected cohesion between the code and tests relating to the code you’re changing and other tests which are only related via mocks and interfaces… Is that interface too fat, does it do too much?

However, I’m not sure that deleting the tests is a good idea, though of course it depends on the tests themselves. Sure it’s often faster to test working code with tests which test a lot of functionality; integration tests are great and I have a very blurry line between them and my unit tests - my integration tests simply test larger “units” than my unit tests. Deleting unit tests and testing at a higher level is almost certain to make you feel faster when you’re working on a well factored, well tested and stable code base and when the change you’re making works and doesn’t have any strange and unexpected side effects. You can simply make the changes required and fix up fewer tests and run your integration tests. The problem, in my experience, comes from when the change doesn’t go in as easily as expected… The higher the level of your tests the more they’re testing and the less focused they are; test failures may tell you that the functionality doesn’t work any more but they don’t necessarily tell you where the problem is. This may make it take longer for you to find the problem and it may mean that you spend longer debugging… You may end up adding new unit tests to track down the problem, or you may simply spend longer in the debugger.

These things are hard to quantify which is why I often use JIT and initially write fewer tests than I feel I should until I know I need to write them… I think the important thing, as always, is to think and question, rather than just to blindly follow someone else’s advice. So, I urge you to take my blog with a healthy dose of scepticism, think about it, and come to your own conclusions; this stuff works for me, it might not work for you.