Testing

So much for testing

My category-based RSS feeds stopped feeding through to Testing Reflections and Test Driven a couple of weeks ago. They had worked fine, and then they just stopped. I assumed that the problem was at their end; either they didn’t think much of my testing posts or their feed polling software had some weirdness in it… But you know what assume does… I’d got it into my head that because the feeds used to work and now they didn’t, and that I hadn’t changed anything at my end, the problem must be the consumers… Wrong… My feed had crap in it because I don’t really understand how the MT templates work, I’d been sloppy and I hadn’t bothered to test the feed apart from running it through my aggregator and eyeballing it in IE.

Looking back from near the end

My current consulting gig is coming to an end. We’ve been in the hand over phase for a while now and I think it’s slowly starting to work. Now seems like a good time to look back at what worked and what didn’t… This project was heavily focused on testability, from the outset and that’s paid off in the later stages of the project. As we near final integration testing lots of last minute requirements keep popping up; some were mentioned earlier in the project and just got missed out and some are completely new.

Practical Testing: 13 - Missing functionality

Previously, on Practical Testing: we added a multi-threaded version of our timer queue that manages timeouts automatically. This time we’ll integrate our new timer queue into an application that uses the old version of the code and, along the way, discover some functionality that the original version supports but that the new version currently doesn’t. One of the original reasons that we designed this timer queue was to handle per socket, read timeouts in our IO Completion Port based server framework.

Measure twice, cut once

I’m currently working on a small auction server for a client using The Server Framework. It’s a relatively simple server, messages come in, are validated and are then broadcast to interested parties and logged. Yesterday we shipped the first cut of the source to them and I was a bit concerned that our simple test app could thrash the server so easily. Admittedly the first cut of the code hadn’t been tuned at all and the logging implementation was fairly Mickey Mouse but I was a little concerned.

Practical Testing: 12 - Threading is orthogonal

Previously, on Practical Testing: we finished our reworking on the code we were left with a simpler timer queue that passed all of our tests, but was single threaded and needed manual intervention from client code to manage timeouts. Today we’ll add back the multi-threaded functionality so that the queue manages timeouts automatically. Our original implementation of the callback timer queue used a thread to manage the timeouts. The threading was integral to the timer queue code and this made testing the queue logic difficult because, to test the queue, we had to control the thread.

11th October is Thanksgiving Day in Canada

I’ve had an interesting week. I’m back with the Refactoring Project and, although things were looking up last time I was here, they’ve managed to adopt some bad habits in my absence. The latest build that’s live with users isn’t tagged in CVS; we have 66 other builds that are tagged and the ability to rebuild an arbitrary release has helped on numerous occasions, but this time they decided not to bother.

Tests as tours

I’ve discovered something quite amazing this week; something quite simple that shouldn’t really have been much of a surprise, but it was. If you have a substantial set of unit tests then you can use them as the backdrop for documentation that tours the code base in a controlled and manageable way. Kinda like a sing-along, the reader can be directed through several key tests in a debugger whilst the document describes what’s going on and reasoning behind some of design decisions.

Yup, that's where the value is

Mats writes about how he finally got unit testing by realising that the value comes not when all your tests pass and you feel happy but when one of your tests fails and you feel sad… This is actually something I’m not sure I’d quite realised for myself. I’m pretty good now; I know why I should write tests, I know that I feel more comfortable making changes when I have tests to support me, but I never really accepted that having all the tests passing doesn’t really buy you much.

Code coverage and testing

Recently I mentioned that we were in the process of adding additional tests to our code base. We’d been using JITT to reduce the number of tests that we wrote and now it was time to fill in some of the gaps. This week I started to use some code coverage tools to see how well we were doing with our tests… My client has both Compuware DevPartner Studio, which includes a coverage tool called True Coverage, and Rational Pure Coverage.

Repaying the technical debt

The boss is away, we’ve almost done all the things he asked us to do before he went and, well, there’s a bit of spare time so we’re looking closely at the code. The last few weeks have been a bit of a push for the finish and now I’m looking at stuff and trying to work out how to repay the technical debt we accrued during the push. Sure, I could be surfing the web and kicking back but, well, it just seems appropriate to work out how much we now owe to the code and how we can pay that back before we’re asked to push a little further and incur a little more debt… It doesn’t hurt that I find this kind of work relaxing and not that much like work…