The great source code shuffle...

I’m in the process of moving my source code from a CVS repository to Subversion. The main reason for the move is so that I can move the repository onto one of my NAS devices so that it can be backed up automatically. I have a spare NAS due to the fact that I had a power supply fail and the fastest way to get up and running again was to buy a new one rather than waiting for the replacement PSU… Anyway, the development NAS has SVN running on it which means that I now have a RAIDed, UPS protected SVN server which gets backed up nightly to my ‘off site’ NAS via rsync…

I’ve been meaning to move from CVS for a while but the fact that I made heavy use of CVS ‘modules’ meant that it was complicated to move to SVN as there was no suitable replacement for ‘modules’. I eventually decided to rework my source control usage to fit with SVN and to try and iron out a few of the wrinkles that have developed over the past 6 or so years that I’ve been using CVS.

The most notable improvement so far comes at the cost of duplicating some source… Many of my projects use source from my ‘JetByteTools’ libraries. In CVS I used a single repository for all source and relied on tags and branches to make sure that each project got the right version of shared code. This worked OK but was complicated to manage at times and the tools libs ended up with lots and lots of tags and branches. Every now and then I’d want to upgrade a project to use the latest version of the tools libs and this was complex to achieve. The CVS situation had evolved over several years and, to be honest, was far from ideal. One of the main problems was that it wasn’t easy to remove projects when they were no longer required; old client code is often moved out to an external off-site archive and this can be complex to achieve if everything is in the same repository. So, anyway, when I started my move to SVN I decided to have a repository per project so that I could completely remove and archive old stuff without affecting work in progress (simply burn the repository to dvd and file it). To make sure that all repositories were self contained they contain copies of the tools libs that they need. Rather than simply branching within a repository I effectively branch into a new repository. So far this works well and merging to and from the ‘main’ tools library repository is no more complex than when everything was in one repository and on separate branches…

Once that was done I decided to clean up how The Server Framework’s examples were stored. Way back when I first developed The Server Framework I decided that it was important that each example be buildable and packagable as a standalone thing. At the time I had half a dozen examples and one client might purchase features that required one or two different example servers to demonstrate. I now have around 8 different “packs” of examples, each of which contains several example servers… The problem with the original source tree design was that each example within a “pack” had its own copies of the shared tools library code. This was so that you could extract and zip a single directory to get a single example. Recent releases have included a script which copies a single set of tools lib source into the right places for the whole “pack”, and the only thing that was keeping me from changing to a more useful structure was the momentum of my existing CVS source control system. Well, since I’m changing things anyway I decided to change the way the example “packs” work. Now they’re slightly more fine grained, slightly more descriptive and each “pack” only includes a single copy of the code that’s shared between all of the examples in the pack. I can still build ‘single example’ structures if I need to (and I haven’t needed to for a long, long time) but for the usual use case things are easier. Each pack now has a script which can be run to pull all of required tools libraries from the repository into the right place and another script which will strip out all of the stuff that doesn’t ship to clients (tests, mocks, etc). This seems to work well, reduces duplication, avoids the need for CVS modules and looks like it should scale better.

Finally I’ve taken a long hard look at the tools libraries and decided that some of them aren’t worth keeping as separate libraries. The C++Tools lib was born back in 1997 when it was possible that we’d be developing cross platform code, at the time it made sense to keep stuff that wasn’t tied to the windows platform in its own library. Now most of what’s in there could be replaced by things from boost should I decide to add that as a dependency and the rest may as well live in the Win32Tools library. Likewise the RegistryTools library which was only separate because it was the subject of an article on CodeProject and which has missed out on several improvements because it wasn’t in the Win32Tools lib (and so was easy to ignore).

The result of all this reshuffling wont be seen in The Server Framework until release 6 which is due near the end of Q1 2009. Due to the library reorganisation it’s likely that there will need to be some header file and namespace usage changes in client code, but hopefully these will be minimal.