I’m looking into adding CLR deadlock detection into the CLR hosting code that’s used inside The Server Framework and, once again, the fact that you can’t cleanly shutdown the CLR host is causing me problems…
Since the CLR can’t be stopped by a host without terminating the host process (and that’s by design…) you need to be aware that any of the code that you have plugged into the CLR by way of the hosting interfaces can be called into during process shutdown.
It seems that Vista contains lots of interesting new Win32 API calls and some of these provide built in support for deadlock detection… I guess my deadlock detection tool can operate differently on Vista then…
When I came back from skiing in Colorado I had a bug report from a client and it took me a fair while to isolate the problem for them. The report suggested that a server that I’d built for them a while back was deadlocking in certain situations. The server had always had deadlock problems due to a couple of poor design decisions that I made early on and I’d built a tool to help me remove the potential for deadlock from it by analysing its lock acquisition patterns - the code was too complex for me to do this analysis by just looking at the code and although I should rewrite the poorly designed section of code I don’t have the time to do so (and it only causes problems when I make changes to the code, which doesn’t happen very often, and the problems can be caught by running my tool).
There’s an interesting article by Tomer Abramson in this month’s Dr Dobb’s Journal about deadlock detection. He provides a compile in tool that works in a similar way to my deadlock detection tool and reports on potential deadlocks in code even if they never occur during the program run in question.
His architecture is considerably different to mine but the idea is the same. By recording the sequence of lock acquisition and release on all threads in the program you can then examine the sequence that locks are taken out and spot potential deadlocks.
Jeff Darcy over at “Canned Platypus” writes about “How to add locking to a program”. He laments the lack of a reasonably priced deadlock detection tool. I assume, from his backgrond, that he’s interested in an Linux tool, so my deadlock detection tool wont help him much but it’s good to know that it’s not just me that thinks such a tool would be useful…
Jeff links to some expensive static code analysis tools that do deadlock detection.
Ned Batchelder has written about the code he uses to get a call stack out of a windows program (thanks for the link Barry). I’ve added a snippet of the code I use as a comment to his post.
Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.
I started looking into working with windows call stacks a while ago when I was working on my deadlock detection tool.
I use BlogLines to read my RSS subscriptions. It’s pretty good, and now that the performance issues I had initially seem to have gone away, I like it a lot. It’s very handy to be able to read my feeds from anywhere and always have them up to date and synchronised. One of the features I like is the little “keep new” check box that each item has; check it and the item stays unread.
I’ve been working on my deadlock detection and lock monitoring tool quite a lot this week; that and fixing the issues that it’s been highlighting. Yesterday I decided that I was drowning in data and that I really needed a GUI and, as I thought about how the GUI should work, I realised that I really didn’t have enough data.
Note: the deadlock detector mentioned in this blog post is now available for download from www.
I added some more monitoring to the deadlock tool. You can now view the entire life cycle of the locks, from initialisation, through usage to destruction. The lock usage figures put the contention figures in context as you can see how often the lock was acquired by each thread and compare that to how often there was contention…
Note: the deadlock detector mentioned in this blog post is now available for download from www.
The deadlock detection tool that I was working on last week is coming on nicely. I managed to locate two potential deadlocks that had never caused any problems in code that is running live on a client’s site. Once I had a report from my tool it was relatively easy to fix the problems. Both of these were real bugs that just hadn’t bitten yet.
Today I added some rudimentary lock contention monitoring to the tool and the results seem to be quite useful…
