I’m currently working on a simple auction server for a client using The Server Framework. You can think of it as a specialised chat server, of sorts. One of the things it must do is broadcast messages from one user to a set of users. This is relatively easy to implement in a crude way in The Server Framework but it’s not nice. Time to refactor towards niceness…
The lack of niceness in the quick and dirty implementation of broadcast messages is the fact that the message must be duplicated for each client that will receive the broadcast. This sucks for many reasons; it’s not especially elegant, it’s not especially necessary, it involves doing work that needn’t be done and it hogs memory. Unfortunately our last bout of server framework refactoring stopped short of where we needed to be to be able to fix these issues.
The Server Framework uses fixed sized buffers that are managed by a buffer allocator (which can pool them for later use and pre-allocate a set number of buffers, etc). The buffer consists of a blob of memory that’s used as the actual data buffer plus an
OVERLAPPED structure for using the buffer in overlapped IO operations and a
WSABUF structure for using the buffer in socket operations… When we wish to broadcast a buffer we want to reuse the memory blob and, possibly, the
WSABUF but we need a unique
OVERLAPPED structure so that each overlapped write operation has its own workspace… The framework could allocate its own
WSABUF structures and pool them, etc, but during the original design phase I decided to limit the number of memory allocations required during message processing by incorporating the buffer and all of its supporting structures in a single allocation - seemed like a good idea at the time and it’s stood the test of time pretty well; until now.
What we’d really like to be able to do is attach multiple
OVERLAPPED (and just to be on the safe side,
WSABUF) structures to a single blob of memory. This would allow us to receive a message in a buffer and then transmit that buffer to multiple clients without needing to copy the message from the input buffer into each of the output buffers.
As usual the reason the existing design isn’t quite flexible enough is because of the use of concrete classes rather than interfaces. The server deals in terms of
CBuffer objects rather than in terms of an
IBuffer interface. So, this morning, I slipped in an interface to decouple the users of the buffer class from its current implementation. That was reasonably painless, so we then added a ‘read only buffer handle’ class that implemented
IBuffer and provided its own
WSABUF structures but references the original data in the original
IBuffer implementation. This worked nicely and greatly reduced the memory usage of the server and improved the performance of the broadcasting.
The next step is to generalise the
IAllocateBuffers interface so that we can share the pooling that we use for the real buffers. Once that’s done it would be good to completely decouple the book-keeping parts of the buffer from the data parts, then, perhaps we can support buffers that consist of multiple memory buffers and that present arrays of
WSABUFs to the socket layer…