Socket connection termination

I’ve been putting together a sample server for a client that shows how to cleanly terminate a socket connection. This should have been a simple thing to do, but in doing so I’ve discovered some gnarlyness within The Server Framework and the result has been some new TODO items for the 5.3 release…

When you have an active TCP/IP connection that you wish to terminate cleanly you need to initiate a TCP/IP protocol level shutdown sequence by calling shutdown(). This sends the appropriate packets between the two TCP/IP stacks (server and client) and terminates the connection. Once this is done you can close the socket by calling closesocket(); this cleans up the resources used by the socket (and associated data structures) within your program. Closing the socket without initiating the protocol level shutdown sequence implicitly triggers the shutdown sequence. This is explained here “Graceful shutdown, linger options, and socket closure”.

Simple servers written using The Server Framework tend to operate as follows: When an incoming connection is detected an asynchronous read is issued, this increments a reference count on our socket class. When a read completes the last thing that happens before the function returns to the calling code within the framework is that a new read is issued. If the client closes the connection the pending read within the server returns with 0 bytes read, this is interpreted as a ‘client close’ and no further reads can be issued on the socket. This, eventually, causes the reference count on the socket to fall to 0 and the socket gets cleaned up. Part of that clean up involves calling closesocket(). If the server wants to terminate the connection then it calls Shutdown(ShutdownSend) on its socket to indicate to the client that it has no more data to send and this eventually results in the client shutting down its socket and the server socket cleanup sequence that I described earlier.

Due to the way the server is designed, there’s some ‘clever stuff’ in there to make sure that if you have several writes pending but not yet issued by the framework then the call to shutdown() occurs after the last write has actually been passed off to the TCP/IP stack.

The socket class also exposes a Close() method which calls closesocket() on the socket directly; that is it doesn’t do ‘clever stuff’ to deal with outbound data that is ‘in flight’ within the framework. You probably don’t want to call Close() unless you don’t care if the data gets to the other end or not; or if you know that there’s no data ‘in flight’.

It gets more complex…

Due to either my misreading of the docs for closesocket() (or the fact that they were originally less clear and have since been clarified) it was my belief that a graceful shutdown using closesocket() would block. Since one of the most important design decisions of the framework is that work done on the I/O threads should not block the default behaviour for the automatic socket closure that happens when a socket is being cleaned up is for the close to be a ‘hard’ or ‘abortive’ close. That is we deliberately choose not to linger. Because this isn’t always what you want (no kidding!) there’s some code in there that allows a user to intercept the default behaviour and, potentially, call CloseSocket() yourself or to marshal the CloseSocket() call off to your own threads so that it could block them instead. However, graceful shutdowns that occur due to closesocket() do not block, so, it seems, most of that code isn’t really needed…

Some of the example servers in the 5.2.1 release get this wrong, it doesn’t cause them to lose data, since they’re not doing anything that complex, but a more complex server that has been modelled on one of the examples may have problems. If you’ve been having this problem then I’m sure you’d have contacted me already, but, if not, do get in touch and I’ll help sort things out for you.

So, in summary, at present, in version 5.2.1 of The Server Framework or before, you should generally be calling Shutdown() to terminate your connections and the framework will deal with the resource cleanup and eventual call of closesocket() itself. You can call Close() but you shouldn’t do that unless you KNOW that there cannot be any data ‘in fligh’ that the server has sent but that the client might not have recieved, OR you don’t care if the data gets to the client.

This will become nicer in 5.3, I hope. I plan to make “standard” connection termination easier to manage and provide access to the, currently private, AbortiveClose() method on the socket class; this sets the socket’s linger options in such a way that the socket is closed immediately and all pending data is discarded. What’s especially useful is that this also sends a RST (reset) on the TCP/IP connection and this closes the connection without putting the closer into the TIME_WAIT state; which is useful sometimes.