Using OpenSSL with Asynchronous Sockets

Reprints, Socket Servers, Source Code

OpenSSL is an open source implementation of the SSL and TLS protocols. Unfortunately it doesn’t play well with windows style asynchronous sockets. This article - previously published in Windows Developer Magazine and now available on the Dr. Dobbs site - provides a simple connector that enables you to use OpenSSL asynchronously.

Note: If you need high performance SSL over TCP/IP then this is available as an optional part of The Server Framework; see here for details.

Originally published in Windows Developer Magazine, Copyright 2002.

The Secure Sockets Layer (SSL) is used by every commercial web browser and server to secure web traffic. Every time you use a URL that starts https://, you’re using SSL. Microsoft provides an implementation of SSL via the SChannel API, but it is a low-level API that is quite complex to use. OpenSSL is a higher level, open-source alternative that allows you to easily add the security of an SSL connection to TCP/IP clients and servers. However, since the OpenSSL code has a UNIX heritage, most of the examples available assume a UNIX-style sockets architecture. The preferred Windows architecture of asynchronous sockets doesn’t fit well with this, and this article presents an “asynchronous connector” for OpenSSL that allows you to use OpenSSL in a way that is more appropriate on the Windows platform. The resulting connector can be used with MFC’s CAsyncSocket for client-side use and integrates well with server code that uses IO Completion Ports. Due to the original design of the OpenSSL toolkit, the connector can be built without having to change any of the OpenSSL code and so is standalone and easy to maintain.

OpenSSL

The OpenSSL Project is a collaborative effort to develop a robust, commercial-grade, full-featured, cross-platform, and open-source toolkit implementing the Secure Sockets Layer (SSL v2/v3) and its successor, Transport Layer Security (TLS v1). OpenSSL is based on the SSLeay library developed by Eric A. Young and Tim J. Hudson. The OpenSSL toolkit is licensed under an Apache-style license, which basically means that you are free to get and use it for commercial and noncommercial purposes, subject to some simple license conditions. However, one thing you should be aware of is that OpenSSL uses strong cryptography, and this falls under certain export/import restrictions in some parts of the world. You are strongly advised to pay close attention to any import/export laws or restrictions that apply to you.

The source-code distribution of OpenSSL provides many example programs that show you how to use it. There are some useful tutorials on the Web and books available on the subject (see References). The problem is that although the OpenSSL toolkit source code is cross platform, the example code tends to be written in a UNIX style, using blocking sockets and multiple threads or processes. When developing with sockets on the Win32 platform, it’s common to want to use nonblocking, asynchronous sockets: The most scalable method of developing TCP server applications with Win32 is to use IO Completion Ports, which forces you to deal exclusively with asynchronous sockets; and using blocking sockets in a thread that serves a user interface is a recipe for disaster.

Although developed in C, the OpenSSL toolkit has an object-based design, with various components accessed by passing an object handle as the first parameter to the C API calls. The main object that we’re interested in is the SSL object. This is where the actual SSL protocol is implemented. The developers of the OpenSSL toolkit designed the SSL object to use an abstract IO mechanism, the BIO, rather than hard coding it to use sockets. The BIO abstraction allows you to develop a BIO that suits your own needs and simply plug it in to the SSL object to run the protocol over whatever data stream you like. The toolkit comes with a BIO that works with a memory buffer, and one of the sample programs uses these memory BIOs to test the toolkit by running the protocol over memory buffers in a single process that is acting as both client and server side of the protocol. We can use this memory buffer BIO to provide an interface to the SSL object that’s usable with asynchronous sockets.

The SSL object often needs to read and write to the data stream independently of your application. You may wish to perform a write, but if an SSL handshake is in progress, then the SSL object may need to perform several reads and writes before your application data can be sent. Likewise, data being received from the data stream needs to be processed by the SSL object to decrypt it before passing it up to the application but, due to the nature of the protocol, the arrival of data may require the SSL object to write to the data stream.

A Simple C++ Wrapper

Given the aforementioned information, we can begin to develop a C++ class that provides a clean interface to the OpenSSL memory BIOs that we will use to communicate with the SSL object. The object will act as a filter between the application and the socket. The interface to the class should provide read and write methods for pushing data into the SSL object, and be able to call back into the application when decrypted application data is available and when encrypted writes to the data stream are required. Since we’re intending to operate in an asynchronous manner, and since the SSL object may not always be ready to process our application data immediately, we will need to be able to buffer data prior to passing it through our filter object. The buffer management will vary depending on how you want to use the filter, so rather than including it within the main filter class, we simply provide hooks so that derived classes can implement their own buffering scheme. Removing the uffer management code also allows us to see the actual code required to drive the SSL object more clearly.

COpenSSLConnectorBase provides the derived class with methods to push data into the SSL object. Unencrypted application data is pushed through the SSL object by calling DataToWrite() and will eventually emerge, ready to be written to the data stream, via the OnDataToWrite() virtual function. Encrypted data that arrives from the data stream is pushed into the SSL object by calling DataToRead(), and application data will emerge via the OnDataToRead() virtual function. Since an application write may require the SSL object to process responses read from the peer, the derived class should not call DataToWrite() and DataToRead() directly. Instead, when it has data that needs to be passed through the SSL object, it should buffer the data and then call RunSSL(). This will deal with any SSL-level data transmission and request read and write data from the derived class using the GetPendingOperations(), PerformRead(), and PerformWrite() virtual functions. GetPendingOperations() simply returns two boolean values that are set to True if the derived class has read or write data buffered and ready to pass to the SSL object. When the read or write data is required, PerformRead() or PerformWrite() will be called. These should call DataToRead() or DataToWrite() as appropriate to pass the buffered data to the SSL object. These calls take a pointer to a BYTE buffer and the length of that buffer; they return the number of bytes consumed. The caller should then adjust the buffered data to take into account the bytes that have been consumed and returned.

Since both read and write operations can result in the SSL object generating data that must be transmitted to the peer, the actual transmission is done as part of the RunSSL() method. If, after reading or writing, there is data to be transmitted, SendPendingData() is called and this extracts the data from the SSL’s output BIO and calls OnDataToWrite() to pass it to the derived class for writing to the data stream.

CAsyncConnector is a class derived from COpenSSLConnectorBase that provides an example of simple buffering and integrates the SSL object with the MFC CAsyncSocket class.

The AsyncClient sample code puts all of this together in a simple adaptation of the Microsoft sample code detailed in Microsoft Knowledge Base, Article Q192570. The client presents a dialog interface that allows you to enter a server name to connect to and a URL to retrieve. The application connects to the specified server on port 443, the port commonly used for HTTPS servers, and sends an HTTP Get request for the specified URL. The results are displayed in an edit box. By stepping through the code and watching the debug traces, you will see the SSL connection established and the encrypted request sent. The results arrive asynchronously and are pushed through the SSL protocol to decrypt them before displaying them in the edit box.

Obviously, it would be easier to use the WinInet functions if you actually wanted to write a simple HTTPS client. The example does not attempt to validate the server’s credentials or force the client to require a certificate or dictate which cipher suite to use; all of this is possible and you should consult the OpenSSL samples for details of how to use its more advanced features. The example simply demonstrates the ease at which OpenSSL can be integrated with CAsyncSocket. Using these techniques with the OpenSSL toolkit, you can easily protect any data stream with SSL.

A Potential For Optimization

The OpenSSL toolkit BIO interface includes a method of accessing the internal BIO data buffer directly, rather than passing in a new buffer for the data to be copied into or out of. This interface is exposed via the BIO_nread/nwrite methods. Unfortunately, only the BIO_pair BIO implements these methods. A BIO_pair could be used in place of our two memory BIOs and would allow us to extract data directly from the BIO in DataToWrite() and SendPendingData(); however, doing so complicates the example and so is left as an exercise for the reader.

Building the Sample Source Code

To build the sample source code you need to have OpenSSL installed; see the OpenSSL web site for details of how to obtain and install the toolkit. Once you have installed OpenSSL, you need to adjust the sample project so that it can find the toolkit headers and libraries. In the project settings dialog, on the C/C++ pane, in the Preprocessor category, change the additional include directory from “I:\openssl-0.9.6d\ inc32” to the include directory that is appropriate for your installation of OpenSSL. Likewise on the Link pane, in the input category, change the additional library path from “I:\openssl-0.9.6d\out32dll” to something more appropriate.

References

Download

The following source was built using Visual Studio 6.0 SP5. See above for other requirements.

SSLAsyncClient.zip - the sample code

Revision history

Note: If you need high performance SSL over TCP/IP then this is available as an optional part of The Server Framework; see here for details.