Using Overlapped I/O from Managed Code

§ July 26, 2008 04:01 by beefarino |

I've been working on a project that requires some low-level device driver interaction using the DeviceIoControl Windows API call.  Without going into too much detail, the driver's programming interface requires a thread-heavy implementation to monitor low-level GPIO inputs for signal changes.  Fortunately Windows has built-in an asynchronous programming mode called Overlapped I/O that saves me the overhead of having to manage a bunch of threads for such low-level tripe.  In a nutshell, Overlapped I/O is the backbone of the asynchronous reading and writing of files and network streams in the .NET framework.

Unfortunately, there is no managed equivalent for device communication in the .NET framework; hand-rolling access to the DeviceIoControl API call is simple enough, but leveraging overlapped I/O from managed code requires some digging.  There are very few examples available, so I decided to post one in case anyone else out there happens to be working on such obscure (but very interesting!) projects too.

There are five steps to leveraging Overlapped I/O from managed code:

  1. Access the relevant routines from Kernel32.dll via P/Invoke;
  2. Create a file/device/pipe handle tagged for overlapped I/O;
  3. Bind the handle to the managed Thread Pool;
  4. Prepare a NativeOverlapped pointer;
  5. Pass the NativeOverlapped pointer to the relevant Win32 API (DeviceIoControl in my case).

Kernel32 P/Invoke

Using overlapped I/O requires some direct access to routines from kernel32.dll.  Below is the a static class that imports the routines and defines the constants needed.  If you have any questions or suggestions please feel free to post a comment.

using System;
using System.Runtime.InteropServices;
using System.Threading;
using Microsoft.Win32.SafeHandles;
namespace PInvoke
{
    /// <summary>
    /// Static imports from Kernel32.
    /// </summary>
    static public class Kernel32
    {
        const uint Overlapped = 0x40000000;
        
        [ Flags ]
        public enum AccessRights : uint
        {
            GENERIC_READ                     = (0x80000000),
            GENERIC_WRITE                    = (0x40000000),
            GENERIC_EXECUTE                  = (0x20000000),
            GENERIC_ALL                      = (0x10000000)
        }
        [ Flags ]
        public enum ShareModes : uint
        {
            FILE_SHARE_READ                 = 0x00000001,
            FILE_SHARE_WRITE                = 0x00000002,
            FILE_SHARE_DELETE               = 0x00000004  
        }
        public enum CreationDispositions
        {
            CREATE_NEW          = 1,
            CREATE_ALWAYS       = 2,
            OPEN_EXISTING       = 3,
            OPEN_ALWAYS         = 4,
            TRUNCATE_EXISTING   = 5
        }
        [ DllImport( "kernel32.dll", SetLastError=true, CharSet=CharSet.Unicode ) ]
        public static extern SafeFileHandle CreateFile(
            string lpFileName,
            uint dwDesiredAccess,
            uint dwShareMode,
            IntPtr lpSecurityAttributes,
            uint dwCreationDisposition,
            uint dwFlagsAndAttributes,
            IntPtr hTemplateFile
        );
        [ DllImport( "kernel32.dll", CharSet=CharSet.Unicode ) ]
        public static extern void CloseHandle(
            SafeHandle handle
        );
        [ DllImport( "kernel32.dll", SetLastError=true, CharSet=CharSet.Unicode ) ]
        unsafe public static extern bool DeviceIoControl(
            SafeHandle hDevice,
            uint dwIoControlCode,
            IntPtr lpInBuffer,
            uint nInBufferSize,
            IntPtr lpOutBuffer,
            uint nOutBufferSize,
            ref uint lpBytesReturned,
            NativeOverlapped *lpOverlapped
        );
    }
}

Creating the Device Handle

In order to leverage asynchronous I/O against the device handle, it needs to be opened specifically for asynchronous operations.  To do this use the CreateFile API call, passing the device path as the lpFileName parameter and specifying the Overlapped flag in the dwFlagsAndAttributes parameter:

SafeFileHandle deviceHandle = Kernel32.CreateFile(
    devicePath,
    ( uint )( Kernel32.AccessRights.GENERIC_READ | Kernel32.AccessRights.GENERIC_WRITE ),
    ( uint )( Kernel32.ShareModes.FILE_SHARE_READ | Kernel32.ShareModes.FILE_SHARE_WRITE ),
    IntPtr.Zero,
    ( uint )( Kernel32.CreationDispositions.OPEN_EXISTING ),
    Kernel32.Overlapped, // note the OVERLAPPED flag here
    IntPtr.Zero
);                    

Binding the Device Handle to the Thread Pool

The Windows Thread Pool manages 1000 threads per process dedicated to nothing but handling overlapped I/O.  In order to make use of them, it is necessary to "bind" the device handle returned by calling CreateFile to the thread pool.  This is done using the ThreadPool.BindHandle call:

ThreadPool.BindHandle( deviceHandle );             

The API documentation for BindHandle gives almost no indication as to it's purpose, but I've found this call is absolutely necessary in order to receive notifications of asynchronous operations.  (If you're interested in the details of what this call actually accomplishes, it seems to bind the handle to an I/O Completion Port owned by the Thread Pool; check out Chris Brumme's comments from this post on his blog for more.)

Creating a NativeOverlapped Pointer

Creating a NativeOverlapped pointer to pass to the DeviceIoControl API is relatively simple:

Overlapped overlapped = new Overlapped();
NativeOverlapped* nativeOverlapped = overlapped.Pack(
    DeviceWriteControlIOCompletionCallback,
    null
);             

After creating an instance of the Overlapped class, use its Pack method to create a pointer to a NativeOverlapped structure; the layout of this structure is byte-for-byte identical to the Win32 OVERLAPPED structure, which makes it suitable for passing directly to unmanaged API calls.  Note that the pointer returned by Overlapped.Pack() is fixed (pinned) in memory until the pointer is passed to Overlapped.Unpack and Overlaped.Free.

The first parameter to Overlapped.Pack is an IOCompletionCallback delegate that will be called when the overlapped I/O operation completes.  The second argument could be an array of objects representing buffers of memory used for passing data to and from the device driver; to keep this post as simple as possible I'm leaving that feature out of the example code.

The IOCompletionCallback delegate is mandatory and follows the following pattern:

unsafe void DeviceWriteControlIOCompletionCallback( uint errorCode, uint numBytes, NativeOverlapped* nativeOverlapped )
{    
    try
    {
        // ...
    }
    finally
    {
          System.Threading.Overlapped.Unpack( nativeOverlapped );
          System.Threading.Overlapped.Free( nativeOverlapped );
    }
}

The errorCode parameter will be 0 on a successful overlapped I/O operation, or a standard Win32 error code on a failure.  The numBytes parameter will contain the number of bytes received from the operation.

With whatever else needs to happen post-I/O, the fixed NativeOverlapped pointer must be unpacked and freed, or the application will both leak memory and fragment the managed heap over time.

Passing the NativeOverlapped Pointer to DeviceIoControl

With a NativeOverlapped pointer in hand, call DeviceIoControl:

 

const int ErrorIOPending = 997;
uint bytesReturned = 0;
bool result = Kernel32.DeviceIoControl(
    deviceHandle,
    ( uint )ioCtlCode,
    IntPtr.Zero,
    0,
    IntPtr.Zero,
    0,
    ref bytesReturned,
    nativeOverlapped
);
if( result )
{
    // operation completed synchronously    
    
    System.Threading.Overlapped.Unpack( nativeOverlapped );
    System.Threading.Overlapped.Free( nativeOverlapped );
}
else
{
    int error = Marshal.GetLastWin32Error();    
    if( ErrorIOPending != error )
    {
        // failed to execute DeviceIoControl using overlapped I/O ...
        
        System.Threading.Overlapped.Unpack( nativeOverlapped );
        System.Threading.Overlapped.Free( nativeOverlapped );
    }
}

 

The nativeOverlapped pointer and overlapped device handle cause the kernel to attempt an asynchronous execution of DeviceIoControl.  

The call will return immediately with a boolean result whose semantics are a bit awkward.  If the result is true, it means the operation completed synchronously as opposed to asynchronously.  A false result indicates the operation may have be executing asynchronously.  To verify this, check the Win32 error code via Marshal.GetLastWin32Error: if it's ERROR_IO_PENDING (997), the DeviceIoControl operation is executing asynchronously and the IOCompletionCallback delegate will be called when the operation is finished; otherwise, the error code will indicate why DeviceIoControl failed to execute asynchronously.

If DeviceIoControl executes synchronously, or if it fails to execute asynchronously, the NativeOverlapped pointer must still be unpacked and freed to avoid leaking memory and fragmenting the managed heap.

That's the nut of using Overlapped I/O from managed code.  If you do end up using this for something, please drop me a line, either as a comment to this post or using the contact link on this site; I'd love to hear what you're working on!



On Smarts, Curiosity, and Learning

§ July 6, 2008 14:22 by beefarino |

Justin Etheredge recently posted Being Smart Does Not a Good Developer Make, in which he offers several important perspectives in his usual good-hearted way; the one that sparked my neurons was that drive and curiosity are more valuable than raw knowledge to becoming a "good developer."

I whole-heartedly share this perspective.  This is probably due to the fact that I've painted myself into a professional corner in many respects.  I have no formal education in my chosen profession of software engineering, which has historically put me at a disadvantage in job interviews, when I push to lead a project, and in a lot of interaction with my peers.  Nevertheless, I've been able to land the jobs, get the project leads when I want them, and feel confident - if reservedly so - talking with my fellow engineers without sounding like a total poser.

I attribute my success to three things:

  1. I've been pretty lucky with my opportunities;
  2. I'm insatiably curious about most things;
  3. I have two degrees in psychology. 

Justin calls software engineering a meta-industry; likewise, psychology is probably the second-best example of a meta-science I can think of (with first place going to cryptography).  Digesting all that research on learning, perception, and memory has given me a significant leg-up when it comes to learning new things, understanding someone else's perspective, and communication.

I honestly believe that having and honing these skills is what keeps me employed as a software engineer.

If it was possible for me to add any value to Justin's post, it would be to not limit your exploration to your career.  Learning is an active, forceful process - go find something that interests you and relate it back to software design and engineering.  Psychology may not be your bag, but learning opportunities abound.  Here are two ideas I've taken from my own personal history:

Design and build a birdhouse (or, if you have some modest woodworking skills, a tree house or bookshelf).  If you really want a challenge, have someone else design it.  What sort of problems did you hit during design vs. construction?  How well did you estimate your materials?  What sort of mistakes ended up in the final product, and how would you avoid them next time?

Cook a menu by a fickle four-year-old.  This is a great exercise in communicating across domains of expertise: you could be a Master Chef, but you'll have no idea how to make Glittery Rainbow Unicorn Fishsticks or Chocolate Marshmallow Beans and Rice without some discussion.  Moreover, try explaining to them why their culinary delights don't taste as delightful as they expected.  Work with them to hone their ideas and produce something palatable.  Think about the communication problems you experienced and how you worked around them - they have a lot in common with communication problems you experience at work.