Hacking for Fun and ... more Fun

§ November 12, 2009 16:57 by beefarino |

We had big fun tonight at the first Charlotte ALT.NET user group hack night.  Special thanks to Mike Linnen and Xpient for hosting the event.

The theme of the evening was taken from the recent Raleigh and Carolina Code Camps: Rock Paper Scissors.  Teams coded modules to play rounds of RPS, uploaded their assembles to a server, and duked it out over three rounds of riveting RPS action.

I was a bit apprehensive about the event, mostly afraid of looking like an idiot in front of some really smart people.  But once the hacking started we all fell into a groove and it turned out to be a lot of fun.  I chose to try and predict my opponent's actions using a bit of fuzzy logic and some statistics.  It worked fairly well through the first two rounds of the tournament, but I was given a resounding spanking in the final round by @micahlmartin and @DeveloperInfra.  In the end, a three-way tie for first place was resolved with a fourth round, where  @micahlmartin edged us out with a perfect score!

I may blog about my approach to the puzzle some other time, but I wanted to jot down some things I noticed this evening.

When I'm under a time contraint, I can get a lot of work done quickly.  I don't know why this surprises me, but after tonight I'm considering adding stricter timeboxes to my daily GTD routine.

Hacking is fun.  Lately coding has started to feel monotonous.  A lot of the projects I've got going on are hitting non-techcnial walls, and moving forward is proving to be difficult.  It was a blast to hammer out some code without those pesky business boundaries to bring it all down.  I think scheduling some regular geek time to work on things like Project Euler or some code katas would be a good idea - no expectations past my own enjoyment.

Overall a very positive experience, thanks everyone!



Delegates and Native API Callbacks - Answer

§ April 16, 2009 16:35 by beefarino |

A while back I posted a little puzzle about an exception I was hitting after passing a delegate to a native API call.  In a nutshell, my application was passing an anonymous delegate to an unmanaged library call:

Result Code result = ExternalLibraryAPI.Open(
    deviceHande,
    delegate( IntPtr handle, int deviceId )
    {
        // ...
    }
);
VerifyResult( result ); 

The call returns immediately, and the unmanaged library would eventually invoke the callback in response to a user pressing a button on a device.  After a semi-random period, pressing the button would yield an exception in my application.  The questions I asked were:

  1. What's the exception?
  2. How do you avoid it?

I've waited a bit to post the answers to see if anyone besides Zach would chime in.  Zach correctly identified the nut of the problem - the delegate is being garbage collected because there is no outstanding reference on the anonymous delegate once the unmanaged call returns.  With no one referencing the delegate object, the garbage collector is free to reclaim it.  When the device button is pushed, the native library invokes the callback, which no longer exists in memory.  So, to answer the first question, the specific exception that is raised is a CallbackOnCollectedDelegate:

CallbackOnCollectedDelegate was detected.
Message: A callback was made on a garbage collected delegate of type 'Device.Interop!Device.Interop.ButtonPressCallback::Invoke'. This may cause application crashes, corruption and data loss. When passing delegates to unmanaged code, they must be kept alive by the managed application until it is guaranteed that they will never be called.

The verbage in this exception message answers my second question.  To avoid the exception, you need to hold a reference to any delegate you pass to unmanaged code for as long as you expect the delegate to be invoked.  You need to use an intermediary reference on the delegate to maintain it's life.

Based on this, any of these examples are doomed to fail eventually, because none of them maintain a reference on the delegate object being passed to the unmanaged library:

ExternalLibraryAPI.Open(
    deviceHande,
    delegate( IntPtr handle, int deviceId )
    {
        // ...
    }
);
 
ExternalLibraryAPI.Open(
    deviceHande,
    new ButtonPressCallback( this.OnButtonPress )
);
 
ExternalLibraryAPI.Open(
    deviceHande,
    this.OnButtonPress
);

The correct way to avoid the problem is to hold an explicit reference to the specific delegate instance being passed to unmanaged code:

ButtonPressCallback buttonPressCallback = this.OnButtonPress;
ExternalLibraryAPI.Open(
    deviceHande,
    buttonPressCallback
);
// hold the reference until we're sure no
// further callbacks will be made on the
// delegate, then we can release the
// reference and allow it to be GC'ed
buttonPressCallback = null;

At first I thought Zach's pinning solution was correct; however, you can only pin blittable types, of which delegates are not, so "pinning a delegate" isn't even possible or necessary.  If you're interested, the details of how delegates are marshalled across the  managed/unmanaged boundary are quite interesting, as I found out from Chris Brumme's blog:

Along the same lines, managed Delegates can be marshaled to unmanaged code, where they are exposed as unmanaged function pointers.  Calls on those pointers will perform an unmanaged to managed transition; a change in calling convention; entry into the correct AppDomain; and any necessary argument marshaling.  Clearly the unmanaged function pointer must refer to a fixed address.  It would be a disaster if the GC were relocating that!  This leads many applications to create a pinning handle for the delegate.  This is completely unnecessary.  The unmanaged function pointer actually refers to a native code stub that we dynamically generate to perform the transition & marshaling.  This stub exists in fixed memory outside of the GC heap.

However, the application is responsible for somehow extending the lifetime of the delegate until no more calls will occur from unmanaged code.  The lifetime of the native code stub is directly related to the lifetime of the delegate.  Once the delegate is collected, subsequent calls via the unmanaged function pointer will crash or otherwise corrupt the process. 

Thanks again Zach, and to everyone who reads my blog!



Delegates and Native API Callbacks

§ March 27, 2009 02:07 by beefarino |

For one of my contracts I'm implementing a .NET layer over a native third-party SDK.  The SDK makes heavy use of function pointers, which means, in terms of p/invoke and interoperability, I've got a lot of delegates flying around.  I recently hit a rather nasty bug in my .NET layer - see if you can figure this one out.

Consider this truncated example derived from the SDK:

typedef void (__stdcall *PCALLBACK)( HANDLE deviceHandle, LONG32 deviceID );
// ...
API_CALL RESULT_TYPE __stdcall Open( HANDLE deviceHandle, PCALLBACK pfnCallback );

In a nutshell, when Open is called, the pfnCallback function pointer is registered, along with the handle.  The Open function returns almost immediately, and the native library invokes the callback function periodically in response to user interaction with a device.

Here is the C# that enables the Open method to be called from .NET:

public delegate void Callback( IntPtr deviceHandle, int deviceID );
// ...
[DllImport( "ExternalLibrary.dll")]
public static extern ResultCode Open( IntPtr deviceHandle, Callback callback );

And here it is in use:

Result Code result = ExternalLibraryAPI.Open(
    deviceHande,
    delegate( IntPtr handle, int deviceId )
    {
        // ...
    }
);
VerifyResult( result ); 

The call to Open succeeds, and the callback is invoked whenever the user pushes the appropriate button on the device.  Eventually though, a nasty exception is raised as the user continues to fiddle with the device. 

I will tell you that:

  • there is no context or stack information to the exception;
  • there is not a bug in the third-party SDK;
  • this post has all the information you need to find and fix the problem - i.e., I'm not hiding something from you.

So my questions to you are:

  1. What's the exception?
  2. How do you avoid it?
Leave your answers as comments. I'll post the answer in a few days.