Expanding-File Resource for Spring.NET

§ May 13, 2009 03:48 by beefarino |

I just had a fun emergency - the crew decided to redirect all configuration sources using environment variables, and while Log4Net supports this quite easily, the spring.net context resource-from-a-URI abstraction does not.  E.g., this will not work:

<spring>
    <context>
            <resource uri="file://%PROGRAMFILES%/myapp/ioc.config" />
    </context>
    ...

Knowing that spring.net can leverage custom resource implementations, I set out to extend the file system resource to expand environment variables.  Turns out to be easy-peasy-lemon-squeezie:

public class ExpandableFileSystemResource : Spring.Core.IO.FileSystemResource
{
    public ExpandableFileSystemResource()
        : base()
    {
    }
    public ExpandableFileSystemResource( string resourceName )
        : base( Environment.ExpandEnvironmentVariables( resourceName ?? String.Empty ) )
    {
    }
    public ExpandableFileSystemResource( string resourceName, bool suppressInitialize )
        : base( Environment.ExpandEnvironmentVariables( resourceName ?? String.Empty ), suppressInitialize )
    {
    }
}

Register the resource implementation via config:

<spring>
    <resourceHandlers>
      <handler protocol="filex" type="MyApp.ExpandableFileSystemResource, MyApp"/>
    </resourceHandlers>
    <context>
            <resource uri="filex://%PROGRAMFILES%/myapp/ioc.config" />
    </context>    
    ... 

 And it just works.  Enjoy!



Andy Hunt's Agile Carolinas Talk

§ April 28, 2009 14:11 by beefarino |

I just got back from hearing Andy Hunt discuss his new book, Pragmatic Thinking and Learning.  I hadn't heard him speak before, and I have to admit I was not sure what to expect.

On the one hand, I hold many of this books in very high regard.  In particular, the following books were career-altering reads for me:

And his keystone role in the Agile movement has earned my utmost respect.  However, when I saw the title of this latest book I was a bit worried.  I have a master's degree in cognitive psychology, and I know a lot about learning, memory, and perception.  Most learning books outside of the field are crap, so honestly my first instinct was that this was going to be a cash-grab treatise of self-help porch psychology fuzzy-feel-goods for software developers.

After listening to Andy's presentation, I am happy to say my instincts were way off the mark.

First of all, the book (and Andy's presentation) remains true to the pragmatic banner.  His recommendations are both practical and effective.  For example, a recurring theme during this talk was to write things down.  Carry around a small notebook and jot down every idea that pops into your head.  Maintain a personal wiki.  Mindmap as you read or think.  Solidify your thoughts into something concrete.  The point is not to be able to refer back to them later, but instead to force your brain to keep working, keep producing, keep processing.  To summarize one of his points, that's the first step to an accomplishment.

Second, a lot of what he was saying is supported by academic research.  Granted, Andy takes some license with his metaphors but his points hold water.   E.g., what Andy refers to as the "memory bus" being shared between "dual cores" of the brain is probably more of an attention effect; however, the behavioral effect cannot be denied - the serial and holistic parts of your mind compete when trying to solve a problem.

Based on the presentation tonight, I wouldn't recommend the book to my former psychology colleges - it's too macro to be useful to them.  However, for my fellow geeks, this is actually a useful introduction to becoming a more effective learner.

It was a great talk, a fun evening, and I plan to pick up the book next time I'm at the bookstore.  Oh, and I had a short chat with Andy just before the talk, and I have to say how awesome it is to meet someone you hold as a guru and detect no ego.  Awesome.



Log4Net Recommended Practices pt 2: Isolating Bugs

§ April 21, 2009 04:56 by beefarino |

I realize it's been a while since I've written about logging, but my experiences this morning compelled me to share.  The muse for this post was a very simple bug located in a rat's nest of complexity.

The code in question drives the I/O layer in a device that is filled with proprietary hardware.   There are about three dozen channels of internal communication when this thing is running, which doesn't include any of its networking.  So lots of asynchronous I/O and threading.  All supporting an awesome graphics and physical interface layer that is equally complex on its own.

Except that today, it was a paperweight.  In fact, you couldn't even use the operating system effectively.  Eventually task manager would pop up and show that something in the application was suffocating the CPU.  I was not looking forward to diagnosing this problem given the level of complexity involved.  I expected to spend the morning on it, but in all it took about 10 minutes to find and fix the problem.

I had worked with the application to know what pattern of messages to expect in the log files, but when I opened them I found some of the messages to be missing.  Specifically, the messages pertaining to just one of those three dozen communication channels I mentioned earlier.  After some initial testing, I had isolated the problem to a case where that channel was misconfigured.  I re-ran the application and grabbed the fresh logs.

Since I mirror my class structure in logger objects, isolating the relevant log entries was easy using logparser.  The list was surprisingly short, ending with this message:

initiating PSU polling thread 

I quickly found this message in my code:

void PollForReports()
{
    Log.Info( "initiating PSU polling thread" );
    while( 1 == Interlocked.Read( ref suspendInterlock ) )
    {
        continue;
    }
 
    RequestSerialNumber();
    Log.Debug( "requested PSU serial number ..." );
        
    // ...

As you can see, a "requested PSU serial number..." log message should have been written almost immediately after the last message I found in the log.  Something in the code after the "initiating PSU polling thread" message is written and before the "requested PSU serial number ..." message is locking the CPU. 

Do you see it - the improperly implemented spin lock?  It's pretty obvious when you know where to look, and logging can make it a lot easier to know where to look.



Delegates and Native API Callbacks - Answer

§ April 16, 2009 16:35 by beefarino |

A while back I posted a little puzzle about an exception I was hitting after passing a delegate to a native API call.  In a nutshell, my application was passing an anonymous delegate to an unmanaged library call:

Result Code result = ExternalLibraryAPI.Open(
    deviceHande,
    delegate( IntPtr handle, int deviceId )
    {
        // ...
    }
);
VerifyResult( result ); 

The call returns immediately, and the unmanaged library would eventually invoke the callback in response to a user pressing a button on a device.  After a semi-random period, pressing the button would yield an exception in my application.  The questions I asked were:

  1. What's the exception?
  2. How do you avoid it?

I've waited a bit to post the answers to see if anyone besides Zach would chime in.  Zach correctly identified the nut of the problem - the delegate is being garbage collected because there is no outstanding reference on the anonymous delegate once the unmanaged call returns.  With no one referencing the delegate object, the garbage collector is free to reclaim it.  When the device button is pushed, the native library invokes the callback, which no longer exists in memory.  So, to answer the first question, the specific exception that is raised is a CallbackOnCollectedDelegate:

CallbackOnCollectedDelegate was detected.
Message: A callback was made on a garbage collected delegate of type 'Device.Interop!Device.Interop.ButtonPressCallback::Invoke'. This may cause application crashes, corruption and data loss. When passing delegates to unmanaged code, they must be kept alive by the managed application until it is guaranteed that they will never be called.

The verbage in this exception message answers my second question.  To avoid the exception, you need to hold a reference to any delegate you pass to unmanaged code for as long as you expect the delegate to be invoked.  You need to use an intermediary reference on the delegate to maintain it's life.

Based on this, any of these examples are doomed to fail eventually, because none of them maintain a reference on the delegate object being passed to the unmanaged library:

ExternalLibraryAPI.Open(
    deviceHande,
    delegate( IntPtr handle, int deviceId )
    {
        // ...
    }
);
 
ExternalLibraryAPI.Open(
    deviceHande,
    new ButtonPressCallback( this.OnButtonPress )
);
 
ExternalLibraryAPI.Open(
    deviceHande,
    this.OnButtonPress
);

The correct way to avoid the problem is to hold an explicit reference to the specific delegate instance being passed to unmanaged code:

ButtonPressCallback buttonPressCallback = this.OnButtonPress;
ExternalLibraryAPI.Open(
    deviceHande,
    buttonPressCallback
);
// hold the reference until we're sure no
// further callbacks will be made on the
// delegate, then we can release the
// reference and allow it to be GC'ed
buttonPressCallback = null;

At first I thought Zach's pinning solution was correct; however, you can only pin blittable types, of which delegates are not, so "pinning a delegate" isn't even possible or necessary.  If you're interested, the details of how delegates are marshalled across the  managed/unmanaged boundary are quite interesting, as I found out from Chris Brumme's blog:

Along the same lines, managed Delegates can be marshaled to unmanaged code, where they are exposed as unmanaged function pointers.  Calls on those pointers will perform an unmanaged to managed transition; a change in calling convention; entry into the correct AppDomain; and any necessary argument marshaling.  Clearly the unmanaged function pointer must refer to a fixed address.  It would be a disaster if the GC were relocating that!  This leads many applications to create a pinning handle for the delegate.  This is completely unnecessary.  The unmanaged function pointer actually refers to a native code stub that we dynamically generate to perform the transition & marshaling.  This stub exists in fixed memory outside of the GC heap.

However, the application is responsible for somehow extending the lifetime of the delegate until no more calls will occur from unmanaged code.  The lifetime of the native code stub is directly related to the lifetime of the delegate.  Once the delegate is collected, subsequent calls via the unmanaged function pointer will crash or otherwise corrupt the process. 

Thanks again Zach, and to everyone who reads my blog!