Automation Framework pt 2: End-to-End Example

§ January 27, 2009 00:03 by beefarino |

Having covered the vision and napkin design of an automation framework for our product's core services, it's time for a working end-to-end example.  My goal is to be able to drive one function of our core product: creating a user account.  In addition, I will drive it from both PowerShell and FitNesse to see how well the framework meets the needs from the initial vision.

Getting to Red

I broke ground with this test:

[Test] 
public void CreateUserAccountCommandExecution()
{
    ICommand command = new CreateUserAccountCommand { Name = "joe" }; 
    bool result = command.Execute(); 
    Assert.IsTrue( result ); 
} 

Simple enough - a textbook command pattern; note:

  • an ICommand interface defines the command contract;
  • at the moment, the only member of ICommand is an Execute() method.  It accepts no arguments and returns a boolean to indicate success or failure;
  • CreatePlayerAccountCommand is a concrete implementation of the ICommand contract;
  • CreatePlayerAccountCommand has a Name property that identifies the user name.

Getting to Green

First thing's first - I need the command contract:

public interface ICommand
{ 
    bool Execute(); 
} 

Then I can implement the concrete CreatePlayerAccountCommand type:

public class CreateUserAccountCommand : ICommand 
{ 
    public string Name { get; set; }  
    public bool Execute() 
    { 
        IUserService clientInterface = new RemoteUserService( "http://beefarino:8089" );  
        Credentials credentials = new Credentials( "user-manager", "password" ); 
        Ticket authTicket = clientInterface.Authenticate( credentials );  
        UserData userProperties = new UserData();  
        userProperties.FirstName = Name; 
        userProperties.LastName = "Smyth"; 
        userProperties.Nickname = Name; 
        userProperties.DateOfBirth = System.DateTime.Now - TimeSpan.FromDays( 365.0d * 22.0d );
         
        string userId = clientInterface.CreateUser( authTicket, userProperties );  
        Ticket userTicket; 
        clientInterface.CreateUserTicket( authTicket, userId,  out userTicket );  
        return null != userTicket;
    } 
} 

I'm not going to discuss this code except to explain that:

  • the logic in the Execute() method performs the minimum amount of activity necessary to create a user account;
  • I'm making assumptions about a lot of the data I need (e.g., the age of the user).  I'm trying to keep the command as simple as unconfigurable as possible, and there are many, many more UserData fields available for account configuration that I'm not using;
  • the command object method does nothing outside of it's intended scope: it creates a user account, that's it.

Use it from PowerShell

Now that I have the command working, I want to see it working in PowerShell.  I'm taking a minimalist approach starting out.  Once I implement a few more commands and plug them into PowerShell, I'll see what implementation patterns emerge and replace this approach with something cleaner.  But for now, this mess will do:

[System.Reflection.Assembly]::LoadFrom( 'automation.commands.dll' ); 
function new-useraccount() 
{ 
    param( [string] name ); 
     
    $cmd = new-object automationcommands.createuseraccountcommand; 
    $cmd.Name = $name;
    $cmd.Execute(); 
}  
new-useraccount -name 'scott'; 

Hmmm ... runs silent, no output ... but looking at the system backend, I can see that it works.  

Use it from FitNesse

I downloaded the latest stable version of FitNesse from http://www.fitnesse.org/ and followed Cory Foy's short tutorial on using it against .NET assemblies (which is still accurate after 3+ years, #bonus) to get things running.  I created a new page and entered the following wikitext and table:

!contents -R2 -g -p -f -h 
!define COMMAND_PATTERN {%m %p} 
!define TEST_RUNNER {dotnet\FitServer.exe} 
!define PATH_SEPARATOR {;} 
!path dotnet\*.dll 
!path C:\dev\spikes\Automation\PokerRoom.Fixtures\bin\Debug\pokerroom.fixtures.dll  
A simple test of the CreateUser command: 
|!-PokerRoom.Fixtures.CreateUserAccount-!| 
|name|created?| 
|phil|true| 
|bob|true| 
|alice|true| 

I hacked up a quick fixture to support the table...

namespace PokerRoom.Fixtures 
{ 
    public class CreateUserAccount : fit.ColumnFixture 
    { 
        public string name { get; set; }  
        public bool created() 
        { 
            ICommand cmd = new CreateUserAccountCommand { Player = name };
    
            return cmd.Execute();
        } 
    } 
} 

... build it, and the FitNesse tests are green ...

After verifying that the users are actually created in the live system using our proprietary tools, I'm satisfied.

Moving Forward

So far so good.  It's very dirty, but it's working.  w00t * 2!

While developing this today I noted a few areas of concern:

  1. In the command object, there are several dependencies that obviously should to be injected.  Namely, the IUserService instance and the authority credentials;
  2. These dependencies are only really needed in the Execute() method;
  3. Looking ahead, I know I'm going to have many of these services, and it will be a pain to inject them all for each command instantiation;
  4. Compositing commands into complex behavior will eventually lead to the need to share state between commands.  I have an idea of how to manage this, but I'm concerned it will be cumbersome;
  5. There needs to be some kind of feedback when using the command from PowerShell; not sure where this should live or what it should look like at the moment...
  6. PowerShell will have a lot more to offer if I integrate with it more deeply.  I'll have to think about what this will look like, so as to minimize the amount of custom scripting necessary to run commands while accessing the full PowerShell feature set;
  7. I need to learn a lot more about FitNesse :).  I've already given the elevator speech to a coworker and demonstrated the fixture - he had a lot more questions than I had answers...

My next few posts will detail how I address these and other concerns.  Next post will detail some prefactoring to take care of items 1-4, maybe demonstrate command compositing.



Overlapped I/O Aborted by Terminating Thread

§ October 17, 2008 07:10 by beefarino |

A while back I posted about using Overlapped I/O from the .NET Framework.  I've started integrating the hardware with the rest of the project and hit a snag.  It seems that if a thread makes an overlapped I/O request and later terminates, the I/O request is aborted and your IOCompletionCallback routine receives error code 995 (system error code ERROR_OPERATION_ABORTED, or System.IO.IOException): "The I/O operation has been aborted because of either a thread exit or an application request".  I haven't looked into why this happens, but functionally it seems that the Windoze kernel assumes that the I/O request is valid only as long as the requesting thread is alive and kicking, which seems both perfectly reasonable and unreasonable depending on your perspective.  If you do happen to know the specifics on the kernel's behavior here, please comment; you'll save me some digging.

Example

Here is a unit test that illustrates the problem; a brief walkthrough follows the code:

[Test]
public void OIOTerminatesOnThreadExit()
{
    TcpListener listener = new TcpListener( 8888 );
    TcpClient client = new TcpClient();

    Exception exception = null;

    Thread listeningThread = new Thread(
        delegate()
        {
            listener.Start();

            // block until we receive a client
            TcpClient myClient = listener.AcceptTcpClient();

            // initiate an overlapped I/O operation on
            //  the underlying socket
            myClient.GetStream().BeginRead(
                new byte[ 16 ], 0, 15,                    
                r => {
                    try
                    {
                        // calling EndRead should
                        //    yield an exception            
                        myClient.GetStream().EndRead( r );
                    }
                    catch( Exception e )
                    {
                        // save the exception for later
                        //  assertion and validation
                        exception = e;
                    }
                },
                null
            );
        }
    );

    // start the listening thread
    listeningThread.Start();

    // connect to the TcpListener, so it can initiate an
    //  overlapped I/O operation
    client.Connect( Dns.GetHostName(), 8888 );

    // wait for the listening thread to finish
    listeningThread.Join();

    // verify
    Assert.IsNotNull( exception );
    Assert.IsInstanceOfType( typeof( IOException ), exception );
    StringAssert.Contains(
        "The I/O operation has been aborted because of either a thread exit or an application request",
        exception.Message
    );
}

Note that for brevity this test contains no error handling or Tcp timeouts, which it really should.

The test creates a thread that starts a TcpListener and waits for a connection.  Once a connection is established the thread issues an overlapped I/O read request on the network stream.  The AsyncCallback handler for the BeginRead operation just calls EndRead, saving any exception that occurs for further scrutiny.  Immediately following the overlapped I/O request, the listeningThread terminates normally.

Once the listeningThread is started, the unit test uses a TcpClient to connect to the TcpListener.  This will allow the listeningThread to make the overlapped I/O request and terminate.  After the TcpClient is connected, the test waits for the listeningThread to terminate.  

At this point, the test verifies that an exception was received from the BeginRead AsyncRequest callback and validates its type and content.

Workaround

My current workaround is pretty simple: kick off the I/O operation from the thread pool instead of an application thread.  Thread pool threads don't really terminate like application threads, they just go back into the pool when their work unit is complete, and the kernel seems to be content to oblige I/O requests from the thread pool even after the thread is returned to the pool (e.g., when you call an asynchronous BeginRead operation from your AsyncResult callback).

Here's the example with the workaround applied:

[Test]
public void OIODoesNotTerminateOnThreadPoolThreadExit()
{
    TcpListener listener = new TcpListener( 8888 );
    TcpClient client = new TcpClient();

    Exception exception = null;

    // start the listeningThread on the thread pool
    ThreadPool.QueueUserWorkItem(
        delegate( object unused )
        {
            listener.Start();

            // block until we receive a client
            TcpClient myClient = listener.AcceptTcpClient();

            // initiate an overlapped I/O operation on
            //  the underlying socket
            myClient.GetStream().BeginRead(
                new byte[ 16 ], 0, 15,                    
                r => {
                    try
                    {
                        myClient.GetStream().EndRead( r );
                    }
                    catch( Exception e )
                    {
                        // save the exception for later
                        //  assertion and validation
                        exception = e;
                    }
                },
                null
            );
        }
    );

    // connect to the TcpListener, so it can initiate an
    //  overlapped I/O operation
    client.Connect( Dns.GetHostName(), 8888 );

    // verify
    Assert.IsNull( exception );            
}       

The only changes here are that the listening thread is queued on the thread pool, and the test verifies that the exception remains null.  Using the thread pool seems to counter the kernel behavior and keep the outstanding overlapped I/O requests active even after the thread work unit is complete.



Coping with the Fear of Changing Code

§ April 20, 2008 14:43 by beefarino |

Another fear I'm seeing on the team is a fear of changing existing code.  The fear may stem from several sources: the code implements complex behavior that is undocumented; the code is inherited from a resident expert who is retasked or otherwise unavailable; the code hasn't been maintained properly and reads like a plate of spaghetti.  Whatever the source, a team member's response to the fear follows the pattern:

When team members are afraid, they will act in either their own interest of self-preservation, or in the interest of team survival.

When confronted with such code, a team member can choose one of two paths: they will choose to do as little with the code as possible, leave it alone as much as they can and still satisfy everyone's expectations; or they own up to the situation and remove the source of the fear, making the code easier for anyone to cope with and understand.

Those who pursue self-preservation take the former track.  They exert a lot of effort to develop an understanding of the code, but don't do anything to persist or share that knowledge.  Chances are they are trying to act in the team's interest by developing specialized knowledge of the code, but in the long-term they benefit only themselves.  The code remains an untenable briar patch for the next poor sod who receives it.

Those who pursue the best interests of the team take the latter approach.  They write unit tests around the existing code; they refactor the code and leverage the common design patterns to make the code comprehensible; they seek clarification from the business heads and write documentation targeted to other developers.  Their actions are targeted, whether implicitly or explicitly, at communicating with the other members of their team.