Log4Net Hack: Customizing Log File Paths After Configuration

§ November 28, 2009 04:48 by beefarino |

I get a lot of log4net questions through my blog because of the tutorials I've written up.  One item that comes up frequently is how to configure a FileAppender to be able to write a single log file from multiple instances of the same application.  The truth is, you can't do it.  Or more precisely, there is no way to do it and expect not to lose log events and have your application performance suffer greatly.

First let me show you how to allow multiple applications access to a single log file.  It's actually quite easy:

 <?xml version="1.0" encoding="utf-8" ?>
 <configuration>
  <configSections>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>  
  </configSections>
  <log4net>
    <appender name="FileAppender" type="log4net.Appender.FileAppender">
      <file value="log-file.txt" />
      <appendToFile value="true" />
      <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
      <layout type="log4net.Layout.SimpleLayout" />         
    </appender>
    <root>
      <level value="DEBUG" />
      <appender-ref ref="FileAppender" />      
    </root>    
  </log4net>
  
</configuration> 

Since I discuss the parameters and pitfalls of the FileAppender elsewhere, I will leave it to you to read up on them more if you want to.  The locking mode being used here is causing log4net to acquire the file handle before each log event, then close the handle after the log event is written.  Doing this allows other applications to get write access to the file when no one else is currently logging, but the technique has a few serious flaws that should prevent you from using it:

  1. All of that file opening and closing seriously hampers performance;
  2. The log file will be shared, but access conflicts will still occur between applications attempting to log events at the same time, resulting in more performance degredation and "dropped" log events.

You may be able to address some of the performance issues using a BufferingForwardingAppender that sends large chunks of events to the minimally-locking FileAppender; however this will not resolve the contention over the single log file that is at the root of the issue.

The simplest solution to this problem is to use a different type of appender that is designed for concurrent use.  For instance, the EventLogAppender or AdoNetAppender use technologies that will manage concurrency issues for you.  If you're dead set on using the filesystem, the next simplest solution is to have each application log to a unique file, thus removing any log file contention at runtime.  The separate log files can be collated once the run is over using a tool like LogParser.  The drawback to this approach is that you have to hack it in: there is no direct way to modify the filename in the FileAppender based on the runtime environment.

That said, it's not hard.  Check out this simple console application:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using log4net;
using log4net.Appender;
namespace log4netPostConfig
{
    class Program
    {       
        static void Main( string[] args )
        {
            log4net.Config.XmlConfigurator.Configure();
            var filePostfix = "_" + Guid.NewGuid().ToString( "N" );
            
            var fileAppenders = from appender in log4net.LogManager.GetRepository().GetAppenders()
                                where appender is FileAppender
                                select appender;
            fileAppenders.Cast<FileAppender>()
                .ToList()
                .ForEach(
                    fa =>
                    {
                        fa.File += filePostfix;
                        fa.ActivateOptions();
                    }
                );
            ILog Log = log4net.LogManager.GetLogger( System.Reflection.MethodBase.GetCurrentMethod().DeclaringType );
            Log.InfoFormat( "this process is using log file postfix [{0}]", filePostfix );
        }
    }
} 

This example loads the logging configuration from the app.config (line 14).  The log4net configuration is searched for instances of FileAppenders (line 16), which have their filename parameters handrolled with some process-specific information (line 25) - a GUID in this case, the current process identifier may be another good choice.  Calling the ActivateOptions on each modified appender is vital (line 26), as it recreates each file handle using the new filename configuration parameter set in the code.

The app.config for this example is just a plain vanilla logging configuration:

<?xml version="1.0" encoding="utf-8" ?>
 <configuration>
  <configSections>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>  
  </configSections>
  <log4net>
    <appender name="FileAppender" type="log4net.Appender.FileAppender">
      <file value="log-file.txt" />
      <appendToFile value="true" />
      <encoding value="utf-8" />
      <layout type="log4net.Layout.SimpleLayout" />         
    </appender>
    <root>
      <level value="DEBUG" />
      <appender-ref ref="FileAppender" />      
    </root>    
  </log4net>
  
</configuration>

Note that the log-file.txt specified in the app.config will be created when the XML logging configuration is loaded (line 13 in my code example above), but it will never be written to.

Edit Notes

I just noticed after publishing this that a very similar example was written almost 2 months ago by Wil Peck.



The Most Moronic Value Converter Ever

§ August 26, 2009 06:43 by beefarino |

I'm really digging that I get to work with WPF at the moment.  Using advanced databinding features against MVVM are making quick work of the menial "datapult" applications I'm pounding out.  I still have a lot to absorb, but I'm getting there.

Being a n00b to XAML and WPF, I often find myself lost in a fog when things don't work.  E.g., this afternoon I spent a good 30 minutes trying to decipher a binding problem.  I was trying to use the data context of a TreeViewItem in the text of it's header template (you can laugh at my XAML, this is just me hacking):

<TreeViewItem ItemsSource="{Binding Profiles}">
    <TreeViewItem.HeaderTemplate>
        <DataTemplate>
              <TextBlock Text="{Binding Count}"/>
        </DataTemplate>
    </TreeViewItem.HeaderTemplate>
</TreeViewItem>

I tried several binding paths in the TextBlock, but nothing would show up in the tree node.  I ended up switching gears and trying to figure out what data context was actually available to the header data template.  Finding nothing from a cursory googling, I ended up creating a value converter that would convert the data object into it's underlying type:

namespace My.Converters
{
    public class ObjectToTypeConverter : IValueConverter
    {
        public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
        {
            if( null == value )
            {
                return "null";
            }
            return value.GetType();
        }
        public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
        {
            throw new NotImplementedException();
        }
    }
}  

The Convert implementation will return the System.Type of the data context (line 12), or the string "null" if no context is available (line 9).  At this point I can alter the XAML to simply spit out the actual type of the data context, giving me a clue as to why I'm clueless:

<Window ...
xmlns:Converters="clr-namespace:My.Converters">
...
<Window.Resources>
<Converters:ObjectToTypeConverter x:Key="ObjectToTypeConverter" />
</Window.Resources>
<TreeViewItem ItemsSource="{Binding Profiles}">
    <TreeViewItem.HeaderTemplate>
        <DataTemplate>
            <TextBlock Text="{Binding Converter={StaticResource ObjectToTypeConverter}}"/>
        </DataTemplate>
    </TreeViewItem.HeaderTemplate>
</TreeViewItem>
... 

Anyway, long story short, in my particular case the data context for the TreeViewItem header template is actually null - which makes no sense to me, I would have assumed it would have been the TreeViewItem's data context.  A quick relative sourcing of the binding solved my issue:

<TreeViewItem ItemsSource="{Binding Profiles}">
    <TreeViewItem.HeaderTemplate>
        <DataTemplate>
            <TextBlock Text="{Binding Path=Profiles.Count}" 
              DataContext="{Binding DataContext,RelativeSource={RelativeSource Mode=FindAncestor,AncestorType={x:Type TreeViewItem}}}"/>
        </DataTemplate>
    </TreeViewItem.HeaderTemplate>
</TreeViewItem> 

 



ConvertFrom-PDF PowerShell Cmdlet

§ July 8, 2009 06:14 by beefarino |

I hate PDFs. 

And now I need to search through several hundred of them, ranging from 30 to 300 pages in length, for cross-references and personnel names which ... um ... well, let's just say they no longer apply.  Sure reader has the search feature built-in, so does explorer, but that's so 1980's.  And I sure don't want to do each one manually...

I poked around the 'net for a few minutes to find a way to read PDFs in powershell, but no donut.  So I rolled my own cmdlet around the iTextSharp library and Zollor's PDF to Text converter project.

There isn't much to the cmdlet code, given that all of the hard work of extracting the PDF text is done in the PDFParser class of the converter project:

using System;
using System.IO;
using System.Management.Automation;
namespace PowerShell.PDF
{
    [Cmdlet( VerbsData.ConvertFrom, "PDF" )]
    public class ConvertFromPDF : Cmdlet
    {
        [Parameter( ValueFromPipeline = true, Mandatory = true )]
        public string PDFFile { get; set; }
        
        protected override void ProcessRecord()
        {
            var parser = new PDFParser();
            using( Stream s = new MemoryStream() )
            {
                if( ! parser.ExtractText(File.OpenRead(PDFFile), s) )
                {
                    WriteError( 
                        new ErrorRecord(
                            new ApplicationException(),
                            "failed to extract text from pdf",
                            ErrorCategory.ReadError,
                            PDFFile
                        )    
                    );
                    return;
                }
                s.Position = 0;
                using( StreamReader reader = new StreamReader( s ) )
                {
                    WriteObject( reader.ReadToEnd() );
                }
            }
        }
    }
}

The code accepts a file path as input; it runs the conversion on the PDF data and writes the text content of the file to the pipeline.  Not pretty, but done.

Usage

Here is the simple case of transforming a single file:

> convertfrom-pdf -pdf my.pdf

or

> my.pdf | convertfrom-pdf 

More complex processing can be accomplished using PowerShell's built-in features; e.g., to convert an entire directory of PDFs to text files:

> dir *.pdf | %{ $_ | convertfrom-pdf | out-file "$_.txt" } 

More relevant to my current situation would be something along these lines:

> dir *.pdf | ?{ ( $_ | convertfrom-pdf ) -match "ex-employee name" } 

Download the source: PowerShell.PDF.zip (1.10 mb) 

Enjoy!




Expanding-File Resource for Spring.NET

§ May 13, 2009 03:48 by beefarino |

I just had a fun emergency - the crew decided to redirect all configuration sources using environment variables, and while Log4Net supports this quite easily, the spring.net context resource-from-a-URI abstraction does not.  E.g., this will not work:

<spring>
    <context>
            <resource uri="file://%PROGRAMFILES%/myapp/ioc.config" />
    </context>
    ...

Knowing that spring.net can leverage custom resource implementations, I set out to extend the file system resource to expand environment variables.  Turns out to be easy-peasy-lemon-squeezie:

public class ExpandableFileSystemResource : Spring.Core.IO.FileSystemResource
{
    public ExpandableFileSystemResource()
        : base()
    {
    }
    public ExpandableFileSystemResource( string resourceName )
        : base( Environment.ExpandEnvironmentVariables( resourceName ?? String.Empty ) )
    {
    }
    public ExpandableFileSystemResource( string resourceName, bool suppressInitialize )
        : base( Environment.ExpandEnvironmentVariables( resourceName ?? String.Empty ), suppressInitialize )
    {
    }
}

Register the resource implementation via config:

<spring>
    <resourceHandlers>
      <handler protocol="filex" type="MyApp.ExpandableFileSystemResource, MyApp"/>
    </resourceHandlers>
    <context>
            <resource uri="filex://%PROGRAMFILES%/myapp/ioc.config" />
    </context>    
    ... 

 And it just works.  Enjoy!