a trace of thought on...BizTalk Server, Team Foundation Server, AppFabric, etc. RSS 2.0
 Wednesday, May 11, 2005

There have been several folks who have discussed how to create messages "from scratch" within an Orchestration context - you can read Matt's thoughts and check out a BizTalk documentation excerpt. This is a common question, and I had an additional technique I thought I would share... 

Background: When you have a schema that has many promoted properties (or distinguished fields), or many elements that can be set via xpath expressions easily, it can be useful to simply start with a "template" document instance and populate the element content that you are interested in. 

In this situation, you will often have a "Message Assignment" shape that looks something like this:

xmlDoc.LoadXml("<ns0:BizTalkSampleS3 
      xmlns:ns0="http://BizTalkSample.Schemas.BizTalkSampleS3">
      <SomeElement></SomeElement></ns0:BizTalkSampleS3>");
someMsg = xmlDoc;
someMsg.SomeElement = "some content";
// (or xpath(someMsg,someElementXPath) = "some content" if we don't have a 
// distinguished field.)
   

One disadvantage of loading up "template" xml documents from either expression shapes or code (via XmlDocument.LoadXml) is that those xml fragments can get easily "lost", and are hard to update early in the development cycle when schemas may still be in flux.  Loading the template files from the file system is problematic because the question arises "where should I store these files, so that I can find them in any environment I deploy to?" (Solvable, but a hassle.)

Instead, why not embed the template xml documents as assembly resources?  For those unfamiliar with that process, I have a short tutorial here (& a helper class.) 

  1. You will need a C# project as part of your overall BizTalk solution.  Place your template xml file(s) in the directory corresponding to this project, and add them as an "existing item" to the project. 
  2. Select this file within the Solution Explorer, and within the Properties window, select "Embedded Resource" as the "Build Action" as shown here:

     

  3. Place this class (text here) within the same C# project that houses the resources you have added.
  4. To construct a message, drag out a "Message Assignment" shape, and within the associated expression write some code like the following.  Simply pass the file name of the template document as an argument to GetXmlDocResource (or GetStringResource.)
    sampleUsingTemplate = 
       BizTalkSample.Components.EmbeddedResourceCache.
          GetXmlDocResource("BizTalkSampleS3_output.xml");
       // Populate the "rest" of the message with distinguished fields, promoted 
       // properties, xpath expressions, etc.
       sampleUsingTemplate.SomeElement = "foo";
                

The class I have supplied will cache the loaded resources in a hashtable for performance sake, and allow you to load resources as both strings and XmlDocuments.

A last thought: Many people ask, "Why can't I just create a message using a new operator or a default constructor of some sort?" Well, because few XSD schemas sufficiently constrain the set of valid instance documents enough for that to be useful - what form would a "default message" take? (Would it have the optional elements you need? Some elements that you don't want?)

Enjoy - feedback appreciated!

Wednesday, May 11, 2005 3:29:14 PM (Central Standard Time, UTC-06:00)  #    Comments [6] -
BizTalk Tools | BizTalk Insights
 Monday, May 9, 2005

Just a few notes on what I'll be up to in the next month or two...

Do you use the Deployment Framework for BizTalk?  Or just have general questions on automating BizTalk deployments?  You can attend my session at TechEd 2005 - Friday June 10th, 10:45am to 12:00pm in the BPI Cabana, where I'll be doing a talk on those topics...

I'll also be speaking at the Microsoft DevCon event on my "other favorite" topic, Windows Mobile.  The focus will be on what is new for Visual Studio 2005 in the mobile arena...The conference will also be covering the new Team System offerings, ASP.NET 2.0, and quite a few other VS2005 topics.  Check it out...

Monday, May 9, 2005 8:10:41 AM (Central Standard Time, UTC-06:00)  #    Comments [3] -
Deployment Framework | General
 Friday, May 6, 2005

(Update: See the latest on the Deployment Framework here.)

Well, its time for another revision of the BizTalk/NAnt deployment tools... 

For folks recently joining us, these tools allow you to quickly create both developer workstation deployments (inside Visual Studio) and MSI-based deployments (for servers) for BizTalk 2004 solutions.  Just about all aspects of BizTalk deployments are handled, including items such as virtual directories, SOAP proxies, and WinXP/2k/2003 differences.

I get quite a few emails from people who are using these tools (which is great!)  A couple folks have suggested that I start referring to the whole thing as a" p framework?. < deployment>

That sounds about right - like all frameworks, there is a core set of functionality that takes care of quite a bit of work, and a given project can supply what needs customization or what is missing. 

In this case, the core NAnt scripts, custom NAnt tasks, WiX generation, and other custom tools form the framework itself - and a given BizTalk project can supply project-specific details, what needs to be customized, and any missing functionality.   So I'll refer to this work as the "Deployment Framework for BizTalk" from now on... 

What has changed in this rev? 

  • When last posting on this topic, I should have been even clearer that as of the last release, the deployment framework requires NAnt .85.
  • The biggest improvement is the ability to use binding files as they are emitted by the BizTalk Deployment Wizard directly.  In the past, the framework had insisted that binding files be split between orchestration bindings and messaging bindings - this is no longer required.  I had several requests for this fix...Also, note that having multiple binding files is supported (like one for each of several assemblies.)  Now, if you wish to continue to use the "old style" split bindings, define the orchBindings property in your project NAnt file as shown below:
  • <property name="orchBindings" value="${nant.project.name}.OrchBindings.xml" overwrite="false" />

  • You can now set the value of NAnt properties from the SettingsFileGenerator spreadsheet.  Recall that this spreadsheet is used (in conjunction with XmlPreProcess) to manage environment-relative configuration - that is, to manage those values that vary depending on whether you are in a development, QA, or production environment.  Until now, it has been difficult to use those configuration values within the NAnt deployment file!  You typically just used them in your binding files or other application-level XML files.  Now, however, you can define a property as shown below to make these values available as NAnt properties.  A good example usage is the case where you want the actual names of the SSO Application Users group and SSO Administrators group to be defined in the SettingsFileGenerator spreadsheet (because the group names may vary by environment you deploy to.)  Just define a property in your project NAnt file called "propsFromEnvSettings" with a comma-separated list of values you want pulled in.  (The sample has been updated to show this, too.)
  • <property name="propsFromEnvSettings" value="ssoAppUserGroup,ssoAppAdminGroup"/>

  • Many people had asked for the DeployResults.txt file (which is created in the DeployResults subdirectory during a deployment) to be available in a history log, so that it would be easy to walk up to a production server and determine what deploy/undeploy activity had taken place (especially useful for diagnosing any failures.)  To that end, the deployment framework will preserve past DeployResult.txt files with this convention: `DeployResults TRACESATURN_05032005_0922.txt'  Note that the machine name, date, and time are preserved - so you can gather these files across all the servers in a BizTalk group, if need be.
  • SSOSettingsFileReader class improvements: In the last release, I introduced the ability to manage settings within the BizTalk SSO using the SettingsFileGenerator spreadsheet that is also used for managing environment-relative configuration.  Part of that functionality included a class called SSOSettingsFileReader, used to retrieve settings at runtime.  This class now has some strongly typed accessors, and no longer sits inside a lengthy namespace, which means that in an orchestration expression shape you can simply do:
  • someVar = SSOSettingsFileReader.ReadString("BizTalkSample","SomeAppConfigItem") // or ReadInt32

  • Ben Cops had found that on large-scale deployments, it was useful to restrict which orchestrations were deployed/started to speed the development cycle.  So, he modified the orchestration NAnt task accordingly.  To use (if you need to), just set the orchestrationsToIgnore property in your project NAnt file with a comma-separated list that reflects what you are currently working on.  Make sure to copy BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory again! (you'll find it in the DeployTools directory in the download.)
  • A bug was fixed with undeployment for the case where an SSO affiliate app had already been deleted...
  • All references in NAnt files to xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" were removed, because they were simply causing heartache (for many reasons.)  Remove this namespace in your project NAnt file when upgrading.
  • Rather than trying to keep the source for the C# tools (NAnt tasks, SetEnvUI, DeployBTRules, SSOSettingsFileImport\SSOSettingsFileReader, etc.) current within the GotDotNet workspace, I'm making them available as a zip

Remember, you do not need to piece together the current functionality set from past blog entries - the complete documentation (in Word form) is in the download and separately here.

Some last thoughts:

  • Some folks have been reluctant to use the deployment framework because of its reliance on NAnt, and a worry over using NAnt in a server environment.  One thing to remember (that might allay this concern) is that a subset of NAnt is delivered with the generated MSI, and NAnt will not require a separate installation procedure by your production operations team.  Moreover, NAnt doesn't create any kind of persistent service or footprint on the production machine (i.e. no automatic file associations that would cause scripts to run with a double-click or anything like that...)
  • Add a Visual Studio External Tool for the updateOrchestration target if you haven't already!  This can speed your development cycle significantly.  See the tools zip for a script that creates all relevant external tools.
  • When using the WiX-based MSI generation, make sure your project NAnt file (for WiX generation, like BizTalkSample.WiXSetup.Build) is choosing "debug" or "release" based on what you want to put in the MSI.  See comments in that file for more detail, and Jon Flanders' post on this topic.

How do I upgrade?

  1. With a safe copy of your BizTalk solution, expand the "Core" zip contents right on your project directory, such that the core scripts will overwrite what you have now (but your project-specific scripts will be preserved.)
  2. Manually copy the WiX-related updates to your *.WiXSetup directory, since they will get unzipped to BizTalkSample.WiXSetup
  3. Remove xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" from your project NAnt file
  4. Copy DeplyTools\BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory
  5. Do a deployment on a developer workstation and a server to test things out...

Downloading

  • You can always use the download links on this blog.
  • Full sample here, core scripts here, extra tools here
  • (But always check the download links to see if there has been an update…)
  • Standalone documentation here (and diagram here.)
  • GotDotNet workspace here
    Friday, May 6, 2005 1:13:00 PM (Central Standard Time, UTC-06:00)  #    Comments [5] -
    Deployment Framework

    (Update: See the latest on the Deployment Framework here.)

    In the latest release of the Deployment Framework for BizTalk “extra tools”, you will find a script that will automate the creation of “external tools” in Visual Studio that are useful for working with BizTalk 2004 and the Deployment Framework.  Modify the script to include all the tools that your team relies on, too.

    The idea that this should be automated came from Ben Cops (though I’ve heard the complaint before!)  Manually creating all the useful external tools was somewhat painful for a large team of BizTalk developers.

    After you run the script, your Tools menu in Visual Studio should include entries for deploying and un-deploying the current solution, as well as for updating just the orchestrations, .NET components, and SSO configuration data (one menu entry.)  In addition, there will be entries for running NAnt on a highlighted target name (useful if you have your build script open), and for running HAT and the Subscription Viewer.  Enjoy…

     

    Friday, May 6, 2005 12:32:03 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    Deployment Framework
     Thursday, March 24, 2005

    A short while ago, I did a presentation for the Twin Cities .NET user group on log4net.  (You might recall an earlier blog entry where I discussed using log4net with BizTalk 2004...) 

    The presentation was not specific to BizTalk 2004 - instead, it attempts to describe why I think the log4net library is so very well thought out.  You can grab it here, if you like.

    The last slide references Loren Halvorsen's comparison of log4net and the new Microsoft Enterprise Library Logging Block, which I would recommend taking a look at (the comparison, that is.)  Tom Hollander (among many others!) later weighed in with this piece.

    Lastly, 1.2.9 beta of log4net has recently become available (release notes here)....However, my extensions to log4net have not yet been updated to reflect this new drop.  I'll be sure to post when I'm able to update.

    Thursday, March 24, 2005 4:07:03 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    General
     Monday, March 21, 2005

    (Update: See the latest on the Deployment Framework here.)

    Just when you thought automated deployments for BizTalk 2004 were safe, another update to the “Deploy with NAnt” template comes along… :) 

    This is a very worthwhile "upgrade" to consider taking advantage of.  One of the big items is support for "configuration info within SSO", but a complete list of features/fixes, etc. is below. 

    SSO Integration: If you are currently using the SettingsFileGenerator.xls spreadsheet (discussed in previous entries) in conjunction with the XmlPreProcess tool to handle "environment-relative" configuration - that is, to handle variations between your dev/QA/production environments - you will be glad to know you can get more leverage out of that process now…

    To review, with this process, you use the names that are defined in the first column of the SettingsFileGenerator spreadsheet (shown below) as "tokens" within your binding file, as in: 

    <!-- ifdef ${ xml preprocess} -->
    <!-- <Address>${FileSendLocation}\%MessageID%.xml</Address> -->
    <!-- else -->
    <Address>C:\temp\BizTalkSample OutDir\%MessageID%.xml</Address>
    <!-- endif -->

    At install time, the value for FileSendLocation that is appropriate to the environment you are deploying to is "plugged in" by XmlPreProcess (but for developers, the file remains legal XML.)  The spreadsheet might look as follows:

    Image0031
    (click)

    Now, however, you can also put in name-value pairs (whether they vary by environment or not) into the spreadsheet that you need access to at run time (i.e. general configuration information.

    The name-value pairs will be deployed into a newly created affiliate application within the SSO, and can be read using the SSOSettingsFileReader class (which serves as a cache, too – and is provided in the download.)  The static methods on this class can be used from an orchestration expression shape, or from assemblies that orchestrations call – as in string test = SSOSettingsFileReader.ReadValue("BizTalkSample","SomeAppConfigItem");  (By the way, the “nested names” shown in the spreadsheet above are just meant to suggest a partitioning mechanism you might want to use, rather than one flat “namespace” of config data.)

    Of course, the deployment scripts handle all the interaction with the SSO database for deployments/un-deployments.  This whole mechanism (SettingsFileGenerator.xls, XmlPreProcess, and now SSOSettingsFileImport.exe) is worth using even if you don’t use the rest of this script infrastructure.  Note that Jon Flanders had a great article on SSO-based configuration using a strongly-typed approach, where as this approach is loosely-typed, but leverages a mechanism you might already be using (i.e. the spreadsheet concept.)  (Note: Jon’s code for creating/deleting affiliate apps helped me complete the SSOSettingsFileImport.exe utility quickly…) 

    Other features/fixes: 

    • Support for multiple schema assemblies, including the case where schema assemblies reference each other due to schema imports.  (Undeployments occur in reverse order of deployments.)
    • Support for multiple orchestration assemblies, including the case where orchestration assemblies reference each other due to the use of Call/Start Orchestration shapes.  (Undeployments occur in reverse order of deployments.) 
    • Support for multiple orchestration and port binding files (just list them as comma separated values in the orchBindings/portBindings properties.) 
    • Fixed bug that occurred when undeploying ISAPI extensions in Windows 2003 / IIS 6. 
    • Fixed bug in undeploying pipeline components (thanks to John Adams) 
    • Added /NOFORCE flag to IISRESET calls in accordance with http://support.microsoft.com/kb/286196 
    • New property "includeInstallUtilForComponents" was added, which can be set to 'true' in your project-specific nant file.  This will cause your component assemblies to be called with "installutil.exe" (with the /u flag for undeployment.)  This is useful if you have .NET Installer-derived classes that need to get called for creating event sources, perf counters, etc.  (Yes, you could certainly argue these should be called by the WiX piece of the script infrastructure instead, but this method works and is more in keeping with the spirit of the entire process.)  Note there is no harm if a given component assembly listed in your components property does not have an installer class. 
    • Eliminated the need for the includeCustomTarget property, and provided additional ways to supply custom functionality.  Project-specific nant file can now supply any or all of customDeployTarget, customPostDeployTarget, customUndeployTarget, customPostUndeployTarget -- and they will be called if they are present. 
    • Integrated the deployment of log4net-specific stuff, if the includelog4net property is set to true.  See the work on BizTalk/log4net integration for more information.  (Log4net sample usage is included in the new sample.)   
    • Now using NAnt version .85 - so be sure to grab the latest NAnt/NAntContrib since the nant scripts are using .85-specific syntax to avoid "deprecated" warnings that were occurring otherwise.  Note that even once you have installed NAnt .85, you will still be able to deploy BizTalk projects that are using old versions of the deployment scripts.  Note also that this is a pretty basic port to .85 - there are no doubt more elegant ways of doing lots of things that I haven't investigated yet, now that nant has expression support. 

    Other notes 

    The scripts like to see BizTalk projects using Debug and Release for target names.  A long while back I had suggested a file-level search/replace within btproj files to change these, but two more elegant options are available: 

    • In project properties, change the output location for the “Development” configurations to “bin\debug” and output location of “Deployment” configurations to “bin\release”.
    • (Via Puneet Gupta): Change the template for BizTalk projects, found in BTSApp.btproj file in %BizTalkInstallDir%\Developer Tools\BizTalkWizards\BTSApp\Templates\1033, to reflect Debug and Release targets. 

    Another note: Lots of folks have noticed that XmlPreProcess can be used for a lot more than port binding files!  Log4net config files, orchestration binding files, you name it.  See the full Word documentation in the download for an example of how to do this within customDeployTarget.

    How do I upgrade?

    1. With a safe copy of your BizTalk solution, expand the “Core” zip contents right on your project directory, such that the core scripts will overwrite what you have now (but your project-specific scripts will be preserved.)
    2. Manually copy the WiX-related updates to your *.WiXSetup directory, since they will get unzipped to BizTalkSample.WiXSetup
    3. Do a deployment on a developer workstation and a server to test things out…

    Downloading

    • You can always use the right-hand links on this blog.
    • Full sample here, core scripts here
    • GotDotNet workspace here.  Because of issues with GotDotNet, I’m only going to be keeping the EXE utility source code up to date – not the entire sample & core tree.

     Enjoy – comments always welcome and appreciated.

    Monday, March 21, 2005 2:31:28 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    Deployment Framework
     Monday, March 14, 2005

    Steve and I wound up working on the same problem at the same time (probably for the same person…)

    When working with the MSMQ adapter, keep in mind that you must reference the Microsoft.BizTalk.Adapter.MSMQ.MsmqAdapterProperties.dll to have access to MSMQ-specific properties.  (The list of properties available is in the adapter documentation.)

    Why? The intellisense in the expression shape (when using the parentheses syntax on messages, ports, etc.) is looking for classes derived from Microsoft.XLANGs.BaseTypes.PropertyBase to present in the drop down list.  Some of those classes are part of your "native" BizTalk installation, some are provided by add-on adapters, and (of course) some are provided by property schemas that you develop.  A reference to the containing assembly is necessary to find them.

    Other things to note relative to the MSMQ adapter:

    • For dynamic send ports, the syntax can look like: SomePort(Microsoft.XLANGs.BaseTypes.Address) = @"MSMQ://FORMATNAME:DIRECT=OS:SOMEMACHINE\PRIVATE$\SOMEQ";
    • In the case of dynamic send ports, the runtime will use non-transactional sends by default.  If you want transactional sends, you will need to set the transactional property: outboundMsg(MSMQ.Transactional) = true;
    • Transactional messages which fail to deliver to the remote queue will be found in the local Transactional Dead Letter queue.  No error will be raised by the adapter.  This may mean that your design requires acknowledgement messages to achieve what you are looking for.
    Monday, March 14, 2005 10:20:25 AM (Central Standard Time, UTC-06:00)  #    Comments [2] -
    BizTalk Insights
     Thursday, February 24, 2005

    I was just reading Mike Holdorf's post on the bts_CleanupMsgbox stored procedure.  The team I work with has been able to make good use of this sproc as well, especially while stress testing.

    One thing to beware of is this: Your habit for doing this kind of thing might lead you to open up Sql Query Analyzer, and right click on the stored procedure - selecting “script to new window as execute.”  This will generate the following:

    DECLARE @RC int
    DECLARE @fLeaveActSubs int
    -- Set parameter values
    EXEC @RC = [BizTalkMsgBoxDb].[dbo].[bts_CleanupMsgbox] @fLeaveActSubs

    Ahhh, but the stored procedure actually defaults fLeaveActSubs to “1”, whereas this code will leave fLeaveActSubs as “0”.  This will cause all your subscriptions to be deleted, and you will have to redeploy your BizTalk applications.  Not that I would know....

    So just do “exec bts_CleanupMsgbox”...

    Thursday, February 24, 2005 1:32:57 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    BizTalk Insights
     Wednesday, January 19, 2005

    (Note - this work has been updated for log4net 1.2.9. See the 'Tools' download in the Deployment Framework)

    Early on when working with BizTalk 2004, it might be tempting to view the ability to track orchestration events (and the use of the orchestration debugger) as a substitute for traditional diagnostic logging.;

    After all, the orchestration debugger can tell you exactly how far you got in an orchestration, and what path through the workflow you took to get there

    However, consider these limitations:

    • The orchestration debugger is used either a) after the orchestration has completed or b) in conjunction with a breakpoint.  Neither of these choices is ideal, and only the latter gives you visibility into the intermediate values of variables/messages.
    • Once an orchestration involves looping constructs, multiple orchestrations connected via messaging, etc. the orchestration debugger often loses quite a bit of diagnostic value.

    Given this, it can be useful to integrate diagnostic logging into your orchestrations and the components they call.  If we are going to invest in this effort, we’d like to use something richer than the built-in System.Diagnostics.Trace/Debug infrastructure.

    Log4net is an Apache-sponsored initiative within the “Apache Logging Services” project.  It provides a rich diagnostic infrastructure for .NET, with support for hierarchical logging and configuration, multiple logging targets, and support for logging context.

    A recent article outlined the advantages of log4net over Microsoft’s Enterprise Instrumentation Framework (EIF), though Microsoft/Avanade will be revamping this logging infrastructure with the release of the Enterprise Template Library

    Given that the log4net initiative has broad support at the API layer (i.e. log4j, log4PLSQL, etc.) and platform layer (including .NET CF) as well as the benefit of maturity, log4net seems like a very reasonable choice for instrumenting BizTalk 2004 applications.  (The Enterprise Template Library logging component will likely be a good choice as well when released.)

    Following is a discussion of log4net itself, along with what is required to use it in a BizTalk 2004 setting.

    Quick Insight into log4net 

    Without diving immediately into the log4net API, let’s get a sense for what it can do.  What do we mean by hierarchical logging and configuration with multiple target support?

    Suppose we have classes with these namespace-qualified names:

    MyCompany.MyDepartment.MyClassA
    MyCompany.MyDepartment.MyClassB

    Each of these classes can obtain a logger specific to their class by declaring a static member as such:

    private static readonly ILog log = LogManager.GetLogger(typeof(Foo));

    Each class (MyClassA/MyClassB) can now emit trace with statements such as log.Info(“hello world”)> or log.Warn(“a potential problem…”).  Default log levels include: Debug, Info, Warn, Error, and Fatal (though these can be expanded.) 

    The concept of “context” can be added to a stream of trace messages via NDC.Push(“some context”)/NDC.Pop or MDC.Set(“somekey”,”somevalue”).  (These class names are short for “Nested Diagnostic Context” and “Mapped Diagnostic Context” respectively.)  These can be extremely useful for multithreaded/multi-user applications, to distinguish the streams of related trace messages from each other.  This context can be optionally formatted into the associated tracing statements, as will be shown shortly.

    If we want to turn on tracing at the ‘Error’ level by default, at the ‘Warn’ level for MyCompany, and at the ‘Debug’ level for MyClassA, we would simply configure as follows (notice the use of the namespace hierarchy):

    <root>
       <level value="ERROR" />
    </root>
    <logger name="MyCompany">
       <level value="WARN" />
    </logger>
    <logger name="MyCompany.MyDepartment.MyClassA">
       <level value="DEBUG" />
    </logger> 

    This can be changed on-demand simply by changing the configuration file, which can be reloaded automatically without restarting the application.  This allows us to a) not be blinded by too much trace information and b) not impose undue server load by tracing, whereas a non-hierarchical approach only allows for one logging level for an entire application. 

    Suppose it is desired to route all trace information (by default) to OutputDebugString, where a utility such as dbmon.exe or DebugView can be used to view it.  Suppose further it is desired to prefix our output with the date, thread Id, logging level, the name of the logger (i.e. MyCompany.MyDepartment.MyClassA), and the current context.  To do this with log4net, we would have our configuration file declare an “appender” of the appropriate type, and include a declaration of a “layout” as follows.  The “appender” is then referenced by a particular logger (or the root logger).

    <appender name="OutputDebugStringAppender" type="log4net.Appender.OutputDebugStringAppender" >
       <layout type="log4net.Layout.PatternLayout">
          <!-- The format specifiers let us add a wide variety of additional info -->
          <param name="ConversionPattern" value="%d [%t] %-5p %c [%x] - %m%n" />        
       </layout>
    </appender>
    <root>
       <level value="ERROR" />
       <appender-ref ref="OutputDebugStringAppender" />
    </root> 

    This might yield trace output as follows:

    16:39:33,022 [316] INFO  MyCompany.MyDepartment.MyClassA [SomeContext] - Hello World

    Again, the specifics of what additional information can be included in trace output can all be configured dynamically at runtime.  Note that other useful information such as method name, file name, line number, caller’s window identity can all be added with the PatternLayout class as well, though it is quite expensive (further adding value to the hierarchy concept!) 

    Moreover, each element of the hierarchy can be directed to a different appender, or to multiple appenders.  Built in appenders include support for ADO.NET, .NET remoting, SMTP, files, rolling files, buffering, and the Event Log (among others.)  Because the Event Log is supported, there is no need to use a separate API to support such logging. 

    Core log4net Concepts 

    Without attempting to present a full tutorial (see this introduction for a more complete discussion), the core concepts in log4net include: 

    Loggers: Use by code to emit trace output at various levels.

    • Loggers are organized into hierarchies just like .NET namespaces (they don't have to correspond, but often should.)  Example: System is parent of System.Text, and an ancestor of System.Text.StringBuilder. 
      • It is typical to have the logger name equal to a fully-qualified class name. 
    • Loggers are retrieved using a static method of LogManager, e.g. LogManager.GetLogger(someStringOrTypeName);
      • An ILog interface is returned with Debug, Info, Warn, Error, Fatal methods on it for tracing. 
    • Logging level is generally inherited from the hierarchy, and be configured anywhere in the hierarchy. 
      • Rule: logging level is equal to first non-null value working UP the tree.
      • Predefined levels: DEBUG, INFO, WARN, ERROR, FATAL, ALL, OFF
      • Default level for root in hierarchy is DEBUG 

    Appenders: An output destination for logging. 

    • Built in: event log, asp.net trace, console, file, rolling file, smtp, etc.
    • Appenders can optionally have "layouts" to specialize how formatting is done and add additional information (windows identity, source code info, context, thread id, timestamp, etc.) 
    • Appenders are attached to a logger.  Log requests will go to a given logger's appender, as well as all appenders attached up the hierarchy (unless "additivity=false" at some level, in which case moving up hierarchy stops…) 

    Filters: A given appender can apply a filter such that only log requests that match some criteria will actually be output.  Filters are applied sequentially.  Built-in filters include string match and regex match. 

    Configurators: Classes which initialize the log4net environment.  Configuration is done either through assembly attributes that specify a config file, or an explicit call to specify a config file.  The config file can be a standard .net config file, or standalone.  Configuration is by default at the appdomain level, but can be finer-grained through the use of log4net "logging domains".

    Using log4net with BizTalk 2004

    There are a few challenges to using log4net in a BizTalk 2004 environment.  They can be summarized as follows: 

    • Assembly-attribute-driven configuration won’t work because BizTalk 2004 does not appear to support the addition of custom attributes for BizTalk assemblies.  (And even if it did, this method would pose issues for BizTalk environments.)  
    • Using the log4net configuration classes for configuration is problematic because there is no clear point at which to make the call.  (How does as an orchestration know if it is “first” and should initialize?  How does it know at any point in an orchestration whether the BizTalk service has been recently recycled, losing the log4net configuration?)  Moreover, there is not a good way to identify where the configuration file should be located.
    • ILog-derived classes (log4net loggers) are not serializable “out-of-the-box”, making it difficult to use them in an orchestration setting.
    • Log4net context-storage mechanisms are thread-relative, which isn’t workable for orchestrations, since orchestrations often a) dehydrate and then re-hydrate on a different thread and b) make use of parallel branches.  We’d like to correctly associate context such as the orchestration instance ID on a logger that is scoped to an orchestration. 

    To address these challenges, I’ve created extensions to log4net that attempt to stay within log4net’s design intent.  These are housed in an assembly called “log4net.Ext.Serializable”.  (Update – these are now currently found in the “Extra Tools” download for the Deployment Framework – see the download links at the blog home page.)

    The main classes defined are as follows: 

    SLog: A serializable implementation of the ILog interface, to be used by orchestrations (or other components) for logging.  This class has an InstanceId property as well as a general properties collection that is carried with the logger itself.  (These can be made available in appender output via a PatternLayout class and the %P{yourpropname} format specifier.  Use %P{InstanceId} for the InstanceId.)  When deserialized, this class can detect an un-initialized state for log4net and recreate the correct configuration. 

    SLogManager: To be used in place of log4net’s LogManager to dispense SLog classes.  This is the accepted way of dispensing custom loggers in log4net’s design. 

    RegistryConfigurator: A class which consults a registry key for the location of a log4net config file, and creates the configuration in the specified “logging domain” using the log4net DomConfigurator class. 

    Specifics for BizTalk 2004 Usage 

    Using the assembly just discussed, the specifics of using log4net with a particular BizTalk project can be described as follows: 

    • Get 1.2.0 Beta8 of log4net at log4net extension described above, as well as a full sample project if you like.  (Update – the current version of this sample will now always be found in the Deployment Framework sample – see download links at the blog home page.)
    • If you are using NAnt to deploy your BizTalk solution, extend your customDeployTarget as shown below to create a registry key that will point to a log4net configuration file within your project’s root directory (whether on a developer desktop or an MSI base directory). (Handled for you in current rev of deployment framework.) Also, create a file with the log4net extension that corresponds to your solution name (i.e. BizTalkSample.log4net)

      <target name="customDeployTarget"> 
         <!-- Write registry key with location of our log4net configuration file.  This step is NOT needed in current rev of Deployment Framework… -->
         <exec program="cscript.exe" failonerror="true" commandline="/nologo ${deployTools}\WriteRegValue.vbs HKLM\SOFTWARE\${nant.project.name}\log4netConfig ${nant.project.basedir}\${nant.project.name}.log4net" />
      </target>
    • Orchestration assemblies should reference log4net.Ext.Serializable as well as log4net.dll, and declare a variable in each orchestration (perhaps named “logger”) of type log4net.Ext.Serializable.SLog.
    • Orchestrations should begin (after an activating receive, if necessary) with an expression shape that looks like:

      logger = log4net.Ext.Serializable.SLogManager.GetLogger(@"BizTalkSample",log4net.helpers.CallersTypeName.Name);
      logger.RegistryConfigurator();
      logger.InstanceId = MyOrchName(Microsoft.XLANGs.BaseTypes.InstanceId);
      logger.Debug("New request arrived...");

      Important: If an orchestration has parallel branches, you will want to declare multiple variables of type log4net.Ext.Serializable.SLog, assigning 2-n to the first one, as in:

      logger2 = logger;

      This prevents the need for synchronizing scopes.  (See this article for more.)&nbps;An alternative is to use scope-local logger variables that are initialized by assigning them to a global instance.

    • At various points in your orchestration, simply make calls to logger.Debug/logger.Info/logger.Error, etc.  Rely on logger.Error for event log entries (with appropriate logger/appender configuration.)
    • Use log4net with the components that your orchestration calls as well.  If those components are serializable, they will want to make use of SLogManager/SLog classes as well in non-static methods.  If you want context (such as the orchestration instance ID) to flow, it might be worthwhile to make an SLog-typed property visible on the class that the orchestration can set.  (This needs more thought.)

    Using log4net with Paul Bunyan 

    Paul Bunyan is a logging tool that has a very nice server-side buffering service as well as a great trace-viewing client that can connect to multiple servers simultaneously (a great plus for BizTalk groups!)  Although Paul Bunyan has its own API, we aren’t interested in it for purposes of this discussion.

    A log4net appender can be easily written for Paul Bunyan, such that log output can be directed there.  Moreover, this appender can be made to explicitly recognize an InstanceId property carried with a log4net logging event, and map it to the “Context” concept within Paul Bunyan!  This allows for viewing the orchestration instance ID within a distinct, filterable column in the Paul Bunyan viewer as such:

    Paul Bunyan View
    (click)

    Sample .log4net configuration File 

    Note that types have to be referred to with fully-qualified names since log4net.dll is being deployed to and loaded from the GAC.

    <?xml version="1.0" encoding="utf-8" ?>
    <log4net>

       <!-- Define some output appenders -->
       <appender name="OutputDebugString" type="log4net.Appender.OutputDebugStringAppender,
            log4net,Version=1.2.0.30714,Culture=Neutral,PublicKeyToken=b32731d11ce58905">
          <layout type="log4net.Layout.PatternLayout,log4net,
               Version=1.2.0.30714,Culture=Neutral,PublicKeyToken=b32731d11ce58905">
             <param name="ConversionPattern" value="%d [%t] %-5p %c - %m [%P{InstanceId}]%n" />
          </layout>
       </appender>

       <appender name="EventLog" type="log4net.Appender.EventLogAppender, log4net, 
            Version=1.2.0.30714, Culture=Neutral, PublicKeyToken=b32731d11ce58905">
          <param name="ApplicationName" value="BizTalkSample" />
          <layout type="log4net.Layout.PatternLayout,log4net,
               Version=1.2.0.30714,Culture=Neutral,PublicKeyToken=b32731d11ce58905">
             <param name="ConversionPattern" value="[%t] %-5p %c - %m [%P{InstanceId}]%n" />
          </layout>
          <!-- Only send error/fatal messages to the event log. -->
          <filter type="log4net.Filter.LevelRangeFilter">
             <param name="LevelMin" value="ERROR" />
             <param name="LevelMax" value="FATAL" />
          </filter>
       </appender>

       <!-- Setup the root category, add the appenders and set the default level -->
       <root>
          <!-- Possible levels (in order of increasing detail):
                         OFF, FATAL, ERROR, WARN, INFO, DEBUG, ALL -->
          <level value="ERROR" />
          <appender-ref ref="EventLog" />
       </root>

       <logger name="BizTalkSample">
          <level value="ALL" />
          <appender-ref ref="OutputDebugString" />
       </logger>
       
    </log4net>
    Wednesday, January 19, 2005 11:43:06 AM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    BizTalk Tools
    Archive
    <May 2005>
    SunMonTueWedThuFriSat
    24252627282930
    1234567
    891011121314
    15161718192021
    22232425262728
    2930311234
    About the author:

    Scott Colestock lives, writes, and works as an independent consultant in the Twin Cities (Minneapolis, Minnesota) area.

    © Copyright 2014
    Scott Colestock
    Sign In
    All Content © 2014, Scott Colestock
    DasBlog theme 'Business' created by Christoph De Baene (delarou)