Have you ever wanted to get real-time metrics on:  How many times has a particular orchestration executed on this node in the BizTalk group?  What does the average execution time for my pipeline component looking like?  How many instances of a particular orchestration are currently executing across the group, not including dehydrated instances?

Questions like this can be tough to answer - the built-in BizTalk counters are too coarse-grained, and Tracking & BAM are only partially useful here (particularly if you want high frequency samples.)  What you would like is the ability to get fine-grained "request metrics" on all of your BizTalk orchestrations and pipelines in a way that is simple to implement and easy to consume.

I've been leading an open source project called QuickCounters for quite some time now - though we've just recently moved into CodePlex.  It is a library that will be of interest to BizTalk developers - but it can be used in much broader settings.  I had a chance to present on this topic at the Twin Cities .NET Users Group (on 11/2/2006 - presentation here.)

QuickCounters is a .NET library which makes it extremely easy to implement the common, request-level performance counters that are interesting in just about any service you might write.  You can use this library to instrument  general .Net components, web services, BizTalk orchestrations, pipeline components, remoting interfaces, enterprise service components...you get the idea.  There is special support in the library for BizTalk and WCF scenarios, but it is quite easy to use in any case.

QuickCounters recognizes that the Windows Performance Counter technology that has been with us for some time is often still the best choice for providing (and consuming) detailed performance metrics.  QuickCounters also recognizes that for any given service request there are several metrics which turn out to be interesting for performance testing, operational health analysis, and historical trending. The idea with QuickCounters is to raise the level of abstraction from that of an individual performance counter up to the request itself, using a simple API.

Suppose you want to gather these metrics for each type of request in your system:

• Requests Started
• Requests Executing
• Requests Completed
• Requests Failed
• Request Execution Time
• Requests/Hour
• Requests/Min
• Requests/Sec

Although the .NET performance classes would give you a good start, it will still be a chunk of work.  With QuickCounters, you describe your requests in a simple xml format, and include a simple code snippet in each request implementation:

void SampleRequest()
{
   RequestType someRequest = RequestType.Attach("MyApplication","someRequest");
   someRequest.BeginRequest();

   try
   {
      // Do useful work...
      someRequest.SetComplete();
   }
   catch
   {
      someRequest.SetAbort();
      throw;
   }
}
        

After a quick "install" of the xml that describes your requests (via the included "QuickCounter viewer" utility) you are on your way.  You can see all eight metrics described above for all of your requests (which each become a Performance object) - in PerfMon, with MOM, or any other performance counter consumer. 

For a BizTalk orchestration, you will simply have a variable of type RequestType, which you will assign in an expression shape at the top of your orchestration via the static "RequestType.Attach" method, followed by a call to BeginRequest: 

quickCounterDemo = QuickCounters.RequestType.Attach(
    "QuickCounterDemo","demo",
    QuickCounterDemo(Microsoft.XLANGs.BaseTypes.InstanceId)); 
quickCounterDemo.BeginRequest();           

Successful completion should end with another expression shape that calls SetComplete, otherwise SetAbort.  (Full sample referenced below.)  And yes, the library understands that an orchestration's execution will often begin on one node in your BizTalk group and continue/complete on a different node. 

Of course, if you have many requests, and many nodes in your BizTalk group, it can be a hassle to add everything to PerfMon.  20 requests * 3 servers = 60 interactions with PerfMon's UI.  You might remember to save an .msc file, but you might not.  Here is where the QuickCounter viewer comes into play.  Since you've already written an xml file that describes your requests, just let the viewer know about that file -  and a comma separated list of servers you are running on:

(click)

(The viewer has benefitted substantially from John Thom's contributions...)

Now, the WCF support in the library is...amazing, and completely the result of contributions from Tomas Restrepo.  Keep an eye on his blog for a full discussion, but the bottom line is that creation of the simple xml format and all API calls (BeginRequest/SetComplete/SetAbort) are handled for you automatically simply by applying an attribute (or a behavior - your choice) to your WCF service implementation.  How cool is that?

Our CodePlex home is here.  Downloads and source code here.  Contributors here.

I expect we'll release about once a month or so for awhile - give it a spin and let us know what you think.  It is being used by some very large BizTalk projects (in production) with good success right now.