a trace of thought on...BizTalk Server, Team Foundation Server, AppFabric, etc. RSS 2.0
 Tuesday, May 24, 2005

As has been noted before in the lore of BizTalk, it sure would be useful to use a real debugger with orchestrations - at least occasionally.

There are times when an expression shape winds up getting a bit sticky (not that I would know...) and a debugger would be just the ticket.  Other times, the exception you are getting from the orchestration engine isn't at all clear.

Still other times, seeing the actual contents of messages or context as you step through would be interesting.

Jon has posted on debugging orchestrations (in IL) with an ILDASM/ILASM loop, and had also discussed building your orchestration assemblies manually (using xsharpp.exe).  (He correctly noted these approaches wouldn't generate anything supportable, but they work for spelunking.) 

I wanted to suggest something a bit different - I wanted to go back instead to the venerable BTS File Dump utility that Charles released in May of 2004 (before the GenerateCSFiles registry key was discovered...) and propose a different technique for symbolic debugging.

First, install Charles' BizTalk File Dump utility - you can download it from here.  Fire it up, and change the output path in the utility to something easy, like c:\temp\BTSFileDump.

Here is the Edit/Debug cycle...(Start by making sure the file dump utility is running, and click the "Start dumping files" button.)

  1. Build your orchestration project (or whole solution if need be.)
  2. Do a Ctrl-Shift-F (Find in Files) in Visual Studio, and change the "Look in:" folder to c:\temp\BTSFileDump
  3. Search for something in one of your expression shapes, say, "MyClass.Execute".  The correct generated file (that the file dump utility grabbed) will appear in your Find Results - open it up, and set a breakpoint.  (Not on the xml designer portion - on the actual code!)
  4. From the Debug menu, choose Processes and attach to BTSNTSvc.exe.  (Have more than one?)  Choose CLR debugging only - not native.  The symbols should be loaded automatically - no need to copy PDBs to the GAC.
  5. Trigger your orchestration however you normally would.  Bask in a picture like this:  (Puts a Petzold-style WindowProc to shame...) 

    Debugview
  6. Use QuickWatch to examine message contents/context, if you like - you'll be interested in expansions like this one (where sampleRequest is a message in the orchestration.)
  7. Find your problem, Debug-Detach All, and fix the problem in the orchestration.
  8. Click "Delete all files in output path" in the file dump utility (to avoid duplicates) and rebuild.  Repeat the process if you need to.  (To save time, consider an external tool that just re-GACs the orchestration assembly and resets the host process.  See here.)

Note that this technique will work in a production setting.  You could copy the PDBs and sources to the production server, and use DbgClr.exe (in the Framework SDK) or cordbg.exe to attach to the appropriate host process.

Happy debugging...

Tuesday, May 24, 2005 12:42:15 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
BizTalk Insights
 Thursday, May 12, 2005
This is a test description.
Thursday, May 12, 2005 9:25:39 PM (Central Standard Time, UTC-06:00)  #    Comments [4] -
BizTalk Insights

Andy Morrison wrote about a technique for identifying the correct host instance to attach to when debugging components associated with orchestrations.

A potentially easier solution is to keep perfmon running with the counter shown below (BizTalk:Messaging/ID Process - all instances.)  The counter values will update every time a host instance recycles.  The Debug-Processes dialog in Visual Studio will show you (and allow you to sort by) process IDs to make this easy. 

Then put PerfMon in "report view" to easily see process IDs by host instance:

Thursday, May 12, 2005 9:32:41 AM (Central Standard Time, UTC-06:00)  #    Comments [3] -
BizTalk Insights
 Wednesday, May 11, 2005

There have been several folks who have discussed how to create messages "from scratch" within an Orchestration context - you can read Matt's thoughts and check out a BizTalk documentation excerpt. This is a common question, and I had an additional technique I thought I would share... 

Background: When you have a schema that has many promoted properties (or distinguished fields), or many elements that can be set via xpath expressions easily, it can be useful to simply start with a "template" document instance and populate the element content that you are interested in. 

In this situation, you will often have a "Message Assignment" shape that looks something like this:

xmlDoc.LoadXml("<ns0:BizTalkSampleS3 
      xmlns:ns0="http://BizTalkSample.Schemas.BizTalkSampleS3">
      <SomeElement></SomeElement></ns0:BizTalkSampleS3>");
someMsg = xmlDoc;
someMsg.SomeElement = "some content";
// (or xpath(someMsg,someElementXPath) = "some content" if we don't have a 
// distinguished field.)
   

One disadvantage of loading up "template" xml documents from either expression shapes or code (via XmlDocument.LoadXml) is that those xml fragments can get easily "lost", and are hard to update early in the development cycle when schemas may still be in flux.  Loading the template files from the file system is problematic because the question arises "where should I store these files, so that I can find them in any environment I deploy to?" (Solvable, but a hassle.)

Instead, why not embed the template xml documents as assembly resources?  For those unfamiliar with that process, I have a short tutorial here (& a helper class.) 

  1. You will need a C# project as part of your overall BizTalk solution.  Place your template xml file(s) in the directory corresponding to this project, and add them as an "existing item" to the project. 
  2. Select this file within the Solution Explorer, and within the Properties window, select "Embedded Resource" as the "Build Action" as shown here:

     

  3. Place this class (text here) within the same C# project that houses the resources you have added.
  4. To construct a message, drag out a "Message Assignment" shape, and within the associated expression write some code like the following.  Simply pass the file name of the template document as an argument to GetXmlDocResource (or GetStringResource.)
    sampleUsingTemplate = 
       BizTalkSample.Components.EmbeddedResourceCache.
          GetXmlDocResource("BizTalkSampleS3_output.xml");
       // Populate the "rest" of the message with distinguished fields, promoted 
       // properties, xpath expressions, etc.
       sampleUsingTemplate.SomeElement = "foo";
                

The class I have supplied will cache the loaded resources in a hashtable for performance sake, and allow you to load resources as both strings and XmlDocuments.

A last thought: Many people ask, "Why can't I just create a message using a new operator or a default constructor of some sort?" Well, because few XSD schemas sufficiently constrain the set of valid instance documents enough for that to be useful - what form would a "default message" take? (Would it have the optional elements you need? Some elements that you don't want?)

Enjoy - feedback appreciated!

Wednesday, May 11, 2005 3:29:14 PM (Central Standard Time, UTC-06:00)  #    Comments [6] -
BizTalk Tools | BizTalk Insights
 Monday, May 9, 2005

Just a few notes on what I'll be up to in the next month or two...

Do you use the Deployment Framework for BizTalk?  Or just have general questions on automating BizTalk deployments?  You can attend my session at TechEd 2005 - Friday June 10th, 10:45am to 12:00pm in the BPI Cabana, where I'll be doing a talk on those topics...

I'll also be speaking at the Microsoft DevCon event on my "other favorite" topic, Windows Mobile.  The focus will be on what is new for Visual Studio 2005 in the mobile arena...The conference will also be covering the new Team System offerings, ASP.NET 2.0, and quite a few other VS2005 topics.  Check it out...

Monday, May 9, 2005 8:10:41 AM (Central Standard Time, UTC-06:00)  #    Comments [3] -
Deployment Framework | General
 Friday, May 6, 2005

(Update: See the latest on the Deployment Framework here.)

Well, its time for another revision of the BizTalk/NAnt deployment tools... 

For folks recently joining us, these tools allow you to quickly create both developer workstation deployments (inside Visual Studio) and MSI-based deployments (for servers) for BizTalk 2004 solutions.  Just about all aspects of BizTalk deployments are handled, including items such as virtual directories, SOAP proxies, and WinXP/2k/2003 differences.

I get quite a few emails from people who are using these tools (which is great!)  A couple folks have suggested that I start referring to the whole thing as a" p framework?. < deployment>

That sounds about right - like all frameworks, there is a core set of functionality that takes care of quite a bit of work, and a given project can supply what needs customization or what is missing. 

In this case, the core NAnt scripts, custom NAnt tasks, WiX generation, and other custom tools form the framework itself - and a given BizTalk project can supply project-specific details, what needs to be customized, and any missing functionality.   So I'll refer to this work as the "Deployment Framework for BizTalk" from now on... 

What has changed in this rev? 

  • When last posting on this topic, I should have been even clearer that as of the last release, the deployment framework requires NAnt .85.
  • The biggest improvement is the ability to use binding files as they are emitted by the BizTalk Deployment Wizard directly.  In the past, the framework had insisted that binding files be split between orchestration bindings and messaging bindings - this is no longer required.  I had several requests for this fix...Also, note that having multiple binding files is supported (like one for each of several assemblies.)  Now, if you wish to continue to use the "old style" split bindings, define the orchBindings property in your project NAnt file as shown below:
  • <property name="orchBindings" value="${nant.project.name}.OrchBindings.xml" overwrite="false" />

  • You can now set the value of NAnt properties from the SettingsFileGenerator spreadsheet.  Recall that this spreadsheet is used (in conjunction with XmlPreProcess) to manage environment-relative configuration - that is, to manage those values that vary depending on whether you are in a development, QA, or production environment.  Until now, it has been difficult to use those configuration values within the NAnt deployment file!  You typically just used them in your binding files or other application-level XML files.  Now, however, you can define a property as shown below to make these values available as NAnt properties.  A good example usage is the case where you want the actual names of the SSO Application Users group and SSO Administrators group to be defined in the SettingsFileGenerator spreadsheet (because the group names may vary by environment you deploy to.)  Just define a property in your project NAnt file called "propsFromEnvSettings" with a comma-separated list of values you want pulled in.  (The sample has been updated to show this, too.)
  • <property name="propsFromEnvSettings" value="ssoAppUserGroup,ssoAppAdminGroup"/>

  • Many people had asked for the DeployResults.txt file (which is created in the DeployResults subdirectory during a deployment) to be available in a history log, so that it would be easy to walk up to a production server and determine what deploy/undeploy activity had taken place (especially useful for diagnosing any failures.)  To that end, the deployment framework will preserve past DeployResult.txt files with this convention: `DeployResults TRACESATURN_05032005_0922.txt'  Note that the machine name, date, and time are preserved - so you can gather these files across all the servers in a BizTalk group, if need be.
  • SSOSettingsFileReader class improvements: In the last release, I introduced the ability to manage settings within the BizTalk SSO using the SettingsFileGenerator spreadsheet that is also used for managing environment-relative configuration.  Part of that functionality included a class called SSOSettingsFileReader, used to retrieve settings at runtime.  This class now has some strongly typed accessors, and no longer sits inside a lengthy namespace, which means that in an orchestration expression shape you can simply do:
  • someVar = SSOSettingsFileReader.ReadString("BizTalkSample","SomeAppConfigItem") // or ReadInt32

  • Ben Cops had found that on large-scale deployments, it was useful to restrict which orchestrations were deployed/started to speed the development cycle.  So, he modified the orchestration NAnt task accordingly.  To use (if you need to), just set the orchestrationsToIgnore property in your project NAnt file with a comma-separated list that reflects what you are currently working on.  Make sure to copy BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory again! (you'll find it in the DeployTools directory in the download.)
  • A bug was fixed with undeployment for the case where an SSO affiliate app had already been deleted...
  • All references in NAnt files to xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" were removed, because they were simply causing heartache (for many reasons.)  Remove this namespace in your project NAnt file when upgrading.
  • Rather than trying to keep the source for the C# tools (NAnt tasks, SetEnvUI, DeployBTRules, SSOSettingsFileImport\SSOSettingsFileReader, etc.) current within the GotDotNet workspace, I'm making them available as a zip

Remember, you do not need to piece together the current functionality set from past blog entries - the complete documentation (in Word form) is in the download and separately here.

Some last thoughts:

  • Some folks have been reluctant to use the deployment framework because of its reliance on NAnt, and a worry over using NAnt in a server environment.  One thing to remember (that might allay this concern) is that a subset of NAnt is delivered with the generated MSI, and NAnt will not require a separate installation procedure by your production operations team.  Moreover, NAnt doesn't create any kind of persistent service or footprint on the production machine (i.e. no automatic file associations that would cause scripts to run with a double-click or anything like that...)
  • Add a Visual Studio External Tool for the updateOrchestration target if you haven't already!  This can speed your development cycle significantly.  See the tools zip for a script that creates all relevant external tools.
  • When using the WiX-based MSI generation, make sure your project NAnt file (for WiX generation, like BizTalkSample.WiXSetup.Build) is choosing "debug" or "release" based on what you want to put in the MSI.  See comments in that file for more detail, and Jon Flanders' post on this topic.

How do I upgrade?

  1. With a safe copy of your BizTalk solution, expand the "Core" zip contents right on your project directory, such that the core scripts will overwrite what you have now (but your project-specific scripts will be preserved.)
  2. Manually copy the WiX-related updates to your *.WiXSetup directory, since they will get unzipped to BizTalkSample.WiXSetup
  3. Remove xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" from your project NAnt file
  4. Copy DeplyTools\BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory
  5. Do a deployment on a developer workstation and a server to test things out...

Downloading

  • You can always use the download links on this blog.
  • Full sample here, core scripts here, extra tools here
  • (But always check the download links to see if there has been an update…)
  • Standalone documentation here (and diagram here.)
  • GotDotNet workspace here
    Friday, May 6, 2005 1:13:00 PM (Central Standard Time, UTC-06:00)  #    Comments [5] -
    Deployment Framework

    (Update: See the latest on the Deployment Framework here.)

    In the latest release of the Deployment Framework for BizTalk “extra tools”, you will find a script that will automate the creation of “external tools” in Visual Studio that are useful for working with BizTalk 2004 and the Deployment Framework.  Modify the script to include all the tools that your team relies on, too.

    The idea that this should be automated came from Ben Cops (though I’ve heard the complaint before!)  Manually creating all the useful external tools was somewhat painful for a large team of BizTalk developers.

    After you run the script, your Tools menu in Visual Studio should include entries for deploying and un-deploying the current solution, as well as for updating just the orchestrations, .NET components, and SSO configuration data (one menu entry.)  In addition, there will be entries for running NAnt on a highlighted target name (useful if you have your build script open), and for running HAT and the Subscription Viewer.  Enjoy…

     

    Friday, May 6, 2005 12:32:03 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    Deployment Framework
     Thursday, March 24, 2005

    A short while ago, I did a presentation for the Twin Cities .NET user group on log4net.  (You might recall an earlier blog entry where I discussed using log4net with BizTalk 2004...) 

    The presentation was not specific to BizTalk 2004 - instead, it attempts to describe why I think the log4net library is so very well thought out.  You can grab it here, if you like.

    The last slide references Loren Halvorsen's comparison of log4net and the new Microsoft Enterprise Library Logging Block, which I would recommend taking a look at (the comparison, that is.)  Tom Hollander (among many others!) later weighed in with this piece.

    Lastly, 1.2.9 beta of log4net has recently become available (release notes here)....However, my extensions to log4net have not yet been updated to reflect this new drop.  I'll be sure to post when I'm able to update.

    Thursday, March 24, 2005 4:07:03 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    General
     Monday, March 21, 2005

    (Update: See the latest on the Deployment Framework here.)

    Just when you thought automated deployments for BizTalk 2004 were safe, another update to the “Deploy with NAnt” template comes along… :) 

    This is a very worthwhile "upgrade" to consider taking advantage of.  One of the big items is support for "configuration info within SSO", but a complete list of features/fixes, etc. is below. 

    SSO Integration: If you are currently using the SettingsFileGenerator.xls spreadsheet (discussed in previous entries) in conjunction with the XmlPreProcess tool to handle "environment-relative" configuration - that is, to handle variations between your dev/QA/production environments - you will be glad to know you can get more leverage out of that process now…

    To review, with this process, you use the names that are defined in the first column of the SettingsFileGenerator spreadsheet (shown below) as "tokens" within your binding file, as in: 

    <!-- ifdef ${ xml preprocess} -->
    <!-- <Address>${FileSendLocation}\%MessageID%.xml</Address> -->
    <!-- else -->
    <Address>C:\temp\BizTalkSample OutDir\%MessageID%.xml</Address>
    <!-- endif -->

    At install time, the value for FileSendLocation that is appropriate to the environment you are deploying to is "plugged in" by XmlPreProcess (but for developers, the file remains legal XML.)  The spreadsheet might look as follows:

    Image0031
    (click)

    Now, however, you can also put in name-value pairs (whether they vary by environment or not) into the spreadsheet that you need access to at run time (i.e. general configuration information.

    The name-value pairs will be deployed into a newly created affiliate application within the SSO, and can be read using the SSOSettingsFileReader class (which serves as a cache, too – and is provided in the download.)  The static methods on this class can be used from an orchestration expression shape, or from assemblies that orchestrations call – as in string test = SSOSettingsFileReader.ReadValue("BizTalkSample","SomeAppConfigItem");  (By the way, the “nested names” shown in the spreadsheet above are just meant to suggest a partitioning mechanism you might want to use, rather than one flat “namespace” of config data.)

    Of course, the deployment scripts handle all the interaction with the SSO database for deployments/un-deployments.  This whole mechanism (SettingsFileGenerator.xls, XmlPreProcess, and now SSOSettingsFileImport.exe) is worth using even if you don’t use the rest of this script infrastructure.  Note that Jon Flanders had a great article on SSO-based configuration using a strongly-typed approach, where as this approach is loosely-typed, but leverages a mechanism you might already be using (i.e. the spreadsheet concept.)  (Note: Jon’s code for creating/deleting affiliate apps helped me complete the SSOSettingsFileImport.exe utility quickly…) 

    Other features/fixes: 

    • Support for multiple schema assemblies, including the case where schema assemblies reference each other due to schema imports.  (Undeployments occur in reverse order of deployments.)
    • Support for multiple orchestration assemblies, including the case where orchestration assemblies reference each other due to the use of Call/Start Orchestration shapes.  (Undeployments occur in reverse order of deployments.) 
    • Support for multiple orchestration and port binding files (just list them as comma separated values in the orchBindings/portBindings properties.) 
    • Fixed bug that occurred when undeploying ISAPI extensions in Windows 2003 / IIS 6. 
    • Fixed bug in undeploying pipeline components (thanks to John Adams) 
    • Added /NOFORCE flag to IISRESET calls in accordance with http://support.microsoft.com/kb/286196 
    • New property "includeInstallUtilForComponents" was added, which can be set to 'true' in your project-specific nant file.  This will cause your component assemblies to be called with "installutil.exe" (with the /u flag for undeployment.)  This is useful if you have .NET Installer-derived classes that need to get called for creating event sources, perf counters, etc.  (Yes, you could certainly argue these should be called by the WiX piece of the script infrastructure instead, but this method works and is more in keeping with the spirit of the entire process.)  Note there is no harm if a given component assembly listed in your components property does not have an installer class. 
    • Eliminated the need for the includeCustomTarget property, and provided additional ways to supply custom functionality.  Project-specific nant file can now supply any or all of customDeployTarget, customPostDeployTarget, customUndeployTarget, customPostUndeployTarget -- and they will be called if they are present. 
    • Integrated the deployment of log4net-specific stuff, if the includelog4net property is set to true.  See the work on BizTalk/log4net integration for more information.  (Log4net sample usage is included in the new sample.)   
    • Now using NAnt version .85 - so be sure to grab the latest NAnt/NAntContrib since the nant scripts are using .85-specific syntax to avoid "deprecated" warnings that were occurring otherwise.  Note that even once you have installed NAnt .85, you will still be able to deploy BizTalk projects that are using old versions of the deployment scripts.  Note also that this is a pretty basic port to .85 - there are no doubt more elegant ways of doing lots of things that I haven't investigated yet, now that nant has expression support. 

    Other notes 

    The scripts like to see BizTalk projects using Debug and Release for target names.  A long while back I had suggested a file-level search/replace within btproj files to change these, but two more elegant options are available: 

    • In project properties, change the output location for the “Development” configurations to “bin\debug” and output location of “Deployment” configurations to “bin\release”.
    • (Via Puneet Gupta): Change the template for BizTalk projects, found in BTSApp.btproj file in %BizTalkInstallDir%\Developer Tools\BizTalkWizards\BTSApp\Templates\1033, to reflect Debug and Release targets. 

    Another note: Lots of folks have noticed that XmlPreProcess can be used for a lot more than port binding files!  Log4net config files, orchestration binding files, you name it.  See the full Word documentation in the download for an example of how to do this within customDeployTarget.

    How do I upgrade?

    1. With a safe copy of your BizTalk solution, expand the “Core” zip contents right on your project directory, such that the core scripts will overwrite what you have now (but your project-specific scripts will be preserved.)
    2. Manually copy the WiX-related updates to your *.WiXSetup directory, since they will get unzipped to BizTalkSample.WiXSetup
    3. Do a deployment on a developer workstation and a server to test things out…

    Downloading

    • You can always use the right-hand links on this blog.
    • Full sample here, core scripts here
    • GotDotNet workspace here.  Because of issues with GotDotNet, I’m only going to be keeping the EXE utility source code up to date – not the entire sample & core tree.

     Enjoy – comments always welcome and appreciated.

    Monday, March 21, 2005 2:31:28 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
    Deployment Framework
    Archive
    <May 2005>
    SunMonTueWedThuFriSat
    24252627282930
    1234567
    891011121314
    15161718192021
    22232425262728
    2930311234
    About the author:

    Scott Colestock lives, writes, and works as an independent consultant in the Twin Cities (Minneapolis, Minnesota) area.

    © Copyright 2014
    Scott Colestock
    Sign In
    All Content © 2014, Scott Colestock
    DasBlog theme 'Business' created by Christoph De Baene (delarou)