It can turn out to be useful to use Health and Activity Tracking (HAT) and the BizTalk Server Administration Console “remotely” (that is, from a machine other than one of the actual servers in the BizTalk group.)  This can be the case if you have multiple environments to administer, and/or using Remote Desktop is not desirable or available.

It should be noted that to accomplish this, you technically have to have the “administrating machine” join the BizTalk Group – though since you won’t have any host instances defined on the machine, that isn’t as significant as it sounds.  You are essentially just configuring the machine to point at a particular BizTalk management database, and configuring some WMI information.  The “administrating machine” does not appear in the “Servers” node of the BizTalk Administration Console.

It should also be noted that you will need to be in the “BizTalk Administrators” Windows group for the BizTalk environment you want to manage. 

To get started, the “administrator” should do an “Admin-only” install of BizTalk on the machine they will be using, where the installation options for BizTalk look like this:

Then, on the desktop, the administrator should put a shortcut to a script that looks like the script below.  This script simply reminds the user what BizTalk Group they are currently administrating, and confirms they want to switch:

If you select OK, you get another warning:

This is basically warning someone who has a full-blown BizTalk installation on their machine that they really don’t want to remove their current configuration unless they happen to have a saved configuration file from their last run of ConfigFramework.exe.

If you select OK, the “ConfigFramework.exe” utility is run (from BizTalk’s installation directory) with the /u switch, to remove the current configuration.  Afterwards, ConfigFramework.exe is run normally, and the only option you will have (for an admin-only install of BizTalk) is to select the database corresponding to the environment you wish to administer:

Once you hit “Next”, the wizard will complete, and you will be presented with a final confirmation dialog:

The VBScript as a text file is here.

I recently went through a really nast bout of troubleshooting with the client I currently work with, related to MSMQT.  Hopefully, my tale can save you similar pain.

The core issues was this: The BizTalk MSMQT adapter can be configured during installation to integrate with Active Directory.  The default is that it will not operate in this fashion, but rather in "workgroup" mode.  There are (at least) two reasons why you might want to have MSMQT integrate with Active Directory: 1) you want to make use of an MSMQ router in your environment or 2) you want to use certificate-based authentication at a protocol level (where the public certificate is managed by AD.)  (Note: I know this now; I didn't know it a couple weeks ago...)

We have been installing our servers in "workgroup" mode.  To install in Active Directory mode requires a special permission granted by the domain administrator.

Now, when you a configure a Send Port within BizTalk and select MSMQT as the transport, the property pages in the BizTalk Explorer offer a checkbox that is labeled "Use MSMQ Authentication".  If you hit the "Help" button on this dialog, the explanation that is provided is this: "Identify whether BizTalk Message Queuing uses protocol authentication every time it sends a message on this port." 

As it turns out, although it isn't documented as such, a Send Port with this option checked can only work if MSMQT has been installed in Active Directory-integrated mode.  If you have the "Use MSMQ Authentication" option checked on a Send Port and you are not in Active Directory-integrated mode, then messages will not flow.  When we eventually discovered this discrepency and fixed our bindings files, the problem was resolved.  (Note: there is a similar option when configuring Receive Locations.)

This checkbox had been checked at the point our initial binding files were exported, and became a part of our scripted deployment.  What was worse, when we encountered this problem a few weeks ago in QA, we began troubleshooting the BizTalk configuration on the server directly and wound up "fixing" the problem by creating an additional Send Port (subscribing to the same traffic as the original) that simply had the MSMQ Auth checkbox off.  But we didn't realize that discrepancy at the time, so we had to troubleshoot the same problem all over again a few weeks later.  We definitely got ourselves into the wrong troubleshooting mindset by assuming that Biztalk was flaky in some way.

Key lesson: If you don't get into a given environment (QA, production, whatever) with your scripted deployment, then you really didn't get there at all….

A few more notes.  As I said above, if you have the "Use MSMQ Authentication" option checked on a Send Port and you are not in Active Directory-integrated mode, then messages will not flow.  What you will see is:

• Messages will appear in the HAT "Queries-Messages Sent in Past Day" report, but they will not actually have arrived in the destination queue.  (Fixed in SP1?)
• You will see strange behavior in the HAT "Operations-Messages" view, but nothing that indicates an error condition.  Retry count will increment on the original service instance.
• There will be no error condition reported in the event log.  (OK, Premier Support indicates in a phone conversation you might see something after 5 days have elapsed, when an exponential backoff algorithm has run its course.)

IMHO, Biztalk 2004 should be more serviceable in this regard, and should give better error information.  And of course, the documentation for MSMQ Send Port configuration should have mentioned that MSMQ Authentication would only work for Active Directory.

Microsoft Premier Support became involved, and after around 18 hours of analysis they said "We see some certificate-related errors in the traces.  Do you use MSMQ authentication?  Are you AD-integrated?"

We looked in our binding files (since the decision had long since been forgotten) and saw this snippet:

See that in the escaped XML?  Yup, that is a property called "Authenticated" that is an old-fashioned Variant of type bool, where "-1" means "true".

Leaps out at you, right?  Determining if you are AD-integrated means looking at

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0\MessageQueuing\MsmqtWorkgroupMode.

From all of this I (gently) conclude that the product instrumentation/tracing should point out this condition more quickly to a support engineer.  In addition, the MSMQT adapter should warn you of a mismatch during configuration with the Biztalk Explorer and, ideally, when you deploy/import bindings.

Hindsight being 20/20, the support engineer should have asked to see our binding file - and should have compared it with one exported from a server that was indeed sending messages (since we had one.)  Of course, we should have made such a comparison, too! (and much earlier...)  The engineer did look at the Biztalk Admin Console, but of course that doesn't give any of the detailed port configuration information - only Visual Studio/BT Explorer does.

Having said that, the support engineers were great to work with and were certainly dedicated to getting to the bottom of our issue.

Key lesson: Diffing binding files will prove to be a key troubleshooting technique with Biztalk...

(Update: See the latest on the Deployment Framework here.)

It has been several months since I initially posted on the topic of using NAnt to coordinate the deployment of BizTalk 2004 solutions, with updates here and here.  Since that time, I've been heads-down in a BizTalk project and have had a chance to refine the practice quite a bit.

So, I took the opportunity to introduce quite a few improvements into the sample I had previously released - hopefully turning it into a "template" that can be used on your projects more easily.  To that end, I've created a GotDotNet workspace where you can download the current version of the build template, and I invite any feedback/suggestions/participation.

The major changes I've introduced can be categorized as follows:

• Split binding files: On a BizTalk project with multiple developers, I believe it is beneficial to split your binding files into content related to orchestrations vs. content relating to "everything else" (send ports, receive ports, etc.)  The new build template sample does just that.
• Improved scripts: Many of the WMI-related scripts I relied on in the past were from BizTalk SDK samples.  These were modified to get better/different error handling behavior.
• More generic handling of send ports, receive ports, receive locations, etc. in the script - though it still requires some maintenance.
• Better NAnt citizen: I'm still not a NAnt expert by any stretch, but thanks to Duncan Millard I realized that I had been remiss in applying quotes to a good deal of the text names I was using (i.e. &quot).  In addition, I started using built-in NAnt tasks for at least a few items where I had overlapped functionality, though not in cases where I needed slightly different behavior.  Finally, the build file naming convention proposed by the build template is "BizTalkSample.sln.deploy.build" (rather than "BizTalkSample.sln.build") to avoid conflicting with a NAnt file that would actually be used to build the project (e.g. with Cruise Control), as opposed to deploying it.
• Use of "Deploy with NAnt" in an MSI scenario: This is the biggest (and I believe most useful) change...Let me explain further.

As you study the build script, you will notice that it now also tackles such tasks as deploying virtual directories and applying permissions to them, patching binding files to match “local conditions”, etc. The initial goal here is to ensure that in a team development scenario, the amount of “out of band” setup required for any given developer to establish their BizTalk 2004 project environment is minimal – and that the current topology for the project is communicated efficiently (i.e. through the build file, as opposed to email or word-of-mouth.)

After a development team has spent weeks or months with this build script, refining it to represent their exact situation and deployment needs, a question might arise: Why not take this well-tested script and use it for production (or just non-development-machine) deployments as well?

To do this, it is helpful to agree that a two-phase deployment of BizTalk-related projects is both acceptable and useful. The first phase is an MSI-based installation that simply installs all of the BizTalk-related assets in a specific directory, but doesn’t deploy them to the BizTalk server. The second phase occurs when the user goes to the Start-Programs-YourProjectName menu, and chooses “Deploy YourProjectName”. The user also has the option to un-deploy (leaving the assets still on the file system until the MSI is uninstalled) or redeploy (to support the case where a single file is “patched”.)  See a picture of the Start menu created by the build template's MSI file.

The reason that two phases are useful is that if the deployment to BizTalk fails, we would like a complete log of the results (which in this sample occurs in the DeployResults directory) for analysis/diagnostics, and we would rather not have the MSI simply roll back. The same holds true of un-deployment.

Dual-purposing the NAnt file you have been maintaining for “real” deployments can be summarized as follows:

• Use the build template's approach (see BizTalkSample.sln.deploy.build in the download) for the “server.deploy” and “server.undeploy” targets, which make the build file able to work with “co-located” binaries in addition to the standard developer tree.
• Create a Visual Studio Setup project that deploys your BizTalk assemblies according to the pattern shown in the BizTalkSample.Setup project. Notice that this setup project includes the NAnt build file, the “DeployTools” directory (with our scripts), as well as a subset of NAnt in a subdirectory of our installation (so that we don’t rely on NAnt being present on the target server.)  It also includes a few convenience batch files referenced by our Start menu entries for invoking NAnt with specific targets.

The BizTalkSample.Setup project in the build file can be built, installed, and deployed to try this out.  (You will want to make sure you haven't already deployed in "developer" mode, because the "Deploy BizTalk Deployment Sample" passes a flag to the NAnt script that will skip the undeploy phase in order to speed up "clean" installations.)  Note that as your deployment grows more sophisticated - multiple hosts, servers, etc. - you will find the NAnt script requires additional properties to govern these variations between server deployments and developer-desktop deployments.

The build template sample also discusses the use of NUnit as a post-deployment verification process.  See the documentation in the sample for a full discussion --  basically all we are doing is making sure our MSI includes a subset of our unit tests, NUnit, and appropriate test files.  This is then wired up to the "Verify Deployment" Start menu option.  This can be an extremely effective means of ensuring your operations team has a means of testing the system interactively (not to replace standard automated heartbeat tests you might have…)

Peter Provost proposed a great generic approach to building BizTalk NAnt files a couple of weeks ago - and I shamelessly stole his approach to reverse the "orchestration ordering" property (very slick!)  If you do not wish to pursue the MSI option just discussed, and you do not wish to integrate the "out of band" setup actions discussed above (i.e. virtual directory creation, etc.), you might prefer his approach or you might want to push the build template sample provided here in that direction (i.e. more property driven.)  However, I believe tackling the kinds of concerns that we do here will generally mean that a real-world project will no longer have a generic-looking NAnt script. 

Again, you can download the build template sample here.  Enjoy, and I hope you find it useful.

A lot has been said regarding the MSMQT adapter for BizTalk 2004 already, but below are a few recent observations that may be of help to you. 

When people ask me what MSMQT is, my short answer goes something like this: "MSMQT is the name of the BizTalk 2004 adapter that implements the MSMQ network protocol directly within BizTalk.  It allows BizTalk 2004 to send/receive MSMQ messages directly, and move messages to/from the MessageBox very quickly - without external (DTC) transaction coordination.  Only private queues are supported for receives."

Using the MSMQT adapter you can:

• Use Send Ports to send to public or private queues on remote machines - but the queues must be transactional.
• Use Send Ports to send to local queues, which will be private since that is all MSMQT supports.
• Use Receive Locations to receive from locally defined queues, which will be private since that is all MSMQT supports.  These queues can be either transactional or non-transactional.  You cannot monitor remote queues (i.e. receive locations can't reference remote queues.)

You cannot using the System.Messaging MSMQ APIs/COM MSMQ APIs/MSMQ C Libraries or the MSMQ MMC console on a BizTalk 2004 machine that is running the MSMQT adapter!  This is because these all rely on the native MSMQ Service running.

You can use the aforementioned APIs (though not the MMC console) from remote machines (that are not running the MSMQT adapter) to put messages in MSMQT-defined queues.

To test MSMQT queues on the BizTalk 2004 server where they are defined, you can:

1. Create a Receive Port and a File Receive Location that will monitor a particular directory...
2. Create a static one-way Send Port that specifies MSMQT & the queue you want to test as the destination.  In the "Filters" portion of the Send Port configuration, define BTS.ReceivePortName = ReceivePortNameFromStep1

Now, when we you drop a file on your directory, it will be routed to your queue.  The receive location could also use HTTP posts, in which case the WFETCH tool from the IIS Resource Kit (or a similar tool) will let you easily get test messages to the queue.

Note: If you have existing applications that used to send to public MSMQ queues, but will now be sending to MSMQT queues, the most common problem is that they are referencing those queues via the PathName property, rather than the FormatName property (with 'DIRECT=OS:machine\private$\qname' syntax.)  This is only a problem for users of the COM/C-Library APIs - the System.Messaging.MessageQueue class actually allows format names to be used with the Path property.

See this Microsoft Support discussion for lots of additional insight into MSMQT.

There are several QFEs for BizTalk 2004 that you may want to know about, in case you encounter the situations described below.

Situation: You attempt to create or edit Receive or Send ports within the BizTalk Explorer in Visual Studio, and CPU consumption hits 100%.  Memory consumption climbs until the IDE crashes with an OutOfMemoryException.

Fix: Ask PSS for the hotfix associated with KB870619 (also known as hotfix 1185)

Situation: You have a CDATA section in an inbound xml document within an orchestration.  The CDATA section contains flat file data, with vital trailing (or leading) whitespace.  After you execute a transform in your orchestration, the CDATA designation is stripped (although the flat data is still there) - and the leading/trailing whitespace is now lost.  Curiously, using the "Test Map" feature (by right-clicking on a map in the solution explorer) doesn't exhibit this behavior.

Fix: Ask PSS for the hotfix associated with KB841563

Situation: You have a scope shape, and scope timeouts aren't working as expected.  Specifically, for blocking calls in an expression shape within the scope (like a DCOM call, etc.) where the timeout is being exceeded, you see 100% CPU consumption in the BizTalk service and the orchestration never terminates.

Fix: Ask PSS for the hotfix associated with KB811250.  (Note: this problem may have been addressed in the fix rollup released in April - I'm not sure what the ordering was here.)

(Update: See the latest on the Deployment Framework here.)

I realized today that in the "deploy.orchestrations" target of the NAnt build file (discussed here) there is a bug in the ordering of operations.  The original file imported the binding file prior to deploying the orchestration - which doesn't work for a "first time" deployment. 

In addition, Hermo Terblanche made a good point (in comments) regarding piplines which are used by Send/Receive ports - i.e. the ports need to be removed prior to removing/updating the corresponding pipeline assembly. 

Therefore, an additional dependency has been added to the "undeploy.piplines" target - namely, "remove.ports".  This target will remove exactly one Send and one Receive port, the names of which are derived from the binding file.  This section of the build file will have to be customized for your purposes, just like the names and ordering of orchestrations.  Note that in my sample, the orchestration uses “Specify Now” ports, and removing the Send/Receive ports is probably not necessary.  However, for “Specify Later” ports (the more typical case) that use custom pipelines, you will get the error referenced in Hermo's comment if you do not remove the ports prior to updating the pipeline assembly.

The zip file with a new build file is still located here.  Current NAnt output from the sample will look like this.  Enjoy!

(Update: See the latest on the Deployment Framework here.)

One of the more complicated aspects for a developer using BizTalk 2004 is the large number of steps required during the edit/run/debug cycle.  Since your BizTalk artifacts can't be run "in place" (that is, where they were compiled) but must instead be deployed to a local (or remote) BizTalk server, life can get quite complicated.

If you have done much BizTalk 2004 development, you know the routine quite well at this point.  If you have orchestrations, schemas, transforms, pipelines, and (say) C# components all partitioned into separate assemblies - and you have a number of orchestrations with "Call/Start Orchestration" shape dependencies that introduce specific start/stop-ordering - you can spend a lot of time doing the whole stop/unenlist/undeploy/redeploy/enlist/start/bounce BizTalk routine. 

Worse, you might not get it right - which can lead to hours spent debugging problems (over the course of your project) that don't really exist.

To alleviate this problem, it can be quite helpful to use a tool like NAnt to coordinate the "update my BizTalk server" process that must occur after each build.  (NAnt is a large topic - suffice to say it is an Xml-driven software build system.)  As long as your NAnt build file (and BizTalk bindings file) are kept up to date, the whole process can be reduced to:

1. Comple your solution (which might have multiple orchestrations, schemas, transforms, pipelines, and external components in separate assemblies)
2. Choose "Deploy to BizTalk" from your Tools menu.
3. Wait 60 seconds or so, enjoying the feeling that you have a consistent & reliable deployment mechanism.

In addition, you can of course use NAnt to kick off full unattended builds for nightly builds or continuous integration systems (like Cruise Control).  Since Visual Studio.NET is the only supported way to build BizTalk projects - despite that tempting XSharpP.exe file sitting in your BizTalk installation directory - this part of your NAnt build file must defer to calling devenv.exe in command-line fashion.  (A short "getting started" for NAnt & directions for using with a VS external tool can be found here.)

So, how about a sample that shows using NAnt to coordinate a reasonably complex deployment of inter-related BizTalk projects?  Available for download is a zip of a BizTalk solution that contains the following projects:

• BizTalkSample.Components (a C# project, with a class called from orchestration)
• BizTalkSample.Orchestrations (that contains two orchestrations, one which calls the other via a Call Orchestration shape)
• BizTalkSample.Pipelines (a send and receive pipeline, which are not needed but included to illustrate the deployment aspects)
• BizTalkSample.Schemas (which includes two schemas)
• BizTalkSample.Transforms (which contains a single transform used by one of the orchestrations)

The item I hope you will find most interesting, however, is a file called BizTalkSample.sln.build.  This is a NAnt build file that has a deployment target which captures the following dependency tree (only partially blown out here, for brevity - but this no doubt seems familiar to you if you've been using Biztalk 2004 for awhile…)

Deploy
   Deploy Schemas
      Undeploy Schemas
         Undeploy Orchestrations
            Unenlist Orchestrations
               Stop Orchestration
         Undeploy Transforms
            Undeploy Orchestrations
   Deploy Components
   Deploy Pipelines
      Undeploy Pipelines
         Undeploy Orchestrations
   Deploy Transforms
      Undeploy Transforms
         Undeploy Orchestrations
   Deploy Orchestrations
   Bounce BizTalk

Perhaps more illustrative is the output of the deployment itself - a sample of which can be seen here.  Note that the NAnt script relies heavily on the EnlistOrch.vbs and StopOrch.vbs script files that ship as samples with BizTalk 2004, as well as the BTSDeploy command line utility.

The BizTalkSample.sln.build script included in the download should represent a BizTalk project organization of sufficient complexity to act as a template for your own projects.  You will want to maintain the list (and correct ordering!) of the names of your orchestrations within the build targets for "deploy.orchestrations" and "unenlist.orchestrations" (this isn't deriveable automatically…)

Note that the build script relies on a set of naming conventions that will be evident once you spend some time with it - namely, that directory names correspond to assembly names.  In addition, the standard "Development" and "Deployment" configurations in BizTalk projects have been replaced with "Debug" and "Release", in order to not create inconsistencies with standard .NET projects (and allow for one "Configuration" property within the build script.)  This replacement was accomplished with a file-level search/replace.

There is probably room for more sophistication in this build file.  It takes a somewhat pessimistic view of what must happen for each edit/run/debug/deploy cycle, but I've found that despite the 60 seconds spent executing the script, your net productivity gain will be quite high given the time you won't waste trying to figure out what aspect of your deployment isn't correct.  Leave comments with suggestions for improvements, if you like.