Andy Morrison wrote about a technique for identifying the correct host instance to attach to when debugging components associated with orchestrations.
A potentially easier solution is to keep perfmon running with the counter shown below (BizTalk:Messaging/ID Process - all instances.) The counter values will update every time a host instance recycles. The Debug-Processes dialog in Visual Studio will show you (and allow you to sort by) process IDs to make this easy.
Then put PerfMon in "report view" to easily see process IDs by host instance:
There have been several folks who have discussed how to create messages "from
scratch" within an Orchestration context - you can read
Matt's thoughts and check out a BizTalk
documentation excerpt. This is a common question, and I had an
additional technique I thought I would share...
Background: When you have a schema that has many promoted properties (or
distinguished fields), or many elements that can be set via xpath expressions
easily, it can be useful to simply start with a "template" document instance
and populate the element content that you are interested in.
In this situation, you will often have a "Message Assignment" shape that looks
something like this:
xmlDoc.LoadXml("<ns0:BizTalkSampleS3
xmlns:ns0="http://BizTalkSample.Schemas.BizTalkSampleS3">
<SomeElement></SomeElement></ns0:BizTalkSampleS3>");
someMsg = xmlDoc;
someMsg.SomeElement = "some content";
// (or xpath(someMsg,someElementXPath) = "some content" if we don't have a
// distinguished field.)
One disadvantage of loading up "template" xml documents from either expression
shapes or code (via XmlDocument.LoadXml) is that those xml fragments can get
easily "lost", and are hard to update early in the development cycle when
schemas may still be in flux. Loading the template files from the file
system is problematic because the question arises "where should I store these
files, so that I can find them in any environment I deploy to?" (Solvable, but
a hassle.)
Instead, why not embed the template xml documents as assembly resources?
For those unfamiliar with that process, I have a short tutorial here (& a
helper class.)
-
You will need a C# project as part of your overall BizTalk solution.
Place your template xml file(s) in the directory corresponding to this
project, and add them as an "existing item" to the project.
-
Select this file within the Solution Explorer, and within the Properties
window, select "Embedded Resource" as the "Build Action" as shown here:
-
Place this
class (text
here) within the same C# project that houses the resources you have
added.
-
To construct a message, drag out a "Message Assignment" shape, and within the
associated expression write some code like the following. Simply pass the
file name of the template document as an argument to GetXmlDocResource (or
GetStringResource.)
sampleUsingTemplate =
BizTalkSample.Components.EmbeddedResourceCache.
GetXmlDocResource("BizTalkSampleS3_output.xml");
// Populate the "rest" of the message with distinguished fields, promoted
// properties, xpath expressions, etc.
sampleUsingTemplate.SomeElement = "foo";
The class I have supplied will cache the loaded resources in a hashtable for
performance sake, and allow you to load resources as both strings and
XmlDocuments.
A last thought: Many people ask, "Why can't I just create a message using a new
operator or a default constructor of some sort?" Well, because few XSD schemas
sufficiently constrain the set of valid instance documents enough for that to
be useful - what form would a "default message" take? (Would it have the
optional elements you need? Some elements that you don't want?)
Enjoy - feedback appreciated!
Just a few notes on what I'll be up to in the next month or two...
Do you use the Deployment Framework for BizTalk? Or just have general questions on automating BizTalk deployments? You can attend my session at TechEd 2005 - Friday June 10th, 10:45am to 12:00pm in the BPI Cabana, where I'll be doing a talk on those topics...
I'll also be speaking at the Microsoft DevCon event on my "other favorite" topic, Windows Mobile. The focus will be on what is new for Visual Studio 2005 in the mobile arena...The conference will also be covering the new Team System offerings, ASP.NET 2.0, and quite a few other VS2005 topics. Check it out...
(Update: See the latest on the Deployment Framework
here.)
Well, its time for another revision of the BizTalk/NAnt deployment tools...
For folks recently joining us, these tools allow you to quickly create both developer workstation deployments (inside Visual Studio) and MSI-based deployments (for servers) for BizTalk 2004 solutions. Just about all aspects of BizTalk deployments are handled, including items such as virtual directories, SOAP proxies, and WinXP/2k/2003 differences.
I get quite a few emails from people who are using these tools (which is great!) A couple folks have suggested that I start referring to the whole thing as a" p framework?. < deployment>
That sounds about right - like all frameworks, there is a core set of functionality that takes care of quite a bit of work, and a given project can supply what needs customization or what is missing.
In this case, the core NAnt scripts, custom NAnt tasks, WiX generation, and other custom tools form the framework itself - and a given BizTalk project can supply project-specific details, what needs to be customized, and any missing functionality. So I'll refer to this work as the "Deployment Framework for BizTalk" from now on...
What has changed in this rev?
- When last posting on this topic, I should have been even clearer that as of the last release, the deployment framework requires NAnt .85.
- The biggest improvement is the ability to use binding files as they are emitted by the BizTalk Deployment Wizard directly. In the past, the framework had insisted that binding files be split between orchestration bindings and messaging bindings - this is no longer required. I had several requests for this fix...Also, note that having multiple binding files is supported (like one for each of several assemblies.) Now, if you wish to continue to use the "old style" split bindings, define the orchBindings property in your project NAnt file as shown below:
<property name="orchBindings" value="${nant.project.name}.OrchBindings.xml" overwrite="false" />
- You can now set the value of NAnt properties from the SettingsFileGenerator spreadsheet. Recall that this spreadsheet is used (in conjunction with XmlPreProcess) to manage environment-relative configuration - that is, to manage those values that vary depending on whether you are in a development, QA, or production environment. Until now, it has been difficult to use those configuration values within the NAnt deployment file! You typically just used them in your binding files or other application-level XML files. Now, however, you can define a property as shown below to make these values available as NAnt properties. A good example usage is the case where you want the actual names of the SSO Application Users group and SSO Administrators group to be defined in the SettingsFileGenerator spreadsheet (because the group names may vary by environment you deploy to.) Just define a property in your project NAnt file called "propsFromEnvSettings" with a comma-separated list of values you want pulled in. (The sample has been updated to show this, too.)
<property name="propsFromEnvSettings" value="ssoAppUserGroup,ssoAppAdminGroup"/>
- Many people had asked for the DeployResults.txt file (which is created in the DeployResults subdirectory during a deployment) to be available in a history log, so that it would be easy to walk up to a production server and determine what deploy/undeploy activity had taken place (especially useful for diagnosing any failures.) To that end, the deployment framework will preserve past DeployResult.txt files with this convention: `DeployResults TRACESATURN_05032005_0922.txt' Note that the machine name, date, and time are preserved - so you can gather these files across all the servers in a BizTalk group, if need be.
- SSOSettingsFileReader class improvements: In the last release, I introduced the ability to manage settings within the BizTalk SSO using the SettingsFileGenerator spreadsheet that is also used for managing environment-relative configuration. Part of that functionality included a class called SSOSettingsFileReader, used to retrieve settings at runtime. This class now has some strongly typed accessors, and no longer sits inside a lengthy namespace, which means that in an orchestration expression shape you can simply do:
someVar = SSOSettingsFileReader.ReadString("BizTalkSample","SomeAppConfigItem") // or ReadInt32
- Ben Cops had found that on large-scale deployments, it was useful to restrict which orchestrations were deployed/started to speed the development cycle. So, he modified the orchestration NAnt task accordingly. To use (if you need to), just set the orchestrationsToIgnore property in your project NAnt file with a comma-separated list that reflects what you are currently working on. Make sure to copy BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory again! (you'll find it in the DeployTools directory in the download.)
- A bug was fixed with undeployment for the case where an SSO affiliate app had already been deleted...
- All references in NAnt files to xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" were removed, because they were simply causing heartache (for many reasons.) Remove this namespace in your project NAnt file when upgrading.
- Rather than trying to keep the source for the C# tools (NAnt tasks, SetEnvUI, DeployBTRules, SSOSettingsFileImport\SSOSettingsFileReader, etc.) current within the GotDotNet workspace, I'm making them available as a zip.
Remember, you do not need to piece together the current functionality set from past blog entries - the complete documentation (in Word form) is in the download and separately here.
Some last thoughts:
- Some folks have been reluctant to use the deployment framework because of its reliance on NAnt, and a worry over using NAnt in a server environment. One thing to remember (that might allay this concern) is that a subset of NAnt is delivered with the generated MSI, and NAnt will not require a separate installation procedure by your production operations team. Moreover, NAnt doesn't create any kind of persistent service or footprint on the production machine (i.e. no automatic file associations that would cause scripts to run with a double-click or anything like that...)
- Add a Visual Studio External Tool for the updateOrchestration target if you haven't already! This can speed your development cycle significantly. See the tools zip for a script that creates all relevant external tools.
- When using the WiX-based MSI generation, make sure your project NAnt file (for WiX generation, like BizTalkSample.WiXSetup.Build) is choosing "debug" or "release" based on what you want to put in the MSI. See comments in that file for more detail, and Jon Flanders' post on this topic.
How do I upgrade?
- With a safe copy of your BizTalk solution, expand the "Core" zip contents right on your project directory, such that the core scripts will overwrite what you have now (but your project-specific scripts will be preserved.)
- Manually copy the WiX-related updates to your *.WiXSetup directory, since they will get unzipped to BizTalkSample.WiXSetup
- Remove xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" from your project NAnt file
- Copy DeplyTools\BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory
- Do a deployment on a developer workstation and a server to test things out...
Downloading
- You can always use the download links on this blog.
- Full sample here, core scripts here, extra tools here.
(But always check the download links to see if there has been an update…)
- Standalone documentation here (and diagram here.)
- GotDotNet workspace here.
(Update: See the latest on the Deployment Framework
here.)
In the latest release of the Deployment Framework for BizTalk “extra tools”, you will find a script that will automate the creation of “external tools” in Visual Studio that are useful for working with BizTalk 2004 and the Deployment Framework. Modify the script to include all the tools that your team relies on, too.
The idea that this should be automated came from Ben Cops (though I’ve heard the complaint before!) Manually creating all the useful external tools was somewhat painful for a large team of BizTalk developers.
After you run the script, your Tools menu in Visual Studio should include entries for deploying and un-deploying the current solution, as well as for updating just the orchestrations, .NET components, and SSO configuration data (one menu entry.) In addition, there will be entries for running NAnt on a highlighted target name (useful if you have your build script open), and for running HAT and the Subscription Viewer. Enjoy…
A short while ago, I did a presentation for the Twin Cities .NET user group on log4net. (You might recall an earlier blog entry where I discussed using log4net with BizTalk 2004...)
The presentation was not specific to BizTalk 2004 - instead, it attempts to describe why I think the log4net library is so very well thought out. You can grab it here, if you like.
The last slide references Loren Halvorsen's comparison of log4net and the new Microsoft Enterprise Library Logging Block, which I would recommend taking a look at (the comparison, that is.) Tom Hollander (among many others!) later weighed in with this piece.
Lastly, 1.2.9 beta of log4net has recently become available (release notes here)....However, my extensions to log4net have not yet been updated to reflect this new drop. I'll be sure to post when I'm able to update.
(Update: See the latest on the Deployment Framework
here.)
Just when you thought automated deployments for BizTalk 2004 were safe, another
update to the “Deploy with NAnt” template comes along…
:)
This is a very worthwhile "upgrade" to consider taking advantage of. One
of the big items is support for "configuration info within SSO", but a complete
list of features/fixes, etc. is below.
SSO Integration: If you are currently using the
SettingsFileGenerator.xls spreadsheet (discussed in previous entries) in
conjunction with the XmlPreProcess
tool to handle "environment-relative" configuration - that is, to handle
variations between your dev/QA/production environments - you will be glad to
know you can get more leverage out of that process now…
To review, with this process, you use the names that are defined in the first
column of the SettingsFileGenerator spreadsheet (shown below) as "tokens"
within your binding file, as in:
<!-- ifdef ${ xml preprocess} -->
<!-- <Address>${FileSendLocation}\%MessageID%.xml</Address>
-->
<!-- else -->
<Address>C:\temp\BizTalkSample OutDir\%MessageID%.xml</Address>
<!-- endif -->
At install time, the value for FileSendLocation that is appropriate to the
environment you are deploying to is "plugged in" by
XmlPreProcess (but for developers, the file remains legal XML.)
The spreadsheet might look as follows:
(click)
Now, however, you can also put in name-value pairs (whether they
vary by environment or not) into the spreadsheet that you need access to at run
time (i.e. general configuration information.)
The name-value pairs will be deployed into a newly created affiliate
application within the SSO, and can be read using the SSOSettingsFileReader
class (which serves as a cache, too – and is provided in the download.)
The static methods on this class can be used from an orchestration
expression shape, or from assemblies that orchestrations call – as in
string test =
SSOSettingsFileReader.ReadValue("BizTalkSample","SomeAppConfigItem"); (By
the way, the “nested names” shown in the spreadsheet above are just
meant to suggest a partitioning mechanism you might want to use, rather than
one flat “namespace” of config data.)
Of course, the deployment scripts handle all the interaction with the SSO
database for deployments/un-deployments. This whole mechanism
(SettingsFileGenerator.xls, XmlPreProcess, and now SSOSettingsFileImport.exe)
is worth using even if you don’t use the rest of this script
infrastructure. Note that Jon Flanders had a
great article on SSO-based configuration using a strongly-typed
approach, where as this approach is loosely-typed, but leverages a mechanism
you might already be using (i.e. the spreadsheet concept.) (Note:
Jon’s code for creating/deleting affiliate apps helped me complete the
SSOSettingsFileImport.exe utility quickly…)
Other features/fixes:
-
Support for multiple schema assemblies, including the case where schema
assemblies reference each other due to schema imports. (Undeployments
occur in reverse order of deployments.)
-
Support for multiple orchestration assemblies, including the case where
orchestration assemblies reference each other due to the use of Call/Start
Orchestration shapes. (Undeployments occur in reverse order of
deployments.)
-
Support for multiple orchestration and port binding files (just list them as
comma separated values in the orchBindings/portBindings properties.)
-
Fixed bug that occurred when undeploying ISAPI extensions in Windows 2003 / IIS
6.
-
Fixed bug in undeploying pipeline components (thanks to John Adams)
-
Added /NOFORCE flag to IISRESET calls in accordance with
http://support.microsoft.com/kb/286196
-
New property "includeInstallUtilForComponents" was added, which can be set to
'true' in your project-specific nant file. This will cause your component
assemblies to be called with "installutil.exe" (with the /u flag for
undeployment.) This is useful if you have .NET Installer-derived classes
that need to get called for creating event sources, perf counters, etc.
(Yes, you could certainly argue these should be called by the WiX piece of the
script infrastructure instead, but this method works and is more in keeping
with the spirit of the entire process.) Note there is no harm if a given
component assembly listed in your components property does not have an
installer class.
-
Eliminated the need for the includeCustomTarget property, and provided
additional ways to supply custom functionality. Project-specific nant
file can now supply any or all of customDeployTarget, customPostDeployTarget,
customUndeployTarget, customPostUndeployTarget -- and they will be called if
they are present.
-
Integrated the deployment of log4net-specific stuff, if the includelog4net
property is set to true. See the work on
BizTalk/log4net integration for more information. (Log4net sample
usage is included in the new sample.)
-
Now using NAnt version .85 - so be sure to grab the latest
NAnt/NAntContrib
since the nant scripts are using .85-specific syntax to avoid "deprecated"
warnings that were occurring otherwise. Note that even once you have
installed NAnt .85, you will still be able to deploy BizTalk projects that are
using old versions of the deployment scripts. Note also that this is a
pretty basic port to .85 - there are no doubt more elegant ways of doing lots
of things that I haven't investigated yet, now that nant has expression
support.
Other notes
The scripts like to see BizTalk projects using Debug and Release for target
names. A long while back I had suggested a file-level search/replace
within btproj files to change these, but two more elegant options are
available:
-
In project properties, change the output location for the
“Development” configurations to “bin\debug” and output
location of “Deployment” configurations to
“bin\release”.
-
(Via Puneet Gupta): Change the template for BizTalk projects, found in
BTSApp.btproj file in %BizTalkInstallDir%\Developer
Tools\BizTalkWizards\BTSApp\Templates\1033, to reflect Debug and Release
targets.
Another note: Lots of folks have noticed that XmlPreProcess can be used for a
lot more than port binding files! Log4net config files, orchestration
binding files, you name it. See the full Word documentation in the
download for an example of how to do this within customDeployTarget.
How do I upgrade?
-
With a safe copy of your BizTalk solution, expand the “Core” zip
contents right on your project directory, such that the core scripts will
overwrite what you have now (but your project-specific scripts will be
preserved.)
-
Manually copy the WiX-related updates to your *.WiXSetup directory, since they
will get unzipped to BizTalkSample.WiXSetup
-
Do a deployment on a developer workstation and a server to test things
out…
Downloading
-
You can always use the right-hand links on this blog.
-
Full sample
here, core scripts
here
-
GotDotNet workspace
here. Because of issues with GotDotNet, I’m only going to be
keeping the EXE utility source code up to date – not the entire sample
& core tree.
Enjoy – comments always welcome and appreciated.
Steve and I wound up working on the same problem at the same time (probably for the same person…)
When working with the MSMQ adapter, keep in mind that you must reference the Microsoft.BizTalk.Adapter.MSMQ.MsmqAdapterProperties.dll to have access to MSMQ-specific properties. (The list of properties available is in the adapter documentation.)
Why? The intellisense in the expression shape (when using the parentheses syntax on messages, ports, etc.) is looking for classes derived from Microsoft.XLANGs.BaseTypes.PropertyBase to present in the drop down list. Some of those classes are part of your "native" BizTalk installation, some are provided by add-on adapters, and (of course) some are provided by property schemas that you develop. A reference to the containing assembly is necessary to find them.
Other things to note relative to the MSMQ adapter:
- For dynamic send ports, the syntax can look like: SomePort(Microsoft.XLANGs.BaseTypes.Address) = @"MSMQ://FORMATNAME:DIRECT=OS:SOMEMACHINE\PRIVATE$\SOMEQ";
- In the case of dynamic send ports, the runtime will use non-transactional sends by default. If you want transactional sends, you will need to set the transactional property: outboundMsg(MSMQ.Transactional) = true;
- Transactional messages which fail to deliver to the remote queue will be found in the local Transactional Dead Letter queue. No error will be raised by the adapter. This may mean that your design requires acknowledgement messages to achieve what you are looking for.
I was just reading Mike Holdorf's post on the bts_CleanupMsgbox stored procedure. The team I work with has been able to make good use of this sproc as well, especially while stress testing.
One thing to beware of is this: Your habit for doing this kind of thing might lead you to open up Sql Query Analyzer, and right click on the stored procedure - selecting “script to new window as execute.” This will generate the following:
DECLARE @RC int
DECLARE @fLeaveActSubs int
-- Set parameter values
EXEC @RC = [BizTalkMsgBoxDb].[dbo].[bts_CleanupMsgbox] @fLeaveActSubs
Ahhh, but the stored procedure actually defaults fLeaveActSubs to “1”, whereas this code will leave fLeaveActSubs as “0”. This will cause all your subscriptions to be deleted, and you will have to redeploy your BizTalk applications. Not that I would know....
So just do “exec bts_CleanupMsgbox”...
|