(Update: See the latest on the Deployment Framework
here.)
Just when you thought automated deployments for BizTalk 2004 were safe, another
update to the “Deploy with NAnt” template comes along…
:)
This is a very worthwhile "upgrade" to consider taking advantage of. One
of the big items is support for "configuration info within SSO", but a complete
list of features/fixes, etc. is below.
SSO Integration: If you are currently using the
SettingsFileGenerator.xls spreadsheet (discussed in previous entries) in
conjunction with the XmlPreProcess
tool to handle "environment-relative" configuration - that is, to handle
variations between your dev/QA/production environments - you will be glad to
know you can get more leverage out of that process now…
To review, with this process, you use the names that are defined in the first
column of the SettingsFileGenerator spreadsheet (shown below) as "tokens"
within your binding file, as in:
<!-- ifdef ${ xml preprocess} -->
<!-- <Address>${FileSendLocation}\%MessageID%.xml</Address>
-->
<!-- else -->
<Address>C:\temp\BizTalkSample OutDir\%MessageID%.xml</Address>
<!-- endif -->
At install time, the value for FileSendLocation that is appropriate to the
environment you are deploying to is "plugged in" by
XmlPreProcess (but for developers, the file remains legal XML.)
The spreadsheet might look as follows:
(click)
Now, however, you can also put in name-value pairs (whether they
vary by environment or not) into the spreadsheet that you need access to at run
time (i.e. general configuration information.)
The name-value pairs will be deployed into a newly created affiliate
application within the SSO, and can be read using the SSOSettingsFileReader
class (which serves as a cache, too – and is provided in the download.)
The static methods on this class can be used from an orchestration
expression shape, or from assemblies that orchestrations call – as in
string test =
SSOSettingsFileReader.ReadValue("BizTalkSample","SomeAppConfigItem"); (By
the way, the “nested names” shown in the spreadsheet above are just
meant to suggest a partitioning mechanism you might want to use, rather than
one flat “namespace” of config data.)
Of course, the deployment scripts handle all the interaction with the SSO
database for deployments/un-deployments. This whole mechanism
(SettingsFileGenerator.xls, XmlPreProcess, and now SSOSettingsFileImport.exe)
is worth using even if you don’t use the rest of this script
infrastructure. Note that Jon Flanders had a
great article on SSO-based configuration using a strongly-typed
approach, where as this approach is loosely-typed, but leverages a mechanism
you might already be using (i.e. the spreadsheet concept.) (Note:
Jon’s code for creating/deleting affiliate apps helped me complete the
SSOSettingsFileImport.exe utility quickly…)
Other features/fixes:
-
Support for multiple schema assemblies, including the case where schema
assemblies reference each other due to schema imports. (Undeployments
occur in reverse order of deployments.)
-
Support for multiple orchestration assemblies, including the case where
orchestration assemblies reference each other due to the use of Call/Start
Orchestration shapes. (Undeployments occur in reverse order of
deployments.)
-
Support for multiple orchestration and port binding files (just list them as
comma separated values in the orchBindings/portBindings properties.)
-
Fixed bug that occurred when undeploying ISAPI extensions in Windows 2003 / IIS
6.
-
Fixed bug in undeploying pipeline components (thanks to John Adams)
-
Added /NOFORCE flag to IISRESET calls in accordance with
http://support.microsoft.com/kb/286196
-
New property "includeInstallUtilForComponents" was added, which can be set to
'true' in your project-specific nant file. This will cause your component
assemblies to be called with "installutil.exe" (with the /u flag for
undeployment.) This is useful if you have .NET Installer-derived classes
that need to get called for creating event sources, perf counters, etc.
(Yes, you could certainly argue these should be called by the WiX piece of the
script infrastructure instead, but this method works and is more in keeping
with the spirit of the entire process.) Note there is no harm if a given
component assembly listed in your components property does not have an
installer class.
-
Eliminated the need for the includeCustomTarget property, and provided
additional ways to supply custom functionality. Project-specific nant
file can now supply any or all of customDeployTarget, customPostDeployTarget,
customUndeployTarget, customPostUndeployTarget -- and they will be called if
they are present.
-
Integrated the deployment of log4net-specific stuff, if the includelog4net
property is set to true. See the work on
BizTalk/log4net integration for more information. (Log4net sample
usage is included in the new sample.)
-
Now using NAnt version .85 - so be sure to grab the latest
NAnt/NAntContrib
since the nant scripts are using .85-specific syntax to avoid "deprecated"
warnings that were occurring otherwise. Note that even once you have
installed NAnt .85, you will still be able to deploy BizTalk projects that are
using old versions of the deployment scripts. Note also that this is a
pretty basic port to .85 - there are no doubt more elegant ways of doing lots
of things that I haven't investigated yet, now that nant has expression
support.
Other notes
The scripts like to see BizTalk projects using Debug and Release for target
names. A long while back I had suggested a file-level search/replace
within btproj files to change these, but two more elegant options are
available:
-
In project properties, change the output location for the
“Development” configurations to “bin\debug” and output
location of “Deployment” configurations to
“bin\release”.
-
(Via Puneet Gupta): Change the template for BizTalk projects, found in
BTSApp.btproj file in %BizTalkInstallDir%\Developer
Tools\BizTalkWizards\BTSApp\Templates\1033, to reflect Debug and Release
targets.
Another note: Lots of folks have noticed that XmlPreProcess can be used for a
lot more than port binding files! Log4net config files, orchestration
binding files, you name it. See the full Word documentation in the
download for an example of how to do this within customDeployTarget.
How do I upgrade?
-
With a safe copy of your BizTalk solution, expand the “Core” zip
contents right on your project directory, such that the core scripts will
overwrite what you have now (but your project-specific scripts will be
preserved.)
-
Manually copy the WiX-related updates to your *.WiXSetup directory, since they
will get unzipped to BizTalkSample.WiXSetup
-
Do a deployment on a developer workstation and a server to test things
out…
Downloading
-
You can always use the right-hand links on this blog.
-
Full sample
here, core scripts
here
-
GotDotNet workspace
here. Because of issues with GotDotNet, I’m only going to be
keeping the EXE utility source code up to date – not the entire sample
& core tree.
Enjoy – comments always welcome and appreciated.
Steve and I wound up working on the same problem at the same time (probably for the same person…)
When working with the MSMQ adapter, keep in mind that you must reference the Microsoft.BizTalk.Adapter.MSMQ.MsmqAdapterProperties.dll to have access to MSMQ-specific properties. (The list of properties available is in the adapter documentation.)
Why? The intellisense in the expression shape (when using the parentheses syntax on messages, ports, etc.) is looking for classes derived from Microsoft.XLANGs.BaseTypes.PropertyBase to present in the drop down list. Some of those classes are part of your "native" BizTalk installation, some are provided by add-on adapters, and (of course) some are provided by property schemas that you develop. A reference to the containing assembly is necessary to find them.
Other things to note relative to the MSMQ adapter:
- For dynamic send ports, the syntax can look like: SomePort(Microsoft.XLANGs.BaseTypes.Address) = @"MSMQ://FORMATNAME:DIRECT=OS:SOMEMACHINE\PRIVATE$\SOMEQ";
- In the case of dynamic send ports, the runtime will use non-transactional sends by default. If you want transactional sends, you will need to set the transactional property: outboundMsg(MSMQ.Transactional) = true;
- Transactional messages which fail to deliver to the remote queue will be found in the local Transactional Dead Letter queue. No error will be raised by the adapter. This may mean that your design requires acknowledgement messages to achieve what you are looking for.
I was just reading Mike Holdorf's post on the bts_CleanupMsgbox stored procedure. The team I work with has been able to make good use of this sproc as well, especially while stress testing.
One thing to beware of is this: Your habit for doing this kind of thing might lead you to open up Sql Query Analyzer, and right click on the stored procedure - selecting “script to new window as execute.” This will generate the following:
DECLARE @RC int
DECLARE @fLeaveActSubs int
-- Set parameter values
EXEC @RC = [BizTalkMsgBoxDb].[dbo].[bts_CleanupMsgbox] @fLeaveActSubs
Ahhh, but the stored procedure actually defaults fLeaveActSubs to “1”, whereas this code will leave fLeaveActSubs as “0”. This will cause all your subscriptions to be deleted, and you will have to redeploy your BizTalk applications. Not that I would know....
So just do “exec bts_CleanupMsgbox”...
(Note - this work has been updated for log4net 1.2.9. See the 'Tools' download in the Deployment Framework)
Early on when working with BizTalk 2004, it might be tempting to view the
ability to track orchestration events (and the use of the orchestration
debugger) as a substitute for traditional diagnostic logging.;
After all, the orchestration debugger can tell you exactly how far you got in
an orchestration, and what path through the workflow you took to get there
However, consider these limitations:
-
The orchestration debugger is used either a) after the orchestration has
completed or b) in conjunction with a breakpoint. Neither of these
choices is ideal, and only the latter gives you visibility into the
intermediate values of variables/messages.
-
Once an orchestration involves looping constructs, multiple orchestrations
connected via messaging, etc. the orchestration debugger often loses quite a
bit of diagnostic value.
Given this, it can be useful to integrate diagnostic logging into your
orchestrations and the components they call. If we are going to invest in
this effort, we’d like to use something richer than the built-in
System.Diagnostics.Trace/Debug infrastructure.
Log4net is an Apache-sponsored
initiative within the “Apache Logging
Services” project. It provides a rich diagnostic infrastructure
for .NET, with support for hierarchical logging and configuration, multiple
logging targets, and support for logging context.
A recent
article outlined the advantages of log4net over Microsoft’s Enterprise
Instrumentation Framework (EIF), though Microsoft/Avanade will be revamping
this logging infrastructure with the release of the
Enterprise Template Library
Given that the log4net initiative has broad support at the API layer (i.e.
log4j, log4PLSQL, etc.) and platform layer (including .NET CF) as well as the
benefit of maturity, log4net seems like a very reasonable choice for
instrumenting BizTalk 2004 applications. (The Enterprise Template Library
logging component will likely be a good choice as well when released.)
Following is a discussion of log4net itself, along with what is required to use
it in a BizTalk 2004 setting.
Quick Insight into log4net
Without diving immediately into the log4net API, let’s get a sense for what it
can do. What do we mean by hierarchical logging and configuration with
multiple target support?
Suppose we have classes with these namespace-qualified names:
MyCompany.MyDepartment.MyClassA
MyCompany.MyDepartment.MyClassB
Each of these classes can obtain a logger specific to their class by declaring
a static member as such:
private static readonly ILog log = LogManager.GetLogger(typeof(Foo));
Each class (MyClassA/MyClassB) can now emit trace with statements such as log.Info(“hello
world”)> or log.Warn(“a potential problem…”).
Default log levels include: Debug, Info, Warn, Error, and Fatal (though these
can be expanded.)
The concept of “context” can be added to a stream of trace messages via
NDC.Push(“some context”)/NDC.Pop or MDC.Set(“somekey”,”somevalue”).
(These class names are short for “Nested Diagnostic Context” and “Mapped
Diagnostic Context” respectively.) These can be extremely useful for
multithreaded/multi-user applications, to distinguish the streams of related
trace messages from each other. This context can be optionally formatted
into the associated tracing statements, as will be shown shortly.
If we want to turn on tracing at the ‘Error’ level by default, at the ‘Warn’
level for MyCompany, and at the ‘Debug’ level for MyClassA, we would simply
configure as follows (notice the use of the namespace hierarchy):
<root>
<level value="ERROR" />
</root>
<logger name="MyCompany">
<level value="WARN" />
</logger>
<logger name="MyCompany.MyDepartment.MyClassA">
<level value="DEBUG" />
</logger>
This can be changed on-demand simply by changing the configuration file, which
can be reloaded automatically without restarting the application. This
allows us to a) not be blinded by too much trace information and b) not impose
undue server load by tracing, whereas a non-hierarchical approach only allows
for one logging level for an entire application.
Suppose it is desired to route all trace information (by default) to
OutputDebugString, where a utility such as dbmon.exe or
DebugView can be used to view it. Suppose further it is desired to
prefix our output with the date, thread Id, logging level, the name of the
logger (i.e. MyCompany.MyDepartment.MyClassA), and the current context.
To do this with log4net, we would have our configuration file declare an
“appender” of the appropriate type, and include a declaration of a “layout” as
follows. The “appender” is then referenced by a particular logger (or the
root logger).
<appender name="OutputDebugStringAppender"
type="log4net.Appender.OutputDebugStringAppender" >
<layout type="log4net.Layout.PatternLayout">
<!-- The format specifiers let us add a wide
variety of additional info -->
<param name="ConversionPattern" value="%d [%t]
%-5p %c [%x] - %m%n" />
</layout>
</appender>
<root>
<level value="ERROR" />
<appender-ref ref="OutputDebugStringAppender" />
</root>
This might yield trace output as follows:
16:39:33,022 [316] INFO MyCompany.MyDepartment.MyClassA
[SomeContext] - Hello World
Again, the specifics of what additional information can be included in trace
output can all be configured dynamically at runtime. Note that other
useful information such as method name, file name, line number, caller’s window
identity can all be added with the PatternLayout class as well, though it is
quite expensive (further adding value to the hierarchy concept!)
Moreover, each element of the hierarchy can be directed to a different
appender, or to multiple appenders. Built in appenders include support
for ADO.NET, .NET remoting, SMTP, files, rolling files, buffering, and the
Event Log (among others.) Because the Event Log is supported, there is no
need to use a separate API to support such logging.
Core log4net Concepts
Without attempting to present a full tutorial (see
this introduction for a more complete discussion), the core concepts in
log4net include:
Loggers: Use by code to emit trace output at various levels.
-
Loggers are organized into hierarchies just like .NET namespaces (they don't
have to correspond, but often should.) Example: System is parent of
System.Text, and an ancestor of System.Text.StringBuilder.
-
It is typical to have the logger name equal to a fully-qualified class
name.
-
Loggers are retrieved using a static method of LogManager, e.g.
LogManager.GetLogger(someStringOrTypeName);
-
An ILog interface is returned with Debug, Info, Warn, Error, Fatal methods on
it for tracing.
-
Logging level is generally inherited from the hierarchy, and be configured
anywhere in the hierarchy.
-
Rule: logging level is equal to first non-null value working UP the tree.
-
Predefined levels: DEBUG, INFO, WARN, ERROR, FATAL, ALL, OFF
-
Default level for root in hierarchy is DEBUG
Appenders: An output destination for logging.
-
Built in: event log, asp.net trace, console, file, rolling file, smtp, etc.
-
Appenders can optionally have "layouts" to specialize how formatting is done
and add additional information (windows identity, source code info, context,
thread id, timestamp, etc.)
-
Appenders are attached to a logger. Log requests will go to a given
logger's appender, as well as all appenders attached up the hierarchy (unless
"additivity=false" at some level, in which case moving up hierarchy
stops…)
Filters: A given appender can apply a filter such that only log requests
that match some criteria will actually be output. Filters are applied
sequentially. Built-in filters include string match and regex
match.
Configurators: Classes which initialize the log4net environment.
Configuration is done either through assembly attributes that specify a config
file, or an explicit call to specify a config file. The config file can
be a standard .net config file, or standalone. Configuration is by
default at the appdomain level, but can be finer-grained through the use of
log4net "logging domains".
Using log4net with BizTalk 2004
There are a few challenges to using log4net in a BizTalk 2004
environment. They can be summarized as follows:
-
Assembly-attribute-driven configuration won’t work because BizTalk 2004 does
not appear to support the addition of custom attributes for BizTalk
assemblies. (And even if it did, this method would pose issues for
BizTalk environments.)
-
Using the log4net configuration classes for configuration is problematic
because there is no clear point at which to make the call. (How does as
an orchestration know if it is “first” and should initialize? How does it
know at any
point in an orchestration whether the BizTalk service has been recently
recycled, losing the log4net configuration?) Moreover, there is not a
good way to identify where the configuration file should be located.
-
ILog-derived classes (log4net loggers) are not serializable “out-of-the-box”,
making it difficult to use them in an orchestration setting.
-
Log4net context-storage mechanisms are thread-relative, which isn’t
workable for orchestrations, since orchestrations often a) dehydrate and then
re-hydrate on a different thread and b) make use of parallel branches.
We’d like to correctly associate context such as the orchestration instance ID
on a logger that is scoped to an orchestration.
To address these challenges, I’ve created
extensions to log4net that attempt to stay within log4net’s design
intent. These are housed in an assembly called
“log4net.Ext.Serializable”. (Update – these are now
currently found in the “Extra Tools” download for the Deployment Framework –
see the download links at the blog home
page.)
The main classes defined are as follows:
SLog: A serializable implementation of the ILog interface, to be used by
orchestrations (or other components) for logging. This class has an
InstanceId property as well as a general properties collection that is carried
with the logger itself. (These can be made available in appender output
via a PatternLayout class and the %P{yourpropname} format specifier. Use
%P{InstanceId} for the InstanceId.) When deserialized, this class can
detect an un-initialized state for log4net and recreate the correct
configuration.
SLogManager: To be used in place of log4net’s LogManager to dispense
SLog classes. This is the accepted way of dispensing custom loggers in
log4net’s design.
RegistryConfigurator: A class which consults a registry key for the
location of a log4net config file, and creates the configuration in the
specified “logging domain” using the log4net DomConfigurator class.
Specifics for BizTalk 2004 Usage
Using the assembly just discussed, the specifics of using log4net with a
particular BizTalk project can be described as follows:
-
Get 1.2.0 Beta8 of log4net at log4net
extension described above, as well as a
full sample project if you like. (Update
– the current version of this sample will now always be found in the
Deployment Framework sample – see download links at the
blog home page.)
-
If you are using NAnt to deploy your BizTalk solution, extend your
customDeployTarget as shown below to create a registry key that will point to a
log4net configuration file within your project’s root directory (whether on a
developer desktop or an MSI base directory). (Handled
for you in current rev of deployment framework.) Also, create a
file with the log4net extension that corresponds to your solution name (i.e.
BizTalkSample.log4net)
<target name="customDeployTarget">
<!-- Write registry key with location of our log4net
configuration file. This step is NOT needed in current rev
of Deployment Framework… -->
<exec program="cscript.exe" failonerror="true"
commandline="/nologo ${deployTools}\WriteRegValue.vbs
HKLM\SOFTWARE\${nant.project.name}\log4netConfig
${nant.project.basedir}\${nant.project.name}.log4net" />
</target>
-
Orchestration assemblies should reference log4net.Ext.Serializable as well as
log4net.dll, and declare a variable in each orchestration (perhaps named
“logger”) of type log4net.Ext.Serializable.SLog.
-
Orchestrations should begin (after an activating receive, if necessary) with an
expression shape that looks like:
logger =
log4net.Ext.Serializable.SLogManager.GetLogger(@"BizTalkSample",log4net.helpers.CallersTypeName.Name);
logger.RegistryConfigurator();
logger.InstanceId = MyOrchName(Microsoft.XLANGs.BaseTypes.InstanceId);
logger.Debug("New request arrived...");
Important: If an orchestration has parallel branches, you will want to declare
multiple variables of type log4net.Ext.Serializable.SLog, assigning 2-n to the
first one, as in:
logger2 = logger;
This prevents the need for synchronizing scopes. (See
this article for more.)&nbps;An alternative is to use scope-local
logger variables that are initialized by assigning them to a global instance.
-
At various points in your orchestration, simply make calls to
logger.Debug/logger.Info/logger.Error, etc. Rely on logger.Error for
event log entries (with appropriate logger/appender configuration.)
-
Use log4net with the components that your orchestration calls as well. If
those components are serializable, they will want to make use of
SLogManager/SLog classes as well in non-static methods. If you want
context (such as the orchestration instance ID) to flow, it might be worthwhile
to make an SLog-typed property visible on the class that the orchestration can
set. (This needs more thought.)
Using log4net with Paul Bunyan
Paul Bunyan is a logging tool that
has a very nice server-side buffering service as well as a great trace-viewing
client that can connect to multiple servers simultaneously (a great plus for
BizTalk groups!) Although Paul Bunyan has its own API, we aren’t
interested in it for purposes of this discussion.
A log4net appender can be easily written for Paul Bunyan, such that log output
can be directed there. Moreover, this appender can be made to explicitly
recognize an InstanceId property carried with a log4net logging event, and map
it to the “Context” concept within Paul Bunyan! This allows for viewing
the orchestration instance ID within a distinct, filterable column in the Paul
Bunyan viewer as such:
(click)
Sample .log4net configuration File
Note that types have to be referred to with fully-qualified names since
log4net.dll is being deployed to and loaded from the GAC.
<?xml version="1.0" encoding="utf-8" ?>
<log4net>
<!-- Define some output appenders -->
<appender name="OutputDebugString" type="log4net.Appender.OutputDebugStringAppender,
log4net,Version=1.2.0.30714,Culture=Neutral,PublicKeyToken=b32731d11ce58905">
<layout type="log4net.Layout.PatternLayout,log4net,
Version=1.2.0.30714,Culture=Neutral,PublicKeyToken=b32731d11ce58905">
<param name="ConversionPattern" value="%d [%t] %-5p %c - %m [%P{InstanceId}]%n" />
</layout>
</appender>
<appender name="EventLog" type="log4net.Appender.EventLogAppender, log4net,
Version=1.2.0.30714, Culture=Neutral, PublicKeyToken=b32731d11ce58905">
<param name="ApplicationName" value="BizTalkSample" />
<layout type="log4net.Layout.PatternLayout,log4net,
Version=1.2.0.30714,Culture=Neutral,PublicKeyToken=b32731d11ce58905">
<param name="ConversionPattern" value="[%t] %-5p %c - %m [%P{InstanceId}]%n" />
</layout>
<!-- Only send error/fatal messages to the event log. -->
<filter type="log4net.Filter.LevelRangeFilter">
<param name="LevelMin" value="ERROR" />
<param name="LevelMax" value="FATAL" />
</filter>
</appender>
<!-- Setup the root category, add the appenders and set the default level -->
<root>
<!-- Possible levels (in order of increasing detail):
OFF, FATAL, ERROR, WARN, INFO, DEBUG, ALL -->
<level value="ERROR" />
<appender-ref ref="EventLog" />
</root>
<logger name="BizTalkSample">
<level value="ALL" />
<appender-ref ref="OutputDebugString" />
</logger>
</log4net>
BizTalk 2004 has some very useful behavior around parallel execution and
scope-level timeouts that it is helpful to have a good understanding of.
What follows is a series of experiments & associated findings that should
shed some light on this area.
Experiment One:
One restriction when using the parallel shape is that if multiple branches make
use of a given orchestration variable, you can get a compiler error:
error X2226: 'someVar': if shared data is updated in a parallel then all
references in every task must be in a synchronized or atomic scope
In the experiment below, we are accesing ‘someVar’ in both parallel
branches - each within an expression shape that also calls Thread.Sleep (the
left hand for 10 seconds, the right hand for 30.) To address the compiler
error just noted, we have a scope around each usage with the
‘Synchronized’ flag on the scopes set to ‘true’.
The trace messages shown to the right of the diagram tell us that, indeed, the
execution of Scope 2 and Scope 3 is completely serialized (in this case, Scope
2 completes before Scope 3 begins.)
This experiment also tells us something unrelated (but useful): that an
exception will not interrupt “user code”. By
“user code” we refer to code in an expression shape, i.e. not
using an orchestration “intrinsic” such as a standard Delay shape
or a Receive shape, etc. Notice via the timings that the exception thrown
in the left-hand branch doesn’t abort the Thread.Sleep in the right-hand
branch. The exception is caught only after the Thread.Sleep in the
right-hand branch completes (though the final trace message in the right hand
branch – ‘after 30 sec’ – does not execute.) This
is important to understand if you have expression shapes in orchestrations
which are making blocking calls to .NET components, DCOM servers, etc.
Experiment Two:
If we eliminate the ‘someVar’ reference in the expression shapes
above, we find that the scopes do not execute serially – whether
the scope synchronization flag is set to true OR false. Notice
below (via the timings) that the sleep operations are executed at the same time
- so it is the presence of the common variable in the expression that forces
the synchronization!
We now have a 20 second gap after sleeping for 10 seconds (because the
exception we throw still doesn’t interrupt us).
Experiment Three:
We would like to use a given instance of a .NET object without imposing
the use of synchronized scopes. As it happens, if we have a .NET object
that is pointed at by two references (i.e. someVar and someVar2 - where
someVar2 was set equal to someVar with a simple assignment), the requirement
that the orchestration compiler normally imposes regarding the use of a
synchronized scope goes away.
In the orchestration below, the left branch is using someVar, and the right
branch uses someVar2. The trace indicates that the sleep operations
happen at same time (though once again the exception doesn’t interrupt
the right-hand side and is caught only when the right-hand completes.)
Lesson: If you have a .NET component (that you know to be thread-safe) you wish
to use in an orchestration - and you wish to use a single instance of it - you
will need to have multiple references to that same instance to use in each
branch of a parallel shape. (The synchronized-scope alternative is likely
unacceptable!) The first variable declaration of your component might use
the default constructor, while the others will have “Use Default
Constructor” set to false and will be assigned to the first.
 
Experiment Four:
Now, there is more to be said regarding exceptions and what they will interrupt
in parallel branches. If instead of using a Thread.Sleep we use a Delay
shape (or a Receive shape, etc.) we find that throwing an exception in one
branch of the parallel shape will indeed interrupt the other branch(es).
Notice in the timings below that the Delay 30sec shape does not complete
once the exception in the left-hand branch in thrown. Particularly when
you are structuring real-world business flows, this is an important and quite
useful behavior.
Experiment Five:
The behavior of exceptions in parallel flows is closely related to another
behavior in BizTalk 2004 - that of what BizTalk is prepared to "interrupt" when
a long-running scope has exceeded the timeout value that has been configured
for the scope. A Delay shape (or Receive, etc.) will indeed be
interrupted when the timeout expires (and a TimeoutException will be
thrown). (Note that for an atomic scope, the timeout governs the maximum
time to allow prior to aborting the transaction.) See the timings below
and note that the Delay 30sec shape does not complete.
 
On the other hand, as you might expect at this point, a blocking call in an
expression shape (like a Thread.Sleep or a DCOM call, etc.) will not be
interrupted. However, the exception will be raised when the blocking call
eventually returns:
 
Summary:
-
The Synchronized flag on a scope will only cause synchronized (serialized)
behavior among “peers in parallel branches” when shared variable or
message state is involved. (This is ignoring any transactional semantics
you might layer on top – which is beyond the scope of this article!)
-
If you have a thread-safe .NET component that you wish to use in an
orchestration from multiple parallel branches, strongly consider having
multiple variables point to a common instance. The first instance might
have “Use Default Constructor = true”, while the remaining
variables will have that flag set equal to false and be assigned to the first
instance in an expression shape:
someVar2 = someVar1;
someVar3 = someVar1;
someVar4 = someVar1, etc.
An alternative is to use scope-local variables that are assigned to a global
instance.
-
A line of code in an Expression shape will not be interrupted by either
an exception in a parallel branch or a TimeOutException arising from a
timed-out scope.
-
A Delay shape or Receive shape, etc. will be interrupted by either an
exception in a parallel branch or a TimeOutException arising from a
timed-out scope.
(Update: See the latest on the Deployment Framework
here.)
It has been about three months since my initial submission to the BizTalk 2004
developer competition that I discussed
here. The contents of that contest entry took a prize (woohoo!)
but still left a lot to be desired…
Since then, I’ve had a chance to make what I believe are major
improvements to the deployment story I’ve been discussing since last
May. In addition, I’ve had a chance to see the practice being used
across several large BizTalk projects, and have done a lot of learning as a
result. (No guarantees or warranty implied, but derivatives of the
scripts/tools I discuss here have been in use for some time by a very large
BizTalk customer I work with.)
The most important changes you will see in this release are as follows:
-
The “core” NAnt functionality has now been separated from the piece
that an individual BizTalk project must “own” and maintain.
This is a huge simplification, and allows for more rapid adoption -- as
well as the ability to deploy bug fixes & feature enhancements in a way
that just wasn’t possible when stuff was intermingled. (Peter
Provost was right
about that aspect of my first attempts.) The NAnt script that a
particular project must own can now be reduced to the following for a simple
case (notice the ‘include’ reference for the core):
<project name="BizTalkSample" default="debugDeploy">
<sysinfo/>
<include buildfile="BizTalkDeploymentInclude.nant"/>
<-- Set following properties to true or false to
include various pieces of a BizTalk deployment. -->
<property name="includeSchemas" value="true" />
<property name="includeOrchestrations" value="true" />
<property name="includeTransforms" value="true" />
<property name="includePipelines" value="true" />
<property name="includeComponents" value="true" />
<property name="includePipelineComponents" value="false" />
<property name="includeCustomFunctoids" value="false" />
<property name="includeVocabAndRules" value="false" />
<property name="includeVirtualDirectories" value="true" />
<property name="includeMessagingBindings" value="true" />
<property name="includeDeploymentTest" value="true" />
<property name="includeCustomTarget" value="true" />
<project/>
-
You now have the ability to supply a simple xml-driven Wizard-based UI when you
are deploying after an MSI install. You can use this UI to
gather Windows identity information for virtual directories (for SOAP/HTTP
receive ports), to indicate whether the scripts should deploy to the BizTalk
management database (for working in BizTalk groups), and for many other
purposes. I call this “Install-Time Configuration” in the
documentation below. Here are some example Wizard screens:
 
-
You now have the ability to have an Operations-managed
“spreadsheet-driven” mechanism for environment-relative
configuration. What do I mean? I mean those aspects of your
deployment that are particular to your development, QA, or production BizTalk
environments. Aspects that wind up appearing in your BizTalk binding
files! Queue names, database names, file shares, FTP locations,
etc. This is information that you don’t want to manage with the
Wizard UI above (there is too much of it) and you want to maintain “as a
set” for each environment. The spreadsheet (pictured below) can
start life with your developers, and ownership can gradually migrate to
Operations. This spreadsheet generates environment-specific
“settings” files, and at the point you deploy, the values for the
environment you are deploying to are automatically substituted into your
binding file…Slick!
(click)
-
You now can use a highly templated (reusable)
WiX based setup, rather than a Visual Studio Setup Project, to generate
your MSI. WiX, in a nutshell, is a set of tools and an Xml grammar that
allow you to specify the contents of an MSI. One of the clear pieces of
feedback I got on the last rev I released is that reproducing a Visual Studio
Setup Project was far too manual.
There are a large number of other changes that I’ll just enumerate
quickly here:
-
You no longer need to have your NAnt file enumerate your orchestration names or
their deployment order – a new custom NAnt task eliminates the
need for this.
-
You no longer need to have your NAnt file enumerate how many receive ports/send
ports/etc. you have – a new custom NAnt task eliminates the need
for this.
-
Support for BizTalk groups, where machines 2-n do not require deploying to the
BizTalk management database.
-
Support for not following the naming recommendations I made in previous
releases – you can have custom project names, assembly names, directory
names – its just a little more work.
-
Support for multiple assemblies of the various types (i.e. multiple
orchestration assemblies, schema assemblies, etc.)
-
Inclusion of a utility I call SetEnvUI.exe for creating the xml-based wizards
described above.
-
Inclusion of a utility I call DeployBTRules.exe for deploying BizTalk Rule
Engine policies and vocabularies.
-
Inclusion of Loren Halvorsen’s
XmlPreProcess tool for managing the environment-relative configuration
discussed above.
-
Support for creating IIS6 application pools (with specified Windows identity)
for HTTP and SOAP-based receive locations, and for adding virtual directories
to application pools.
-
Support for registering the btshttpreceive.dll ISAPI extension with IIS6.
-
Support for selectively restarting multiple BizTalk host instances, for
deploying custom functoids, and for dealing with send port groups, and
more…
There are now two downloads. The first contains all of the core scripts
and utilities plus a sample BizTalk solution that uses them. The
second download contains just the core scripts and utilities, and is designed
to allow a BizTalk solution to accept updates/bug-fixes/etc. over time (i.e.
the zip can be expanded into your directory structure on a developer
workstation so you can test it out.)
It should be noted, though, that some manual work will be required to
“upgrade” from the previous version I released in September –
this release is quite different, but I think you will find it is well worth the
time.
Full download is here.
Core scripts and utilities only are
here.
GotDotNet Workspace is
here.
The full documentation for this release (which includes more detail than this
blog entry…) is included in the zip files, but it can be viewed directly
here (with a diagram here.)
Enjoy! And remember: if you didn’t make into a given environment
(QA, production, etc.) with a scripted deployment of some kind, you
didn’t get there at all.
(Not necessarily this stuff, but something automated, at any rate…)
Some time ago, I published a set of
proposed naming conventions for BizTalk 2004.
In the interim, I’ve had a chance to see these used in a few different
projects and to review them with several people -- and they have evolved &
grown a fair bit.
So I thought I’d publish the new version - the Word version can be found
here, and an HTML version can be found
here. (I’ve found it helpful to open the HTML version
directly in VS.NET for quick reference.)
Hopefully these will be of value to your project. I’ve certainly
found that the documentation value to be had within Orchestrations and within
the operational tools can be hugely improved by following these
conventions – especially if you are using the
UK SDC BizTalk 2004 Documenter tool (you are, aren’t you?)
My comments on the use of such documentation
in my old post still apply, I believe.
Leave comments with questions or suggestions…
I had the pleasure of presenting on BizTalk 2004 at the
Heartland Developer’s Conference 2004. The whole conference
was a lot of fun, and as
Kent Tegels has noted, Joe
Olsen did a great job in organizing the logistics for this event –
you wouldn’t have known it was the first-of-a-kind (save for Joe’s
mea culpa on caffeine…)
I had a chance to talk to Sam Gentile
about the work he has been doing at Addesso. This was enlightening, since
in my non-BizTalk-work-life I do quite a bit mobile work in the
field-force-automation space (using ruggedized Pocket PC devices – great
fun!)
As to the presentation itself – one of the topics I discussed was BizTalk
2004’s scaling model, and the power of having being able to create
multiple host instances for a single logical host definition that contains your
orchestrations. This gives you a “competing consumer” effect
– multiple processes on multiple servers, all pulling from a common work
queue. In my talk, I suggested that this idea wasn’t new and
related it back to a
1998 article in Enterprise Developer’s magazine. In that
article, an architect from Merrill Lynch was discussing the scaling
difficulties in using NT4/MTS – the load balancing options weren’t
pretty. He proposed a pattern he dubbed “Auctioning” where
clients would submit work to an MSMQ queue, and multiple servers (all running
common server-side components) would pull work from that queue. The
benefits he articulated relate quite closely to what you will find in the
parallel aspects of BizTalk 2004.
Check out this graphic from the article – it should look a little
familiar to BizTalk folks.
Below is my paraphrase of the article’s main points, recast with BizTalk
terms (and exactly relevant to BizTalk, I believe…)
-
To load balance multiple servers, servers objects [orchestrations] should pull
work from a global [per-host] work queue that clients place their requests in.
-
Auctioning works well precisely because it relies on the message queue server
[messagebox], not DCOM [or transport of choice], to route the request to the
server object [orchestration].
-
Server objects [orchestrations] compete for requests on the queue, so load
balancing happens naturally; the server with the most available processing
capability will automatically check the queue more often and service more
requests. [unlike IP load balancing]
-
If you add more server machines, the requests just get processed faster by more
instances of the object [orchestration]
-
If one server goes down, the remaining servers continue to process the
requests. There's no single point of failure.
You can find the whole presentation (which also discussed BizTalk compared with
traditional application servers) right here.
It can turn out to be useful to use Health and Activity Tracking (HAT) and the
BizTalk Server Administration Console “remotely” (that is, from a machine other
than one of the actual servers in the BizTalk group.) This can be the
case if you have multiple environments to administer, and/or using Remote
Desktop is not desirable or available.
It should be noted that to accomplish this, you technically have to have the
“administrating machine” join the BizTalk Group – though since you won’t
have any host instances defined on the machine, that isn’t as significant as it
sounds. You are essentially just configuring the machine to point at a
particular BizTalk management database, and configuring some WMI
information. The “administrating machine” does not appear in the
“Servers” node of the BizTalk Administration Console.
It should also be noted that you will need to be in the “BizTalk
Administrators” Windows group for the BizTalk environment you want to
manage.
To get started, the “administrator” should do an “Admin-only” install of
BizTalk on the machine they will be using, where the installation options for
BizTalk look like this:
Then, on the desktop, the administrator should put a shortcut to a script that
looks like the script below. This script simply reminds the user what
BizTalk Group they are currently administrating, and confirms they want to
switch:
If you select OK, you get another warning:
This is basically warning someone who has a full-blown BizTalk installation on
their machine that they really don’t want to remove their current configuration
unless they happen to have a saved configuration file from their last run of
ConfigFramework.exe.
If you select OK, the “ConfigFramework.exe” utility is run (from BizTalk’s
installation directory) with the /u switch, to remove the current
configuration. Afterwards, ConfigFramework.exe is run normally, and the
only option you will have (for an admin-only install of BizTalk) is to select
the database corresponding to the environment you wish to administer:
Once you hit “Next”, the wizard will complete, and you will be presented with a
final confirmation dialog:
The VBScript as a text file is
here.
|