a trace of thought on...BizTalk Server, Team Foundation Server, AppFabric, etc. RSS 2.0
 Friday, July 22, 2005

Will this be the last place on the web that trumpets the BizTalk 2006 beta?  Likely not, but I did see it in quite a few places today.

Head to http://beta.microsoft.com with your passport and 'BizTalkBetaTeam' for a guest ID, and then wait patiently.  (While you're waiting, consider building out a VPC image with VS2005 beta 2 and, presumably, the latest SQL 2005 bits.  SQL 2000 would be fine as well, as BizTalk 2006 will not require SQL 2005.)

This...is going to be a great release.  Nothing so revolutionary that you can't leverage all the skills that you (or your staff) have already learned.  Yet, there are many, many important feature additions and "rough edges" removed.

Rattling off a few of the new items:

  • In-order delivery for any adapter that supports it (i.e. MSMQ, MQSeries, etc.)  In 2004, only MSMQ/T supported this.  (Of course, a faulty orchestration can break first-in-first-out - more on that in a later post.)
  • The introduction of an "Application" concept for grouping BizTalk assets - which extends to orchestrations, role links, send port groups, send ports, receive ports, receive locations, policies, schemas, maps, pipelines, other resources (e.g. soap proxies),  you name it!  Just as importantly, the management infrastructure understands applications - so health/management views can be narrowed down appropriately.
  • The management infrastructure has been completely encapsulated in an MMC - HAT is largely hidden.  More interesting is that the MMC can manage multiple BizTalk groups - and can do so remotely (by definition...)
  • A packaging/deployment solution that looks good - we'll have more to say about that in the coming weeks!  The developer experience in particular looks to be quite good.  Likely still some value-add to be done on the server side.
  • Ability to route failed messages - and subscribe in your orchestrations.
  • Calling pipelines from within orchestrations (no more loopback adapter or similar solutions needed...)
  • Zoom and expand/collapse-state-preservation within orchestrations.  (So when you collapse that big group or scope shape, it will stay collapsed across close/re-open.)
  • BAM integration with SQL Notification Services.
  • "Operator Role" has been defined to make allocating administration tasks a bit easier from a security perspective.
  • Pipelines can have per-instance configuration - saving you from recreating what were essentially a lot of duplicate pipelines!  (This was possible in 2004, I believe - but not exposed cleanly.)

This will be fun...I look forward to exploring the beta bits (man, the CTP was pretty short-lived...!)

 

Friday, July 22, 2005 5:09:29 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
General
 Friday, June 17, 2005

A whole raft of whitepapers for BizTalk have been released over the last several weeks - see here and here.

I completed a whitepaper a short while back (though just released) on getting the most out of the BizTalk 2004 Management Pack and Microsoft Operations Manager 2005.  The paper serves as a good reference for the management pack, but I hope it also serves another useful purpose.  Specifically, the "operational hand-off" phase of the software development life cycle often gets short shrift - and it can cost organizations a lot of money, downtime, and late nights.  So, much of the paper discusses the importance of having a development team accurately communicate the "instrumentation surface area" of their completed efforts to an operations team. 

What do I mean by "instrumentation surface area"?  To start, we can consider the sum of all diagnostic logging, event logging, WMI events, performance counters (custom or built-in), and all other mechanisms your application uses to communicate its current operational & health state.  Moreover, we need to capture "interpretations" of this information stream that are specific to the application.  (Not just "this send port isn't working..." but "We are currently not talking to our primary shipping provider...")  Finally, we need to capture suggested responses and remediation - also specific to the application.

"Communicated to the operations team"...how exactly?  With a Word doc?  Well, in particular, I talk about how to do this for a BizTalk-focused solution using a custom MOM Management Pack that "derives" from the Microsoft-supplied BizTalk 2004 Management Pack.  Done right, this will provide the highest fidelity knowledge transfer from development to operations.

See what you think - the paper is titled "Advanced Microsoft BizTalk Server 2004 and Microsoft Operations Manager 2005 Scenarios." (What a mouthful...)  

Comment on this post (if you like) with your thoughts on the paper or experience in this area...

Friday, June 17, 2005 7:31:32 AM (Central Standard Time, UTC-06:00)  #    Comments [1] -
BizTalk Insights

TechEd 2005 was a great time, and I really enjoyed the extended look at BizTalk 2006 - lots of great presentations.  Much more has been done in this release than I had anticipated, and I look forward to digging into the CTP build.

Thanks to those who attended my talk on automating BizTalk application deployments - the questions asked during the session were great!  It was fun to talk to the folks who are using the Deployment Framework stuff...The deck I presented is here.  BizTalk 2006 is doing a lot to make deployment easier - no comparison to 2004!  I expect a subset of the tools in the Deployment Framework to remain useful - but that is another post for another day.

If you attended my DevCon 2005 talk on mobility (and even if you didn't) you can find the deck available here.  The talk was specifically on the .NET Compact Framework 2.0 and Sql Server 2005 Mobile Edition.

Friday, June 17, 2005 7:23:39 AM (Central Standard Time, UTC-06:00)  #    Comments [1] -
Deployment Framework
 Tuesday, May 24, 2005

As has been noted before in the lore of BizTalk, it sure would be useful to use a real debugger with orchestrations - at least occasionally.

There are times when an expression shape winds up getting a bit sticky (not that I would know...) and a debugger would be just the ticket.  Other times, the exception you are getting from the orchestration engine isn't at all clear.

Still other times, seeing the actual contents of messages or context as you step through would be interesting.

Jon has posted on debugging orchestrations (in IL) with an ILDASM/ILASM loop, and had also discussed building your orchestration assemblies manually (using xsharpp.exe).  (He correctly noted these approaches wouldn't generate anything supportable, but they work for spelunking.) 

I wanted to suggest something a bit different - I wanted to go back instead to the venerable BTS File Dump utility that Charles released in May of 2004 (before the GenerateCSFiles registry key was discovered...) and propose a different technique for symbolic debugging.

First, install Charles' BizTalk File Dump utility - you can download it from here.  Fire it up, and change the output path in the utility to something easy, like c:\temp\BTSFileDump.

Here is the Edit/Debug cycle...(Start by making sure the file dump utility is running, and click the "Start dumping files" button.)

  1. Build your orchestration project (or whole solution if need be.)
  2. Do a Ctrl-Shift-F (Find in Files) in Visual Studio, and change the "Look in:" folder to c:\temp\BTSFileDump
  3. Search for something in one of your expression shapes, say, "MyClass.Execute".  The correct generated file (that the file dump utility grabbed) will appear in your Find Results - open it up, and set a breakpoint.  (Not on the xml designer portion - on the actual code!)
  4. From the Debug menu, choose Processes and attach to BTSNTSvc.exe.  (Have more than one?)  Choose CLR debugging only - not native.  The symbols should be loaded automatically - no need to copy PDBs to the GAC.
  5. Trigger your orchestration however you normally would.  Bask in a picture like this:  (Puts a Petzold-style WindowProc to shame...) 

    Debugview
  6. Use QuickWatch to examine message contents/context, if you like - you'll be interested in expansions like this one (where sampleRequest is a message in the orchestration.)
  7. Find your problem, Debug-Detach All, and fix the problem in the orchestration.
  8. Click "Delete all files in output path" in the file dump utility (to avoid duplicates) and rebuild.  Repeat the process if you need to.  (To save time, consider an external tool that just re-GACs the orchestration assembly and resets the host process.  See here.)

Note that this technique will work in a production setting.  You could copy the PDBs and sources to the production server, and use DbgClr.exe (in the Framework SDK) or cordbg.exe to attach to the appropriate host process.

Happy debugging...

Tuesday, May 24, 2005 12:42:15 PM (Central Standard Time, UTC-06:00)  #    Comments [0] -
BizTalk Insights
 Thursday, May 12, 2005
This is a test description.
Thursday, May 12, 2005 9:25:39 PM (Central Standard Time, UTC-06:00)  #    Comments [4] -
BizTalk Insights

Andy Morrison wrote about a technique for identifying the correct host instance to attach to when debugging components associated with orchestrations.

A potentially easier solution is to keep perfmon running with the counter shown below (BizTalk:Messaging/ID Process - all instances.)  The counter values will update every time a host instance recycles.  The Debug-Processes dialog in Visual Studio will show you (and allow you to sort by) process IDs to make this easy. 

Then put PerfMon in "report view" to easily see process IDs by host instance:

Thursday, May 12, 2005 9:32:41 AM (Central Standard Time, UTC-06:00)  #    Comments [3] -
BizTalk Insights
 Wednesday, May 11, 2005

There have been several folks who have discussed how to create messages "from scratch" within an Orchestration context - you can read Matt's thoughts and check out a BizTalk documentation excerpt. This is a common question, and I had an additional technique I thought I would share... 

Background: When you have a schema that has many promoted properties (or distinguished fields), or many elements that can be set via xpath expressions easily, it can be useful to simply start with a "template" document instance and populate the element content that you are interested in. 

In this situation, you will often have a "Message Assignment" shape that looks something like this:

xmlDoc.LoadXml("<ns0:BizTalkSampleS3 
      xmlns:ns0="http://BizTalkSample.Schemas.BizTalkSampleS3">
      <SomeElement></SomeElement></ns0:BizTalkSampleS3>");
someMsg = xmlDoc;
someMsg.SomeElement = "some content";
// (or xpath(someMsg,someElementXPath) = "some content" if we don't have a 
// distinguished field.)
   

One disadvantage of loading up "template" xml documents from either expression shapes or code (via XmlDocument.LoadXml) is that those xml fragments can get easily "lost", and are hard to update early in the development cycle when schemas may still be in flux.  Loading the template files from the file system is problematic because the question arises "where should I store these files, so that I can find them in any environment I deploy to?" (Solvable, but a hassle.)

Instead, why not embed the template xml documents as assembly resources?  For those unfamiliar with that process, I have a short tutorial here (& a helper class.) 

  1. You will need a C# project as part of your overall BizTalk solution.  Place your template xml file(s) in the directory corresponding to this project, and add them as an "existing item" to the project. 
  2. Select this file within the Solution Explorer, and within the Properties window, select "Embedded Resource" as the "Build Action" as shown here:

     

  3. Place this class (text here) within the same C# project that houses the resources you have added.
  4. To construct a message, drag out a "Message Assignment" shape, and within the associated expression write some code like the following.  Simply pass the file name of the template document as an argument to GetXmlDocResource (or GetStringResource.)
    sampleUsingTemplate = 
       BizTalkSample.Components.EmbeddedResourceCache.
          GetXmlDocResource("BizTalkSampleS3_output.xml");
       // Populate the "rest" of the message with distinguished fields, promoted 
       // properties, xpath expressions, etc.
       sampleUsingTemplate.SomeElement = "foo";
                

The class I have supplied will cache the loaded resources in a hashtable for performance sake, and allow you to load resources as both strings and XmlDocuments.

A last thought: Many people ask, "Why can't I just create a message using a new operator or a default constructor of some sort?" Well, because few XSD schemas sufficiently constrain the set of valid instance documents enough for that to be useful - what form would a "default message" take? (Would it have the optional elements you need? Some elements that you don't want?)

Enjoy - feedback appreciated!

Wednesday, May 11, 2005 3:29:14 PM (Central Standard Time, UTC-06:00)  #    Comments [6] -
BizTalk Tools | BizTalk Insights
 Monday, May 9, 2005

Just a few notes on what I'll be up to in the next month or two...

Do you use the Deployment Framework for BizTalk?  Or just have general questions on automating BizTalk deployments?  You can attend my session at TechEd 2005 - Friday June 10th, 10:45am to 12:00pm in the BPI Cabana, where I'll be doing a talk on those topics...

I'll also be speaking at the Microsoft DevCon event on my "other favorite" topic, Windows Mobile.  The focus will be on what is new for Visual Studio 2005 in the mobile arena...The conference will also be covering the new Team System offerings, ASP.NET 2.0, and quite a few other VS2005 topics.  Check it out...

Monday, May 9, 2005 8:10:41 AM (Central Standard Time, UTC-06:00)  #    Comments [3] -
Deployment Framework | General
 Friday, May 6, 2005

(Update: See the latest on the Deployment Framework here.)

Well, its time for another revision of the BizTalk/NAnt deployment tools... 

For folks recently joining us, these tools allow you to quickly create both developer workstation deployments (inside Visual Studio) and MSI-based deployments (for servers) for BizTalk 2004 solutions.  Just about all aspects of BizTalk deployments are handled, including items such as virtual directories, SOAP proxies, and WinXP/2k/2003 differences.

I get quite a few emails from people who are using these tools (which is great!)  A couple folks have suggested that I start referring to the whole thing as a" p framework?. < deployment>

That sounds about right - like all frameworks, there is a core set of functionality that takes care of quite a bit of work, and a given project can supply what needs customization or what is missing. 

In this case, the core NAnt scripts, custom NAnt tasks, WiX generation, and other custom tools form the framework itself - and a given BizTalk project can supply project-specific details, what needs to be customized, and any missing functionality.   So I'll refer to this work as the "Deployment Framework for BizTalk" from now on... 

What has changed in this rev? 

  • When last posting on this topic, I should have been even clearer that as of the last release, the deployment framework requires NAnt .85.
  • The biggest improvement is the ability to use binding files as they are emitted by the BizTalk Deployment Wizard directly.  In the past, the framework had insisted that binding files be split between orchestration bindings and messaging bindings - this is no longer required.  I had several requests for this fix...Also, note that having multiple binding files is supported (like one for each of several assemblies.)  Now, if you wish to continue to use the "old style" split bindings, define the orchBindings property in your project NAnt file as shown below:
  • <property name="orchBindings" value="${nant.project.name}.OrchBindings.xml" overwrite="false" />

  • You can now set the value of NAnt properties from the SettingsFileGenerator spreadsheet.  Recall that this spreadsheet is used (in conjunction with XmlPreProcess) to manage environment-relative configuration - that is, to manage those values that vary depending on whether you are in a development, QA, or production environment.  Until now, it has been difficult to use those configuration values within the NAnt deployment file!  You typically just used them in your binding files or other application-level XML files.  Now, however, you can define a property as shown below to make these values available as NAnt properties.  A good example usage is the case where you want the actual names of the SSO Application Users group and SSO Administrators group to be defined in the SettingsFileGenerator spreadsheet (because the group names may vary by environment you deploy to.)  Just define a property in your project NAnt file called "propsFromEnvSettings" with a comma-separated list of values you want pulled in.  (The sample has been updated to show this, too.)
  • <property name="propsFromEnvSettings" value="ssoAppUserGroup,ssoAppAdminGroup"/>

  • Many people had asked for the DeployResults.txt file (which is created in the DeployResults subdirectory during a deployment) to be available in a history log, so that it would be easy to walk up to a production server and determine what deploy/undeploy activity had taken place (especially useful for diagnosing any failures.)  To that end, the deployment framework will preserve past DeployResult.txt files with this convention: `DeployResults TRACESATURN_05032005_0922.txt'  Note that the machine name, date, and time are preserved - so you can gather these files across all the servers in a BizTalk group, if need be.
  • SSOSettingsFileReader class improvements: In the last release, I introduced the ability to manage settings within the BizTalk SSO using the SettingsFileGenerator spreadsheet that is also used for managing environment-relative configuration.  Part of that functionality included a class called SSOSettingsFileReader, used to retrieve settings at runtime.  This class now has some strongly typed accessors, and no longer sits inside a lengthy namespace, which means that in an orchestration expression shape you can simply do:
  • someVar = SSOSettingsFileReader.ReadString("BizTalkSample","SomeAppConfigItem") // or ReadInt32

  • Ben Cops had found that on large-scale deployments, it was useful to restrict which orchestrations were deployed/started to speed the development cycle.  So, he modified the orchestration NAnt task accordingly.  To use (if you need to), just set the orchestrationsToIgnore property in your project NAnt file with a comma-separated list that reflects what you are currently working on.  Make sure to copy BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory again! (you'll find it in the DeployTools directory in the download.)
  • A bug was fixed with undeployment for the case where an SSO affiliate app had already been deleted...
  • All references in NAnt files to xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" were removed, because they were simply causing heartache (for many reasons.)  Remove this namespace in your project NAnt file when upgrading.
  • Rather than trying to keep the source for the C# tools (NAnt tasks, SetEnvUI, DeployBTRules, SSOSettingsFileImport\SSOSettingsFileReader, etc.) current within the GotDotNet workspace, I'm making them available as a zip

Remember, you do not need to piece together the current functionality set from past blog entries - the complete documentation (in Word form) is in the download and separately here.

Some last thoughts:

  • Some folks have been reluctant to use the deployment framework because of its reliance on NAnt, and a worry over using NAnt in a server environment.  One thing to remember (that might allay this concern) is that a subset of NAnt is delivered with the generated MSI, and NAnt will not require a separate installation procedure by your production operations team.  Moreover, NAnt doesn't create any kind of persistent service or footprint on the production machine (i.e. no automatic file associations that would cause scripts to run with a double-click or anything like that...)
  • Add a Visual Studio External Tool for the updateOrchestration target if you haven't already!  This can speed your development cycle significantly.  See the tools zip for a script that creates all relevant external tools.
  • When using the WiX-based MSI generation, make sure your project NAnt file (for WiX generation, like BizTalkSample.WiXSetup.Build) is choosing "debug" or "release" based on what you want to put in the MSI.  See comments in that file for more detail, and Jon Flanders' post on this topic.

How do I upgrade?

  1. With a safe copy of your BizTalk solution, expand the "Core" zip contents right on your project directory, such that the core scripts will overwrite what you have now (but your project-specific scripts will be preserved.)
  2. Manually copy the WiX-related updates to your *.WiXSetup directory, since they will get unzipped to BizTalkSample.WiXSetup
  3. Remove xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd" from your project NAnt file
  4. Copy DeplyTools\BizTalk.NAnt.Tasks.dll to your program files\nant\bin directory
  5. Do a deployment on a developer workstation and a server to test things out...

Downloading

  • You can always use the download links on this blog.
  • Full sample here, core scripts here, extra tools here
  • (But always check the download links to see if there has been an update…)
  • Standalone documentation here (and diagram here.)
  • GotDotNet workspace here
    Friday, May 6, 2005 1:13:00 PM (Central Standard Time, UTC-06:00)  #    Comments [5] -
    Deployment Framework
    Archive
    <July 2005>
    SunMonTueWedThuFriSat
    262728293012
    3456789
    10111213141516
    17181920212223
    24252627282930
    31123456
    About the author:

    Scott Colestock lives, writes, and works as an independent consultant in the Twin Cities (Minneapolis, Minnesota) area.

    © Copyright 2014
    Scott Colestock
    Sign In
    All Content © 2014, Scott Colestock
    DasBlog theme 'Business' created by Christoph De Baene (delarou)