Microsoft BizTalk Server 2009 Microsoft Corporation

April 2009

Abstract

Microsoft® BizTalk® Server 2009 helps organizations meet the challenges of creating automated business processes that connect diverse systems. From its initial roots in EAI and B2B integration, BizTalk Server continues to add new capabilities and engine imcreate powerful service-oriented architectures and business process integration and management solutions, and that enable administrators and business users to more effectively monitor ongoing business processes. BizTalk Server 2009 represents the next release in Microsoft’s longto enable the connected enterprise.

This technical overview provides you with a guided tour owalkthrough of the important features and business primer on how developers, administrators, and business users use BizTalk Server to develop and manage business process solutions.

Microsoft BizTalk Server 2009 Technical Overview

Microsoft® BizTalk® Server 2009 helps organizations meet the challenges of creating automated business processes that connect diverse systems. From its initial roots in EAI and B2B integration, BizTalk Server continues to add new capabilities and engine improvements that allow developers to

oriented architectures and business process integration and management solutions, and that enable administrators and business users to more effectively monitor ongoing

Server 2009 represents the next release in Microsoft’s longto enable the connected enterprise.

provides you with a guided tour of BizTalk Server 2009. It provides a walkthrough of the important features and business benefits of BizTalk Server and provides a detailed primer on how developers, administrators, and business users use BizTalk Server to develop and manage business process solutions.

Technical Overview

Microsoft® BizTalk® Server 2009 helps organizations meet the challenges of creating automated business processes that connect diverse systems. From its initial roots in EAI and B2B integration,

provements that allow developers to oriented architectures and business process integration and management

solutions, and that enable administrators and business users to more effectively monitor ongoing Server 2009 represents the next release in Microsoft’s long-term strategy

provides a benefits of BizTalk Server and provides a detailed

primer on how developers, administrators, and business users use BizTalk Server to develop and

2

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This technical overview is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred.

© 2009 Microsoft Corporation. All rights reserved.

Microsoft, BizTalk, Hyper-V, InfoPath, PerformancePoint, SharePoint, Visual Studio, Windows, Windows PowerShell, and Windows Server are trademarks of the Microsoft group of companies.

SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

All other trademarks are property of their respective owners.

3

Contents

Microsoft BizTalk Server 2009 Technical Overview ................................................................................... 1

Abstract ................................................................................................................................................... 1

PRODUCT OVERVIEW ...............................................................................................5

BizTalk Server 2009 Key Capabilities .......................................................................................................... 5

New Features in BizTalk Server 2009 .......................................................................................................... 6

Updated Platform Support ....................................................................................................................... 6

Hyper-V Virtualization Support ................................................................................................................ 6

Improved Failover Clustering .................................................................................................................. 6

Enhanced Business Activity Monitoring (BAM) ....................................................................................... 6

Enhanced Support for AS2 ...................................................................................................................... 7

Enhanced Support for EDI ...................................................................................................................... 7

Improvements to Host Integration Server 2009 and the BizTalk Adapters for Host Systems .................. 7

BizTalk RFID ........................................................................................................................................... 7

Universal Description Discovery and Integration (UDDI) ......................................................................... 7

Enterprise Service Bus 2.0 ...................................................................................................................... 7

BizTalk Adapter Pack .............................................................................................................................. 9

Changed features and tools ......................................................................................................................... 9

The Role of BizTalk Server in a Service-Oriented Architecture .............................................................. 10

BizTalk Server Business Scenario............................................................................................................. 10

How BizTalk Server Works ......................................................................................................................... 11

The Publish/Subscribe Model ................................................................................................................ 12

How Messaging and Orchestration Services Process Messages ......................................................... 13

Building a BizTalk Application ................................................................................................................... 14

BizTalk Projects and Assemblies .......................................................................................................... 14

CREATING A MESSAGING APPLICATION ................................................................... 16

Building Schemas .................................................................................................................................. 16

Mapping Data ........................................................................................................................................ 17

Connecting Through Adapters .............................................................................................................. 21

Processing Messages Through a Pipeline ............................................................................................ 24

BUILDING A BIZTALK ORCHESTRATION ................................................................... 28

Orchestration Designer ......................................................................................................................... 28

The BizTalk Orchestration Engine ......................................................................................................... 30

4

Calling External Assemblies .................................................................................................................. 31

Service Integration Scenarios ............................................................................................................... 31

BizTalk Service Integration Capabilities ................................................................................................ 32

BizTalk Orchestration in SOA Designs .................................................................................................. 32

THE BUSINESS RULES FRAMEWORK ....................................................................... 33

The Business Rule Composer ............................................................................................................... 33

Business Rule Policy ............................................................................................................................. 35

The Rule Engine Deployment Wizard ................................................................................................... 35

BUSINESS-TO-BUSINESS INTEGRATION ................................................................... 37

EDI Parties ............................................................................................................................................ 38

BizTalk Server Accelerators .................................................................................................................. 39

BUILDING RFID SOLUTIONS ................................................................................... 41

BizTalk RFID Infrastructure ................................................................................................................... 41

MONITORING BUSINESS ACTIVITY ........................................................................... 43

Business Activity Monitoring (BAM)....................................................................................................... 43

BAM Activities ....................................................................................................................................... 44

BAM Views ............................................................................................................................................ 44

Aggregating and Filtering Data .............................................................................................................. 45

Gaining Better Visibility in SOA Solutions ............................................................................................. 45

MANAGEMENT AND OPERATIONS ............................................................................ 46

Deploying a BizTalk Application ............................................................................................................ 46

Tracking and Debugging BizTalk Processes Using the Group Hub Page ............................................. 47

Monitoring BizTalk Applications Using MOM ......................................................................................... 49

Implementing Load Balancing ............................................................................................................... 50

Securing Applications ............................................................................................................................ 50

SUMMARY ............................................................................................................. 52

5

Product Overview

BizTalk Server is Microsoft's premier server for building business process and integration solutions. BizTalk Server 2009, the sixth major version of the product, builds on the innovation and success introduced by the previous four versions and now includes full support and integration with Windows Server® 2008, SQL Server® 2008, and Visual Studio® 2008.

BizTalk Server 2009 offers significant improvements and capabilities over BizTalk Server 2006 R2 for connecting applications and businesses, automating and managing business processes, and building solutions based on a service-oriented architecture. What used to take customers months and years to design and implement can now be accomplished in just days and weeks.

BizTalk Server 2009 Key Capabilities

BizTalk Server provides a host of capabilities that enable developers to create powerful business process integration and management solutions and allow administrators and business users to effectively monitor ongoing business processes. BizTalk Server 2009 includes the following core capabilities:

• Messaging. BizTalk Server enables the efficient processing of incoming and outgoing messages. This capability provides connectivity to disparate systems and trading partners through a variety of file formats and adapters and is enforced by message-level security.

• Orchestration. BizTalk orchestration provides transactional and non-transactional message processing through centrally managed business processes. Orchestrations enable the automation and standardization of complex processes that are composed in an intuitive visual drawing and executed by the BizTalk Orchestration engine at run time.

• Business Rules Framework. The Business Rules Framework enables the creation of business rule policies that define the logic for a given business process. The policy logic abstracts the business process logic out of an orchestration. This enables updates to be made to the business process logic without requiring recoding of the orchestration.

• Business-to-Business integration. Electronic transactions with trading partners can play a vital role in enterprise business processes. BizTalk Server enables business-to-business integration through industry standards and well-established protocols such as Electronic Data Exchange (EDI) data (including X12, EDIFACT, and HIPAA support) and Availability Statement 2 (AS2) data for EDI over the Internet. BizTalk Server Accelerators speed up the development of B2B solutions within specific industry segments that adhere to specific protocols such as SWIFT, HL7, and RosettaNet.

• BizTalk RFID. BizTalk RFID provides a device management and event processing platform that enables the development, deployment, and management of Radio Frequency ID (RFID) and sensor solutions.

• Business Activity Monitoring (BAM). BAM provides real-time monitoring and archived statistical data of BizTalk processes. BAM enables business users to gain true end-to-end visibility of these processes.

• Management and operations. BizTalk Server has robust management of the BizTalk Server runtime environment, including application management, application deployment, host management, and process execution tracking and reporting.

• Tools. BizTalk Server provides a number of tools that help configure, design, deploy, manage, and view BizTalk processes and capabilities. These tools come in a variety of forms; some are integrated into Microsoft Visual Studio 2008, some are add-ins to the Microsoft Management Console (MMC), while others are Web-based.

6

This technical overview will provide you with a technical overview of these BizTalk Server 2009 capabilities. For more detailed information, refer to the BizTalk Server 2009 Capabilities Guide and Poster.

New Features in BizTalk Server 2009

BizTalk Server 2009 builds on the core architecture of BizTalk Server 2006 and makes strides in many aspects of application-to-application, business-to-business, and business-process automation. The following is a list of both new and enhanced features in this release.

Updated Platform Support

BizTalk Server 2009 supports the latest Microsoft platform technologies, including Windows Server 2008, SQL Server 2008, Visual Studio 2008 SP1, and the .NET Framework 3.5 SP1. These platform updates enable greater scalability and reliability, and many advances in the latest developer tools.

• Integration with Windows Server 2008 offers modular, minimal installation, improved network performance and control, improved high availability features, enhanced administration and management with Server Manager and Windows PowerShell™ command-line interface, and enhanced virtualization with Hyper-V™.

• Integration with SQL Server 2008 (while maintaining support for SQL Server 2005) offers better manageability and scalability, an optimized virtual SQL Server implementation, improved performance (especially in a 64-bit environment), BAM enhancements, support for Unified Dimensional Model (UDM) cubes in SQL Server Analysis Services, and scalable real-time aggregation.

• Integration with Visual Studio 2008 SP1 introduces a number of improvements to the underlying Visual Studio-based BizTalk project system, which enhances debugging support for artifacts such as BizTalk maps (XSLT), pipeline components, and XLANG orchestrations, BizTalk Server 2009 enables support for unit testing with Microsoft Visual Studio Team System 2008 Test Edition. With new support for the latest version of .NET Framework and improved integration with Visual Studio, developers can now take advantage of BizTalk artifact property pages being integrated into Project Designer tabs, as well as migration with the Visual Studio Project Update Wizard, support for the Microsoft Build Engine (MSBUILD), and support for both release and debug builds.

One new component of Microsoft Visual Studio Team System 2008 Team Foundation Server is the Application Life-Cycle Management (ALM) feature. This enables development teams to leverage the integrated source control, bug tracking, support for team development, integration with Microsoft Office Project Server 2007, and support for automating builds by using MSBuild.

Hyper-V Virtualization Support

BizTalk Server 2009 takes advantage of the latest virtualization improvements included as part of Windows Server 2008 Hyper-V, which can lead to reduced costs through lower hardware, energy, and management overhead, plus the creation of a more dynamic IT infrastructure.

Improved Failover Clustering

By taking advantage of Windows Server 2008 clustering, BizTalk Server can now be deployed in multi-site cluster scenarios, where cluster nodes can reside on separate IP subnets and avoid complicated VLANs.

Enhanced Business Activity Monitoring (BAM)

By expanding the system-provided BAM functionality with SQL Server 2008 Analysis Services, BizTalk Server 2009 provides support for UDM cubes and scalable real-time aggregations, which enhances support for Microsoft PerformancePoint Server 2007.

7

Enhanced Support for AS2

Transmitting EDI transactions over the Internet is an efficient alternative to sending and receiving EDI using value-added networks (VANs). Using the Internet for data exchange reduces costs, increases efficiency, and has advantages in terms of redundancy and scalability.

To support this trend, BizTalk Server 2006 R2 introduced support for EDIINT AS2 (Applicability Statement 2). AS2 is a specification that enables transport of business data over the Internet in a safe and reliable manner. BizTalk Server uses AS2-defined methods to send, receive, encrypt, decrypt, decompress, sign, and verify signatures between partners using HTTP over the Internet. BizTalk Server helps ensure the security of messages through the use of encryption keys, digital signatures, certificates, and non-repudiation.

BizTalk Server 2009 expands these capabilities with support for multiple message attachments, configurable auto message resend, end-to-end file name preservation, improved reporting to address new features, and Drummond recertification.

Enhanced Support for EDI

Electronic Data Interchange (EDI) is one of the most prevalent means by which businesses exchange data electronically. It represents approximately 75 percent of all business-to-business electronic transactions and grows at about 5 to 7 percent per year. EDI usage entails message syntax and standards (including ANSI X12 and UN/EDIFACT), messaging protocols, and transports. BizTalk Server 2006 R2 introduced the ability to process EDI messages using EDI pipeline components, as well as AS2 support.

BizTalk Server 2009 improves EDI support with control of envelope headers, automatic rollover of control numbers, configurable content delimiter character, support for multiple batches per party, HIPAA schemas now supporting equivalent segments, and updated reporting to cover all new features.

Improvements to Host Integration Server 2009 and the BizTalk Adapters for Host Systems

In addition to the new platform support, Host Integration Server now offers two new features: the WCF Channel for WebSphere MQ (Transport Channel, Data Format Channel Encoder), and the WCF Service for Host Applications (based on Transaction Integrator). Host Integration Server also offers support for new versions of IBM products such as CICS, IMS, CICS HTTP transport, DB2, DB2/400, and DB2 Universal Database. Transaction Integrator now has a fully managed runtime, extended data conversions, and performance improvements. The Host Files & DB2 .NET Data Provider offer extended data conversions, performance improvements, the entity data model provider, Workflow Foundation for data activity scenario, an offline file load scenario, the BizTalk Adapter for WebSphere MQ, and a pipeline data conversion component which works with Visual Studio Designer.

BizTalk RFID

BizTalk RFID has been extended to mobile devices (such as handheld devices and forklift readers), and integrates with BizTalk Server. It also offers support for key industry standards, enabling the use of new readers with Low Level Reader Protocol (LLRP), machine readable tag data standards (TDT for EPC), and Web Services for device management and Discovery, Configuration, Initialization (DCI).

Universal Description Discovery and Integration (UDDI)

Version 3.0 of UDDI supports registry affiliation, subscription API, digital signatures, and other extended discovery features.

Microsoft Enterprise Service Bus Guidance 2.0

Microsoft Enterprise Service Bus (ESB) Guidance 2.0 is an offering from the Microsoft Patterns & Practices (P&P) team (http://go.microsoft.com/fwlink/?LinkId=148337) that includes architectural guidance, patterns, practices, and a set of BizTalk Server 2009 and .NET components to simplify the development of an Enterprise Service Bus (ESB) on the Microsoft platform. ESB Guidance 2.0

8

extends the capabilities of Microsoft BizTalk Server 2009 to support a loosely coupled messaging architecture. Most developers are familiar with code-oriented, procedural, or object-oriented development paradigms; however, when starting to develop BizTalk Server solutions, developers tend to overlook the message-oriented capabilities of BizTalk Server.

BizTalk Server includes a powerful publish/subscribe mechanism that works by creating and filling subscriptions. When a new message arrives in the MessageBox database, a message agent identifies subscribers and sends the message to any endpoints that have subscriptions. Subscriptions can be created in several ways, including binding an orchestration to a receive port, having a correlated receive waiting for a message, or creating a send port with a filter condition that matches a property of the message (such as the type, the receive point, or the value of a routable property).

By providing this efficient and scalable approach, BizTalk Server enables developers to create a series of discrete sub-processes, define the types of messages that trigger their invocation, and not worry about the sequence. A process initiated by the arrival of a message performs its processing on the message, and may then deliver this or another message to the MessageBox database, which in turn may activate one or more sub-processes.

Microsoft provides key building blocks that are required for building comprehensive service-oriented Infrastructures, including Windows Server 2008, .NET Framework, and BizTalk Server 2009. Microsoft ESB Guidance 2.0 is founded on BizTalk Server 2009 because it provides the basis for the most common ESB services, including the following:

• Message routing

• Message validation

• Message transformation

• Extensible adapter framework for connectivity

• Service orchestration

• Business rules engine

• Business activity monitoring

• Web service and WS-* integration (WCF adapter)

Microsoft ESB Guidance 2.0 extends the functionality of BizTalk Server 2009 to provide a range of new capabilities focused on building robust, connected, service-oriented applications. Microsoft ESB Guidance 2.0 treats BizTalk Server components as individual units of work that can be connected as desired to form loosely coupled solutions. The following are some of the core capabilities provided by ESB Guidance 2.0 to enhance BizTalk Server 2009:

• Policy-driven mediation:

o Provides itinerary-based service invocation that supports lightweight service composition at the time of message publication. The Itinerary mechanism dynamically resolves service endpoints and mediation requirements, and routes messages using a registry or the rules engine. This approach enables developers to implement loosely coupled patterns such as VETO/VETRO.

o Adds support for server-side itineraries, processing instructions that are dynamically added to messages upon receipt. Hosting itineraries in a central location enables the ESB to process messages from clients that are completely unaware of itineraries or the internal processes that will process submitted messages.

o Uses an enhanced version of Microsoft ESB Resolver and Adapter Provider Framework, which enables dynamic resolution of endpoints and transformation requirements, effectively decoupling the consumer from the services.

9

• Connecting systems:

o Provides pipeline components that perform normalization of XML message namespaces.

o Provides WebSphere MQ connectivity.

o Facilitates messaging patterns that enable dynamic service aggregation, message routing, message validation, and message transformation.

o Incorporates service registry and repository integration using UDDI 3.0 and WS-MetadataExchange.

o Supports the LOB adapters that are addressed through WCF-Custom.

• Management and monitoring:

o Implements exception mediation and fault management.

o Includes a sample Web portal that facilitates message repair and resubmission.

o Provides BizTalk Server endpoint and registry integration, management, and publication.

o Provides a centralized repository of versioned server-side itineraries.

o Supports reporting and analytics for exceptions, alerts, and registrations.

• SOA governance:

o Integrates with third-party SOA governance solutions, including management agents for BizTalk Server 2009 from AmberPoint and SOA Software.

BizTalk Adapter Pack

The Windows Communication Foundation Line of Business (WCF LOB) SDK & BizTalk Adapter Pack 2.0 is now upgraded to the latest platform. In addition to enhancements to the existing adapters, the pack includes two new adapters: the BizTalk Adapter for SQL Server and BizTalk Adapter for Oracle eBusiness Applications.

Changed features and tools

The following features and tools were available in BizTalk Server 2006, and are replaced by new features in BizTalk Server 2009.

Base EDI Adapter BizTalk Server 2009 includes support for processing EDI and AS2 messages using XSD-based EDI schemas and pipeline components. This feature replaces the Base EDI Adapter and schemas included in previous releases of BizTalk Server.

MSMQt Adapter The MSMQt adapter is not included in BizTalk Server 2009. To communicate with Microsoft Message Queuing (MSMQ,) use the MSMQ adapter.

Human Workflow Services (HWS) HWS is not included in BizTalk Server 2009. As an alternative, you should now use Windows Workflow Foundation, which is installed as part of the .NET Framework.

Business Activity Services (BAS) Business Activity Services (BAS) is not included in BizTalk Server 2009. You can use the EDI and AS/2 features for this functionality.

BizTalk Server Migration Project The BizTalk Server Migration Project enabled migration BizTalk Server 2002 artifacts to a BizTalk Server 2006 project. The BizTalk Server Migration Project is not included in BizTalk Server 2009. BizTalk Server 2009 uses the Visual Studio .NET Migration Wizard to migrate BizTalk Server projects and artifacts from earlier versions of BizTalk Server. (Note that migration of projects and artifacts from versions earlier than BizTalk Server 2006 is not supported.)

10

Health and Activity Tracking (HAT) Tool The tracked services and tracked messages reports that used to reside in the HAT module are now part of the New Query tab of the HUB group page.

The Role of BizTalk Server in a Service-Oriented Architecture

Service-Oriented Architecture (SOA) is an IT architectural style that supports the transformation of IT assets into a set of linked services or repeatable business tasks that can be accessed as needed over a network. The value of implementing BizTalk Server as a platform for SOA is apparent when you consider how an organization can improve operational processes, achieve greater alignment of IT with the business goals, and maximize the reuse of IT assets through composite applications.

The BizTalk messaging capability contributes to service orientation by enabling applications to be exposed as services, or rather by creating service façades that enable the application to interact with other services. BizTalk orchestration provides the ability to design workflows to automate business processes and to compose services from multiple service providers. By providing real-time visibility into BizTalk and non-BizTalk processes, Business Activity Monitoring (BAM) is a major asset in the realm of management and governance.

As an example, BizTalk Server can play a major role in SOA by allowing enterprises to implement very agile service oriented infrastructure using the Enterprise Service Bus (ESB) Guidance 2.0

For more information on creating SOA solutions using Microsoft products and technologies, refer to http://go.microsoft.com/fwlink/?LinkId=140027.

BizTalk Server Business Scenario

BizTalk Server 2009 supports a broad spectrum of business scenarios and industries from financial services to manufacturing and healthcare. To understand how BizTalk Server solves common business problems, we’ll use a sample scenario throughout this technical overview to demonstrate how a fictitious company, Northwind Traders, has implemented BizTalk Server as an integration and business process management solution to support its supply chain business requirements.

Northwind Traders is large retail chain store with locations throughout the United States. Northwind Traders has several LOB applications that are used to manage internal business processes at different levels within the company. A CRM application is used to manage customer account information for the sales department and a centralized ERP application is used to manage inventory for the entire business.

11

Figure1: The Northwind Traders business scenario

Each store has a custom inventory application that is used to maintain inventory for that particular store. Each of these systems presents its own unique integration challenges. Additionally, in the past Northwind Traders has had difficulty with accurate inventory tracking within and to a given store and has decided to implement an RFID strategy to better track inventory.

Northwind Traders has hundreds of suppliers located throughout the world that provide Northwind Traders with its in-store products. Integration with these trading partners has been one of the greatest challenges for Northwind Traders since each partner often uses it own proprietary systems and unique document formats to exchange purchase order information and shipping acknowledgments. Also, changing or adding new suppliers has always been a time-consuming process that requires several layers of approvals and sometimes even changes to the business process itself.

Finally, as with many companies, Northwind Traders has had challenges in enabling its business managers to make timely and critical business decisions due to lack of information about the state of various business processes. Latency in inventory and sales reporting information has often resulted in loss of sales opportunities and prevented managers from executing on critical buying opportunities.

As you review this technical overview, you will learn how Northwind Traders has successfully implemented BizTalk Server 2009 to solve many of its integration and business process automation requirements.

How BizTalk Server Works

At the core of BizTalk Server 2009 are the Messaging Engine and the Orchestration Engine, which provide the underlying architecture for integrating and exchanging messages between various services, both within and outside your organization.

The BizTalk Messaging Engine:

• Receives inbound messages.

• Parses inbound messages to identify their specific formats.

12

• Evaluates message content to identify how the message is to be routed and processed.

• Delivers messages to their respective destinations.

• Tracks the status and state of documents.

The BizTalk Orchestration Engine, in contrast, coordinates and schedules message processing and performs complex logic on the message as it is passed through a defined workflow.

The Publish/Subscribe Model

BizTalk Server implements the publish/subscribe model to route messages. The publish/subscribe model enables developers to design processes and services that subscribe to messages rather than create point-to-point connections between two services. This enables new service providers and consumers to be added or existing services to be modified without having to redesign the application.

In this model (as shown in Figure 1) the message providers, also called the publishers, submit messages to a central store (the MessageBox database). The subscribers, which can be send ports or orchestrations, subscribe to specific messages. After the MessageBox receives a message of interest, the message is delivered to all subscribers.

Subscriptions in BizTalk Server are based on matching expressions to name and value pairs associated with each message that is processed by BizTalk Server. These name and value pairs are known as message context properties. Each BizTalk message has a message context associated with it, which travels with the message as it is processed by BizTalk Server. The message context is represented in memory as an object, and persisted with the message as a set of name and value pairs when stored in the MessageBox database. When a message is received by the MessageBox, certain message context properties are evaluated against known subscriptions, which are expressions made up of potential message context properties and values and operators (such as “And”, “Or”, and “Exists”).

For example, in the Northwind Traders solution, restock orders are submitted by each retail store.

These orders are received by BizTalk Server where they are routed to the orchestration that is

responsible for coordinating the restock process across multiple systems. As this orchestration

processes the restock order, it will eventually create a purchase order and send it to a subscribing

send port to reach a specific supplier.

13

How Messaging and Orchestration Services Process Messages

Figure 2. The BizTalk Server publish/subscribe model

Figure 2 shows how BizTalk Server implements the publish/subscribe model.

1. Messages enter the BizTalk Server system through a receive port. Each receive port contains one or more receive locations. Each receive location is configured with an adapter, which defines the communication method used to connect to and receive data from an external system or application, such as a file folder, an HTTP site, an SQL database, or a third-party application.

2. The received message is processed by the receive pipeline. A pipeline can contain various components that help decrypt a secure message, split batched messages, convert a message from its native format into an XML document, or validate the digital signature of a message.

3. Receive ports can optionally be configured with one or more maps, which transform messages from one data structure to another. Maps are used to transform messages from various disparate formats to a single internal or canonical format used by the BizTalk application.

4. The message is then delivered to a Microsoft SQL Server database, called the MessageBox. When a message arrives in the MessageBox, the metadata associated with the message is evaluated against the existing subscriptions to determine which send ports or orchestrations the message should be delivered to.

5. An orchestration defines the logic that controls a business process workflow. A business process can use one or more orchestrations. Each of these orchestrations consists of specific types of shapes that are used to express conditions, loops, and other behaviors.

14

6. The message, which may or may not be processed by an orchestration, can be delivered to a send port. The send port can transform the message data using a map and then process the message through a send pipeline.

7. The send pipeline may convert the message from the internal XML format used by BizTalk Server to the format required by its destination as well as encrypt the message for secure communication.

8. The send port then uses an adapter to connect and transmit the data to the external system or application.

Building a BizTalk Application

BizTalk Server 2009 provides a rich set of development tools for designing, organizing, and building the various elements of a BizTalk application. The BizTalk project system is hosted in Visual Studio 2008 and provides developers with a fully integrated design experience to create parts of a BizTalk application or an entire business solution. As shown in Figure 3, the core element of a BizTalk solution is a BizTalk project—a collection of items (artifacts) including schemas, orchestrations, pipelines, maps, Web message types, classes, and references. These artifacts are compiled into an assembly before deployment for testing or to a production environment.

Figure 3. BizTalk projects hosted within Visual Studio 2008

BizTalk Projects and Assemblies

A simple BizTalk Server business application may consist of a single BizTalk project compiled into a single assembly. However, a more complex business solution that integrates many disparate systems and processes may require many different assemblies that are deployed individually to several different BizTalk Server computers. Individual BizTalk projects can be developed separately by different developers who are responsible for specific parts of an application. These projects can be deployed individually or combined into a single application solution.

15

A BizTalk project consists of one or more artifacts, which are saved as specific file types. Each type of artifact plays a specific role in the BizTalk solution. The BizTalk project system provides a corresponding graphical design tool for creating and modifying each type of artifact.

• Schemas. A schema provides a definition for the structure of the data within an XML message. While BizTalk Server natively processes XML formatted messages, special extensions of the XSD standards enable BizTalk Server to process EDI and flat-file messages. BizTalk Editor is the design tool that simplifies the process of defining schemas.

• Maps. A map is used to transform the data from one structure to another. BizTalk Mapper is the design tool that presents a source schema and destination schema side-by-side and enables you to define transformations between the data elements of different messages.

• Pipelines. A pipeline performs a variety of operations to prepare incoming or outgoing messages for further processing. Pipeline Designer enables you to implement operations such as encryption and decryption, compression, reformatting, and validation.

• Orchestrations. An orchestration represents the workflow for a business process. Orchestration Designer enables you to design the logic and flow for an orchestration by dragging and configuring different graphical shapes to the designer surface.

16

Creating a Messaging Application

The BizTalk Server 2009 messaging subsystem enables communication with a wide range of systems and applications. It supports the conversion to and from different data formats to handle proprietary protocols and standards-based services.

BizTalk Server relies on the use of structured documents for all internal messaging and orchestration operations. Regardless of the format of the incoming message (for example, XML, flat-file, or EDI) the BizTalk messaging and orchestration engines can only process XML formatted messages internally. To create a basic message processing application, a developer must:

1. Create the schema definitions for all types of messages to be processed by BizTalk Server.

2. Create one or more maps to transform the data from one schema structure to another.

3. Configure the receive ports and receive locations to enable the receiving of messages.

4. Configure the send ports for the sending of data to external systems.

5. Create a custom pipeline for any custom processing that the message data requires.

Building Schemas

The schemas processed by BizTalk Server can come from a variety of different sources. A schema definition might be predefined by some external application or it might be sent by a trading partner. To integrate with an existing EDI application, BizTalk Server provides over 8,000 different schemas to support existing EDIFACT and X12 message standards. However, in many cases you will have to create the schemas yourself for XML or flat-file document structures by using BizTalk Editor.

Supported Schema Types

BizTalk Server 2009 natively supports the following schema types:

• XML. The BizTalk messaging and orchestration engines require that all messages be in XML format. An XML schema defines the hierarchical structure of an XML message. BizTalk Server identifies and validates all messages against an associated schema.

• Flat-file. A flat-file schema defines the structure of messages that are received in a flat-file format. A flat file can be either delimited or positional. The XML Schema Definition language (XSD) does not natively support the flat-file structure; therefore, BizTalk Server uses the annotation capabilities of XSD to store this extra information within the XSD schema itself. BizTalk Server defines a rich set of specific annotation tags that can be used to store all of the required additional information.

• EDI. BizTalk Server enables the creation and use of schemas to represent standard EDI document formats such in EDIFACT and X12. An EDI message is a variation of a text message and does not use typical delimiters such as carriage returns and linefeeds. As with flat-file schemas, BizTalk Server uses the annotation capabilities of XSD to store the extra information related to the format of the EDI messages.

It's not uncommon for a single solution to use all three schema types. In the case of Northwind

Traders, all communications between Northwind Traders and its suppliers use flat-file or standard EDI

message formats as defined by X12 or EDIFACT. Communications among BizTalk Server, internal

systems, and Web services use XML.

17

BizTalk Editor

You use BizTalk Editor to create, edit, and manage the schemas for a BizTalk application without needing to learn all the intricacies of the XSD syntax. This tool runs within the Microsoft Visual Studio 2008 environment and starts automatically when you either add a new schema to a BizTalk project or open an existing schema in the project. As shown in Figure 4, BizTalk Editor displays a hierarchical order of records, field elements, and field attributes to represent the structure of message instances.

Figure 4. BizTalk Editor

Creating and Validating Schemas

You can use the following methods to create and validate schemas by using BizTalk Editor:

• Create a schema from scratch. You may use this method to create XML, flat-file, or EDI schemas for those messages which do not have any instances, or for messages meant only for internal use. You can also use this method when other tools do not provide the necessary functionality.

• Create a schema from scratch in conjunction with other schemas. In real-world scenarios, you will mostly create complex schemas by modifying the existing schemas by using the XSD processes of importing, including, and redefining schemas created previously.

• Modify existing schemas. Regardless of the original source of a schema, you can use the BizTalk Schema Editor to modify and ensure the validity of an XSD schema.

• Use the Flat File Schema Wizard. The Flat File Schema Wizard provides a simple-to-use wizard interface to assist in the development of flat-file schemas.

• Generate a schema from an instance message. You can generate an XML schema that corresponds to a particular instance message that consists of well-formed XML.

Mapping Data

You use a BizTalk map to convert an input message that conforms to one schema into an output message that conforms to a different schema. These conversions can be either simple or complex, involving calculations and consolidation of information.

18

A map defines the relationship between an input and output schema by using links and functoids. A link defines a direct data copy of a record or field. A link specifies the basic function of copying data from an element or attribute in an input instance message to an element or attribute in an output instance message. You create links between records and fields in the source and destination schemas at design time. As a result, during run time, links direct the creation of an output instance message conforming to the destination schema from an input instance message conforming to the source schema. Links may either directly connect to items in the other schema or form connections through functoids. Functoids are described in a later section.

Transforming and Translating Messages

Maps enable you to transform and translate messages.

Transformation is the process of converting an XML document that conforms to one schema into an XML document that conforms to another schema. In other words, transformation is the process of taking information from one message and inserting it into another message. A transformation can simply change the formatting applied to the data, but more often, data transformation results in some structural changes in the document.

In addition to simply mapping data between two messages, the transformation process can include operations such as:

• Flattening records received in a message that has a hierarchical format to one with a flatter design

• Averaging data from multiple input nodes and sending the output to a single field in the destination message

• Applying mathematical functions on values in the source message and then writing the result to the destination message

• Concatenating multiple elements from the source message into a single field in the destination message

• Looking up a value from the source message in a database or an in-memory table and extracting new values to be written to the destination message

Translating Messages

Translation is a special case of data transformation that involves changing the format of an instance message, typically from non-XML (flat-file or EDI) to XML format, or vice versa. For example, if your internal processes utilize XML data but your trading partner needs to receive messages in a flat-file format, you can perform the necessary translation before sending messages to the trading partner. Data translation can be especially helpful in solving enterprise application integration problems by rendering a given type of message into alternative formats required by existing systems.

Data transformation and translation is typically common when building integration solutions. BizTalk maps allow you to translate or transform messages. You can use maps in orchestrations or when processing a message in a send or receive port. This section discusses the role played by BizTalk maps in the BizTalk Server architecture.

Message transformation and translation are an important part of the Northwind Traders

implementation. The restock orders from each store are received in XML format. The data in the

restock message must be transformed into an IDOC structure in order to update the SAP ERP system.

Additionally, Northwind Traders uses a single XML format when processing orders internally. However,

each supplier requires its own unique format, either flat-file or EDI, for purchase orders sent by

Northwind Traders. When the purchase order is being processed by the send port, a map is used to

19

transform the message from the standard internal XML format to the supplier-specific EDI or flat-file

format.

BizTalk Mapper

BizTalk Mapper is a tool that runs within the Microsoft Visual Studio 2008 environment and enables you to create and edit maps. You use BizTalk Mapper to define the relationship between an input and an output schema by using links and functoids as shown in Figure 5. BizTalk Mapper supports one-to-one and one-to-many links. For example, a link can connect a single record or field from the source schema to a single record or field in the destination schema. A link can also connect a single record or field from the source schema to multiple records or fields in the destination schema. BizTalk Mapper supports complex structural transformations from records and fields in the source schema to records and fields in the destination schema.

Figure 5. BizTalk Mapper

BizTalk Mapper provides a solution for a variety of mapping scenarios, ranging from simple parent-child tree-type operations to detailed operations that are complex and involve looping records and hierarchies. BizTalk Mapper stores maps in a file with a .btm extension. The file saves design information about the map, such as the locations of icons that represent functoids, the links between schema items and functoids, etc. When you build or compile the map, BizTalk Mapper converts the information about the map into the corresponding XSLT stylesheet.

As with most other BizTalk artifacts, before the map can be used by BizTalk Server, the project containing the map needs to be compiled into an assembly. As you develop a map, BizTalk Server generates compiler errors if it encounters any type mismatches between the source and destination schemas. The compiled map is generated into XSLT code that is executed when the map is applied by BizTalk Server during run time.

Functoids

While links copy values from one message to another, functoids perform more complex data manipulations on the contents of the message, such as:

20

• Adding the value of two fields in the source schema and copying the result to the destination schema

• Converting a character to its American Standard Code for Information Interchange (ASCII) format

• Returning the average of a field in a repeating record and copying the result to a field in the destination schema

Functoids enable you to graphically create complex transformations more easily than you can with standard XSLT. They allow you to extend the functionality of the map to perform a variety of operations on data as the data is being transformed from the source message to the destination message.

There are approximately 70 functoids that provide simple mathematical functions, string manipulation, date and time insertion, and complex scientific calculations. The most common use of a functoids is to perform numeric calculations, such as summing the total number of products ordered. Functoids that can have zero or more inbound links can be chained to provide additional functionality. The basic functoid categories for predefined functoids include:

Functoid Description

Conversion Perform conversions between numeric bases, such as from hexadecimal

to decimal.

Cumulative Perform accumulation operations for values that occur multiple times within

an instance message.

Date and Time Introduce the current date, time, or both into the message or add days to a

specified date, in the output data. This enables you to insert the processed

date and time into a message or calculate an anticipated ship date.

Logical Perform specific logical tests at run time or determine whether output

instance data is created at run time. These functoids return either True or

False.

Mathematical Perform calculations by using specific values (arguments) in a specified

order or structure.

Scientific Convert a numeric value to a scientific value. For example, the Cosine

functoid takes a value in radians from a field or record and returns the

value of the cosine.

String Manipulate data strings by using string functions. The String Concatenate

functoid combines two or more inputs, such as nodes, constants, or other

functoids, and builds a string. The String Find functoid finds one text string

within another text string and returns the position of the first character of

the found string.

Functoids perform calculations by using predefined formulas and specific values, called arguments. These calculations are executed based on the designated order of the records and fields. To use a functoid in a BizTalk application, you simply need to drag it from the Toolbox to the grid page and link it to the source and destination schema nodes as shown in Figure 6.

21

Figure 6. BizTalk Functoids

Note: For more information on different categories of functoids available in BizTalk Server, refer to the “Functoid Categories” article on http://msdn2.microsoft.com/en-us/library/aa546768.aspxMSDN at http://go.microsoft.com/fwlink/?LinkId=140029.

In addition to using the predefined functoids, you can also write your own functoids as a script file or a .NET assembly. The BizTalk Server SDK contains examples on how to use scripting functoids as well as create custom functoids.

Connecting Through Adapters

BizTalk Server 2009 requires specialized adapters to exchange messages with disparate applications and systems. An adapter is a.NET-based software component that enables BizTalk Server to interface with different types of systems through standards-based protocols and with specialized applications that use proprietary communication mechanisms.

Most adapters support both send and receive operations, whereas others support communication in a single direction only. You define the behavior of an adapter by configuring the properties of the send port or the receive location for a given instance of an adapter.

Receive adapter

A receive adapter works in conjunction with a receive port and a receive pipeline to retrieve messages from the source location, also known as the endpoint. Once received, the messages are saved to the MessageBox database.

Each receive adapter has specific context information about the message or the metadata that is associated with the protocol that it supports. This metadata can include information such as the original file name, the time the message was received, and the sender information. The metadata is saved along with the message data and can be accessed by the BizTalk Messaging Engine when evaluating subscriptions for routing purposes or by the BizTalk Orchestration Engine to make processing decisions within an orchestration instance.

Send adapter

A send port adapter and the send pipeline work together via the messaging engine to send the message from the MessageBox to a specific endpoint.

Native Adapters

22

BizTalk Server 2009 natively provides adapter support for communicating with many different systems. The following table describes the native adapters available in BizTalk Server:

Adapter Description

File For transferring files in and out of BizTalk Server through the local file system.

The File adapter consists of two adapters, a receive adapter and a send

adapter.

FTP For exchanging data between an FTP server and BizTalk Server, and allows

for the integration of vital data stored on a variety of platforms with line-of-

business (LOB) applications. The FTP adapter consists of two adapters, a

receive adapter and a send adapter.

HTTP For exchanging data between BizTalk Server and an application by means of

the HTTP protocol. Applications can send messages to a server by sending

HTTP POST or HTTP GET requests to a specified HTTP URL.

SOAP For receiving and sending data through Web service requests. The SOAP

adapter supports bi-directional communication. This adapter is provided for

backward compatibility. For new implementations the WCF-BasicHttp adapter

should be used in place of the SOAP adapter.

MSMQ Supports Microsoft Message Queuing 2.0 and Message Queuing 3.0 from

BizTalk Server 2009. Enables communications across heterogeneous

networks and systems that may be temporarily offline.

MQ Series Serves as a bridge between BizTalk Server and IBM MQSeries servers.

SQL For exchanging data between BizTalk Server and a SQL Server database. The

SQL adapter can be used to poll data from one or more data tables and then

transmit the data as one or more XML messages to BizTalk Server. Also, you

can use the SQL adapter to insert, update, and delete the data in SQL Server

tables by using SQL updategrams or by invoking stored procedures.

Windows®

SharePoint®

Services

Exchanges messages between BizTalk Server and Windows SharePoint

Services. Enables two-way transformations of messages to and from Microsoft

Office InfoPath®.

POP3 To receive messages from a POP3-enabled mailbox to BizTalk Server.

SMTP To send messages to an e-mail address by using the SMTP protocol.

In the case of Northwind Traders, the ability to use native adapters included with BizTalk Server 2009

enabled them to easily integrate users, internal systems, and trading partners. For example, each

store manager submits restock orders through a SharePoint site that is monitored by BizTalk Server

using the Windows SharePoint Services adapter. During the processing of orders, updates are sent to

users via e-mail using the SMTP adapter. Approval messages are received via e-mail using the POP3

adapter. The SAP adapter is used to communicate with the ERP system, and the SQL adapter is used to

send to and receive data from the warehouse inventory system. Northwind Traders service-enabled

each of the remote store inventory systems to enable communication using the WCF adapter.

Northwind Traders communicates with its many suppliers by using adapters for FTP, HTTP, SOAP, and

POP3/SMTP.

23

WCF Adapters

A major addition to BizTalk Server 2009 is several new adapters that enable communication to and from BizTalk Server and Web services-based applications via Windows Communication Foundation (WCF). The WCF adapters support HTTP/HTTPS, SOAP, MTOM, TCP, MSMQ, and Named Pipes transports. The WCF adapters include:

• WCF-BasicHttp

• WCF-WsHttp

• WCF-NetTcp

• WCF-NetMsmq

• WCF-NetNamedPipe

• WCF-Custom

• WCF-CustomIsolated

Each adapter includes a custom UI for configuring the most commonly used features of that particular binding as well as a custom binding adapter that supports WCF extensibility.

Some of the supported use cases for the new WCF adapters include:

• Exposing BizTalk orchestrations as a WCF web service

• Exposing BizTalk Content-Based Routing applications as WCF Web services

• Consuming WCF services from BizTalk orchestrations

• Consuming WCF services from Content-Based Routing applications

• Transactional message receive

• Transactional message send

• Using WS-* headers for routing and message processing

• Using custom headers for routing and message processing

• Using custom binding elements

• Using custom bindings

• Using BizTalk dynamic send ports

• Using BizTalk Server as a SOAP intermediary

Rather than building a custom adapter, Northwind Traders created WCF service façades for the store

inventory systems, which BizTalk Server integrates with by using one of the WCF adapters.

In addition to these WCF adapters, BizTalk Server 2009 provides the BizTalk WCF Service Publishing Wizard and the BizTalk WCF Service Consuming Wizard.

• The BizTalk WCF Service Publishing Wizard helps you create and publish BizTalk orchestrations as WCF services and publish schemas as WCF services.

• The BizTalk WCF Service Consuming Wizard helps you generate BizTalk artifacts, such as orchestrations and types, to consume a WCF service based on the metadata document of the WCF service.

The BizTalk Line-of-Business (LOB) Adapter Pack

24

Microsoft also provides a broad array of technology and application adapters that enable BizTalk Server to connect to numerous line-of-business (LOB) applications such as SAP, PeopleSoft, JD Edwards, Oracle, Siebel, Microsoft Dynamics, and many others. Adapters are released or updated regularly. You can see a list of the LOB adapters currently available here: http://go.microsoft.com/fwlink/?LinkID=88542.

Additional Sources for Adapters

In addition to the adapters included in BizTalk Server 2009, there are two additional sources of BizTalk adapters:

• BizTalk partner adapters. For unique and specialized integration scenarios, adapters have been created by a number of third-party companies. These application-specific and technology-specific adapters can be purchased from companies who specialize in developing adapters or from those companies that provide adapters to enable integration with their proprietary applications. Some examples of third-party adapters include SAP, Lotus Notes, and CICS.

• Microsoft WCF Line of Business Adapter SDK. If you are unable to locate an adapter to support your communication requirements, you can develop your own custom adapter. To simplify this process Microsoft provides a consistent framework for developers to build line–of-business adapters

based on Windows Communication Foundation (WCF). The WCF LOB adapter SDK includes a rich set

of development tools to automate and simplify adapter development in a consistent and repeatable

manner. You can download the SDK from http://go.microsoft.com/fwlink/?LinkID=96184.

• Pre-existing Custom Adapters. Existing custom adapters that were built with the Adapter Framework (prior to the BizTalk Server 2009 release) are still supported for backward compatibility.

Processing Messages Through a Pipeline

The purpose of a pipeline is to prepare a message for processing by the server after it has been received by an adapter or to prepare a message for sending through an adapter after it has been processed by BizTalk Server. A pipeline is a set of .NET components that process messages in a predefined sequence to complete a specific task, such as encrypting a document or validating a document against a schema. Pipelines enable pre- and post-processing of messages as they enter or leave the MessageBox database, which means that pipelines can process messages either as they are received or just before they are sent out through a send port.

Pipelines provide additional processing to messages such as encoding and decoding, encrypting and decrypting, and other processing that might be required when working with messages in BizTalk Server. You can also call a pipeline component from within an orchestration.

Pipeline Processing Stages

Pipelines are divided into categories of work called processing stages. The stages determine the sequence in which the processes are performed. Processing stages are implemented in a prescribed order that cannot be modified. Each stage of a pipeline contains one or more pipeline components that can be configured to work with the specific requirements of the messaging solution or orchestrated business process. Each stage defines logical work groups, determines which components can go in that stage, and specifies how the pipeline components in the stage are executed.

The processing stages for a pipeline depend upon its intended use. BizTalk Server provides two types of pipelines: receive and send. These pipelines require separate categories of work, such as message encoding versus decoding. The pipeline also governs the process sequence by the use of policy files that specify the order in which each stage needs to be executed. For instance, an incoming message must be decoded or decrypted before it can be disassembled.

25

Pipeline Processing Tasks

Within each stage, pipeline components perform specific tasks. For example, components within the stages of a receive pipeline may decode, disassemble, and then convert documents from other formats to XML. The components in the stages of a send pipeline do just the opposite; convert documents from XML to other formats, assemble, and finally encrypt the message.

BizTalk Server pipelines perform these transformations as well as other data-specific actions, such as data encryption or decryption and property promotion, on both incoming and outgoing messages. Pipelines commonly perform:

• Data normalization from various formats to XML

• Data transformation from XML to various formats

• Property promotion and demotion to enable routing decisions based on specific data in a message

• Document disassembly and assembly

• Document decoding and encoding

• Document decryption and encryption

• Document signing and digital signature verification

Receive Pipelines

After a message is received by the receive adapter, the receive pipeline takes the initial message, performs some transformations, and disassembles the raw data into zero, one, or multiple messages. The BizTalk Server then processes these individual messages.

You can use receive pipelines to:

• Process messages as they are received by BizTalk Server. For example, you can use receive pipelines to decode and/or decrypt messages as well as verify the sender of messages as they are being received. This is important because messages exchanged over the Internet are frequently encrypted, and it is necessary to confirm the identity of the sender of the message.

• Extract individual messages from a message interchange.

• Validate messages against a schema to ensure that they meet the requirements of your processes.

Send Pipelines

A send pipeline processes messages as they are sent by BizTalk Server. The pipeline takes one message and produces one message to send. For example, you can use send pipelines to encrypt messages using a public key or digitally sign outbound messages using a private key as the proof of the sender. Before a message is sent, you can also use the validate component of a send pipeline to ensure that a message is valid against a known schema.

By default, the send pipeline consists of three empty stages: Pre-assemble, Assemble, and Encode. You can either create a new send pipeline or use one of the two default send pipelines included in BizTalk Server, the pass-through send pipeline and the XML send pipeline.

In the Northwind Traders scenario, different pipelines are used depending on the required data

formats and specific agreements that Northwind Traders has with its various suppliers. If the supplier

requires flat-file messages, then a flat-file pipeline is used to process the messages. If a supplier

requires EDI messaging support then messages are processed through a native EDI pipeline. If the

supplier supports EDI over the Internet then an EDI and AS/2 pipeline is used. If a supplier requires all

26

data exchanges to be encrypted than a pipeline is configured to encrypt outgoing messages and to

decrypt all messages received.

Custom Pipeline Components

BizTalk Server 2009 pipelines allow you to customize the processing of documents received or sent by various adapters. Custom pipeline components extend the behavior of default pipelines by enabling you to process virtually any data format. The only requirement for creating custom receive pipeline components is that the data emerges from the pipeline as XML.

Custom pipelines are a powerful solution for legacy systems that require integration with other products but do not follow standards. For example, your data may contain carriage returns in odd places or contain records that span multiple lines of text.

Other examples of situations when you may consider using a custom pipeline include:

• Validations that cannot be expressed in an XML schema

• Decryption algorithms not supported by BizTalk Server out of the box

• Checks on signature formats that BizTalk Server does not yet recognize

• Conversions that might not be possible by using BizTalk Mapper

• Requirement for a custom disassemble algorithm to split up messages

Pipeline Designer

Pipeline Designer provides a graphical representation of a pipeline and enables you to create or modify send and receive pipelines, view the pipeline templates included with BizTalk Server, move pipeline components within a pipeline, and configure pipelines, stages, and pipeline components. You can navigate between pipelines by clicking the tabs at the top of the design surface as shown in Figure 7. The file extension for both receive and send pipelines is .btp.

Figure 7. Pipeline Designer

BizTalk Messaging in SOA Designs

27

BizTalk schemas, maps, pipelines, adapters, send ports, receive locations, and receive ports provide the core components of a BizTalk messaging application. The BizTalk messaging capabilities provide developers easy-to-use tools for enabling or exposing services. You can use BizTalk messaging to create service facades for older legacy systems that cannot be service enabled by using the WCF SDK and the BizTalk WCF adapters.

28

Building a BizTalk Orchestration

A business process is a set of actions that, taken together, meet a specific business need. Business process automation enables the coordination of common business processes, for example approving a purchase order, with people and business applications—such as Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), or other LOB applications. These processes frequently start as manual tasks that require integration and input from various systems and individuals. The process may take hours, days, weeks, or months to complete.

BizTalk orchestration is a flexible and powerful capability that provides various services and tools to enable you to design, automate, and manage business processes. Similar to traditional procedural programming languages, an orchestration represents the underlying logic for processing messages.

BizTalk orchestration provides a transactional programming model that includes support for exception handling and recovery from failed transactions. You can define two types of transactions when creating an orchestration:

• Atomic transaction. Enables a transaction to automatically role back to a previous state in case the transaction does not successfully complete.

• Long running transaction. Can span days, weeks, and longer time durations, contain nested transactions, and use custom exception handling to recover from error scenarios.

As a result, orchestrations provide you the flexibility to define the course of action in the event of a business process failure.

In the Northwind Traders scenario, restocking the store's inventory is a process that requires data to

be sent to and received from many different systems both internal and external to the organization.

The restocking process begins when a store manager submits a purchase order. The order is then

processed to determine the appropriate supplier for each item in the order. Then an order is placed

with each appropriate supplier. A supplier sends a notification when the product has shipped. Once all

the products for the order have arrived at the Northwind Traders distribution center, the order is

shipped from the distribution center to the store. When the store receives the shipment the process is

complete.

Orchestration Designer

Orchestration Designer provides a design surface that enables you to create visual representations of your business processes. The design surface is a working canvas where you drag different shapes from the Toolbox as shown in Figure 8. The shapes correspond to the different actions that you need to perform when processing a message. In most cases, you must configure options for each shape that you use when you build the orchestration.

The design surface is divided into three areas: one Process Area and two Port Surfaces. The Process Area contains shapes that describe the actual process flow of the orchestration. For example, scope shapes helps you define transactions in a business process. A scope contains one or more blocks. It has a body and can optionally have any number of exception-handling blocks as well as a compensation block appended to it.

The Process Area of the design surface is flanked on both sides by Port Surfaces, which contain only Port and Role Link shapes that interact with the send and receive shapes in the Process Area. You can use either side (right or left) of a Port Surface to create a send or receive port. This enables you to create well-documented orchestrations that have fewer crisscrossing connectors, making your orchestrations easier to read.

29

Figure 8. Orchestration Designer

Steps to Develop an Orchestration

The steps for developing an orchestration are as follows:

1. Define the schemas to describe the format of the messages to be processed by the orchestration.

2. Add and configure the shapes to represent the various actions that are required to define the business process.

3. Define new message instances to be processed within the orchestration.

4. Define the orchestration ports to receive and send messages.

5. Define and assign orchestration variables to declare and manage the data used in the orchestration.

6. Bind the send and receive shapes to ports, and specify the physical ports that they will use.

7. Build, deploy, and test the orchestration.

30

The BizTalk Orchestration Engine

The BizTalk orchestration run-time engine is a highly optimized service that monitors and executes orchestrations. At run time, the BizTalk Orchestration Engine executes the files that you create using Orchestration Designer.

The BizTalk Orchestration Engine performs the following tasks:

• Creating instances of and executing orchestrations

• Maintaining the state of a running orchestration instance so that it can be restored to memory when required

• Performing optimizations of running orchestrations to maximize scalability, throughput, and efficient use of resources

• Providing a reliable shutdown-and-recovery system

The orchestration engine executes orchestrations by creating individual instances of the business process. The run-time engine coordinates these multiple instances to ensure that a response message gets routed to the correct orchestration instance, by using a specialized routing pattern called correlation.

In the Northwind Traders scenario, the restock process requires, over time, several different message

exchanges to be sent and received between internal systems and a supplier. As a BizTalk

orchestration, multiple instances of the restock processes can be running concurrently, one for each

restock order submitted by a store. Each incoming message instance is correlated or matched up with

the correct running orchestration instance.

Dehydration and Rehydration

When many business processes run at the same time, memory and performance can be compromised. The BizTalk Orchestration Engine solves this problem by dehydrating and rehydrating orchestration instances. Dehydration is the process of saving the state of an active orchestration instance to the MessageBox database and then removing that instance from memory. This can happen when the orchestration engine determines that an instance has been idle for a period of time. Dehydrating the instance frees up the resources that were being used by the instance.

Dehydration

The orchestration engine calculates thresholds to determine how long an orchestration will wait for various actions to take place, and if those thresholds are exceeded, it dehydrates the instance. This can occur under the following circumstances:

• When the orchestration is waiting to receive a message and the wait is longer than a threshold determined by the engine.

• When the orchestration is waiting for a subscribed message.

• When a delay in the orchestration is longer than a threshold determined by the engine.

• Between the retries of an atomic transaction.

The orchestration engine saves the entire state of a running orchestration instance to persistent storage at various points so that the instance can later be completely restored in memory. The dehydration of orchestration instances enables the orchestration engine to have more "in-flight" instances than the physical resources of the computer running BizTalk Server would normally allow. Dehydration is automatically controlled by the orchestration engine; however, you can control the dehydration threshold by changing specific BizTalk Server configuration properties.

31

Rehydration

Rehydration is the reverse of dehydration. It is the process of de-serializing the last running state of an orchestration from the database or restoring the saved state of an orchestration instance from disk back to memory. The orchestration engine is triggered to rehydrate an orchestration instance when it either receives a message or when a time-out expires. It then loads the saved orchestration instance into memory, restores its state, and runs it from the point where it left off.

An orchestration can be configured to run on more than one server. After an orchestration instance has been dehydrated, it can be rehydrated on any of these servers. If one server goes down, the engine continues to run the orchestration on a different server, continuing from its previous state. The engine also takes advantage of this feature to implement load balancing across servers.

Calling External Assemblies

Through the course of a business process, there are times when the execution requires the functionality of another .NET application or a functionality that is better developed in a different common runtime language, such as C#. BizTalk orchestrations can pass parameters to external applications through method calls, and subsequently integrate the functionality of external applications into the BizTalk process.

Service Integration Scenarios

WCF and Web services are used to expose the functionality of a system or application to other applications and are, by far, the most implemented mechanism for service enablement. BizTalk Server 2009 fully supports existing WCF and Web service standards. This support enables developers to both consume external services as part of a business process and publish a business process as service that can be consumed by external applications.

If you require a specific business process to be accessible via the Internet to customers, trading partners, or other applications, you can publish an orchestration as a WCF service. You do this by running the BizTalk WCF Services Publishing Wizard. The wizard creates a WCF-based ASP.NET application that runs within Internet Information Services (IIS).

Some examples of publishing an orchestration as a service include:

• Airlines publish fare information online as a service. A travel Web site can call multiple airlines' services to determine the current price for tickets from multiple airlines.

• A shipping company exposes its business processes as online services. Online retailers can call the shipper’s services to calculate shipping costs and display tracking information about the shipment to their customers.

Some common integration scenarios for integrating services into a business processes include:

• Determining shipping costs from external shippers for a product

• Calculating tax for an item depending on the country from which the item is being purchased

• Obtaining a credit report or credit score from a third-party company when processing a loan

• Accepting or denying a credit card for an online purchase

BizTalk Server 2009 enables you to call services from within an orchestration and through messaging send ports. In addition, you can generate industry-standard Web services and custom WCF services by publishing existing orchestrations or schemas, simplifying typically complex integration scenarios.

Northwind Traders uses WCF services for all internal communications and uses ASMX Web services for

communication with external partners. To achieve this, Northwind Traders used BizTalk Server to

create WCF service facades around each internal system, and published several schemas as Web

services on the Internet so that external partners can communicate with Northwind Traders.

32

BizTalk Service Integration Capabilities

In addition to the WCF adapters, BizTalk Server 2009 also provides the following support for integrating with WCF services:

• Consuming WCF services. You can use the BizTalk WCF Service Consuming Wizard to generate BizTalk artifacts, such as BizTalk orchestrations and types, which consume a WCF service based on the metadata document of the WCF service.

• Publishing orchestrations as WCF services. Using the WCF Publishing Wizard you can publish BizTalk orchestrations as WCF services. Publishing an orchestration as a WCF service exposes the functionality of the business process to external services such as trading partners or customers. Using metadata from orchestration port types and schema types, the wizard automatically creates a WCF-based ASP.NET application which acts as a WCF service façade for the BizTalk orchestration.

• Publishing a schema as a WCF service. Publishing an orchestration as a WCF service binds the message received through the service to the published orchestration. However, publishing a schema as a WCF service provides a simple mechanism to submit messages to the MessageBox database. Once in the MessageBox, the message can be routed to any number of subscribers for processing.

• Publishing receive location metadata as WSDL. WCF services can implement a variety of protocols and communication mechanisms. Some WCF service protocols, such as WCF-NetMSMQ, do not natively expose WSDL information. In the case of the BizTalk in-process WCF adapters, you can use the WCF Publishing wizard to create WSDL that contains service endpoint metadata. Other WCF services can use the WSDL to determine how to communicate with the WCF receive location.

BizTalk Server 2009 continues to support integration with ASMX Web services in the following ways:

• Consuming a Web service. Similar to consuming a WCF service, BizTalk orchestrations can call external ASMX services.

• Publishing an orchestration as a Web service. Similar to publishing an orchestration as a WCF service, you can expose your business processes as ASMX services.

• Publishing a schema as a Web service. Similar to publishing a schema as a WCF service, schemas can be published as ASMX services, to expose the message to external services.

BizTalk Orchestration in SOA Designs

A key principle of SOA is workflow or business process automation, which happens to be a major strength of BizTalk Server 2009. Service aggregation is one of the most common SOA patterns that people use BizTalk Server to implement. Take a travel Web site for example; a travel Web site does not have a single database containing all the ticketing information for all the airlines in the world. Instead each participating airline exposes its fare information by using a Web service. The travel site executes the user's query against each airline's service and returns an aggregated set of results.

33

The Business Rules Framework

BizTalk Server 2009 includes the Business Rules Framework as a stand-alone .NET-compliant class library that includes a number of modules, support components, and tools. The primary modules include the Business Rule Composer for constructing policies, the Rule Engine Deployment Wizard for deploying policies created in the Business Rule Composer, and the Run-Time Business Rule Engine that executes policies on behalf of a host application.

You can integrate business rules into your orchestrations to support a variety of scenarios:

• Use rules instead of coding and recoding constantly changing business policies and logic within your complex business processes. Incorporate a call and allow information workers to update business rules.

• Use rules to evaluate business logic and to determine when a business process requires a variable delay. For example, you might set up a loop to check on the status of an item to determine whether the item is in stock. After initially checking the stock of an item that is not available, the rule delay would be one minute. The next time, the rule would wait five minutes before executing; the time after that, the rule would wait 30 minutes before executing; and so on.

• Use rules to determine the execution path for a business process, basing the determination on the results of the rule execution. For example, if a customer does not exist for a particular purchase order, you could route the document to another business process to add the customer to the database before continuing to process the purchase order.

Northwind Traders uses business rules in several places for the processing of orders. First, when the

restock orders are received from a store, a business rule policy is used to determine which supplier

provides the product being ordered and the conditions under which the order will be submitted to the

supplier. For example, some suppliers require that orders be placed with them on a weekly basis or

monthly basis while others require that orders be placed on the basis of a minimum order amount.

These business rules can be modified to dynamically change the conditions under which orders are

processed. Northwind Traders also uses business rules to compare the RFID tags attached to shipments

received at the warehouse against an electronic advance shipping notice that they have received from

the supplier.

The Business Rule Composer

The Business Rule Composer enables you to create rules by adding predicates and facts and defining actions. You can add facts and actions by dragging them to the Business Rule Composer design surface. The actions update the nodes in the specified document. You can also add AND, OR, and NOT operators to conditions to create complex comparisons.

The Business Rule Composer helps you create, test, publish, and deploy multiple versions of business rule policies and vocabularies to make the management of these artifacts easier. The Business Rule Composer can be used by developers, administrators, and information workers to develop and apply rules and policies.

34

Role Description

Developers • Create vocabularies to make it easier for information workers to edit and

understand business rule policies.

• Create the initial business rule policies.

• Bind business logic to data. Developers create the orchestrations from

which the business rules are called and define the action to be taken

when a decision is returned to the orchestration. As long as the decision

path within the orchestration does not change, you do not need to

redeploy the orchestration.

Administrators • Secure business rule policies. By default, when a new policy or

vocabulary is created in the rule store, only the user who created it and

the BRE administrator have both read/execute and modify/delete access.

The BRE administrator can configure which users have the access level,

or rights, to perform different operations.

• Deploy business rule policies from one physical environment to another.

This can be accomplished by using the Rule Engine Deployment Wizard.

Administrators can also export and import rules by using the MSI process.

Information

Workers • Manage business policies in real time.

Business Rule Vocabularies

Business rule vocabularies are user-defined names for the facts that you use in rule conditions and actions. Vocabulary definitions make rules easier to read, understand, and share among various workers within a particular business domain. For example, the source location for a particular fact is a specific field in one record in a single database and is represented as an SQL query. Instead of using the SQL query in the rule, you can use a vocabulary definition to assign a meaningful name with the query for the benefit of all the relevant parties in the development and deployment process for the rule.

Consider the following example of a business rule:

If the Shopping Cart contains more than $1,000 worth of items, give the customer a 10% discount.

This rule is easy to understand. It is a Boolean comparison (greater than) between two variables, the shopping cart and a value of 1,000. The action to be performed is to apply a 10 percent discount to the total order. In computer terms, this rule will look like this:

If Company.Namespace.ShoppingCart.PurchaseAmount > Qualifying Amount as System.Decimal

Then

Company.Namespace.Customer.DiscountedAmount = Company.Namespace.ShoppingCart .PurchaseAmount * .1

To provide a more user-friendly alias, you can create a business rule vocabulary to abstract difficult concepts by defining an alias to the schema nodes, database fields, and .NET classes. Vocabularies make the creation and maintenance of these rules much easier. Correctly defined vocabularies can empower information workers in maintaining policies.

The Business Rule Composer contains two built-in vocabularies, Predicates and Functions, which are used in the creation of all rules. You can extend these vocabularies if required. For example, the phrase “If the Shopping Cart contains more than $ 1,000 worth of items” is actually a greater than comparison (Shopping Cart > 1,000). You can define an additional vocabulary term to clarify the meaning of the relationship represented by the built-in Greater Than function.

35

Business Rule Policy

A business rule policy is a logical collection of business rules. Each rule is a conditional comparison of facts. If the comparison evaluates to True, the actions defined in the rule are executed. Business rules are versioned together as part of a business policy. Therefore, if a rule changes, you simply need to create a new version of the policy, test it, and deploy it. You do not need to recompile or modify orchestrations or other business processes that use that specific business policy.

Figure 9. Business rule definition in the BizTalk Business Rule Composer

When called from an orchestration, the BRE always executes the latest version of a policy. Changes made to a business rule policy are immediate. The next time the policy is called from an orchestration, the latest version number of the policy that is in a deployed state is used. When you call a policy programmatically, you need to specify the policy version.

Publishing the policy to the rule store makes it available to the BRE. By default, the BRE uses a Microsoft SQL Server database as the rule store; however, you can also export rules to an XML-based rule store as well. Once a policy is published, it is immutable and you can change it only by creating a new version.

Note: Although policies are typically used in conjunction with BizTalk orchestrations, you can also call

them from any .NET assembly by using the supplied APIs.

A business analyst or information worker might explain the logic of the rule in Figure 9 by saying: If the applicant's total monthly income is greater than the loan amount and he has either had the same job for at least six months, or lived in the same location for at least six months, then the loan should be approved.

It's important to notice that this rule alone does not contain all the logic used for loan evaluation. There may be other conditions for approving a loan, and there will be many different conditions for denying the loan as well. This rule is designed to be evaluated as part of a policy that contains other rules.

The Rule Engine Deployment Wizard

The Rule Engine Deployment Wizard provides an easy and efficient way to:

36

• Export a version of a policy or vocabulary from an SQL rule store to an XML file store. • Import a version of a policy or vocabulary from an XML file store to an SQL rule store. • Deploy a version of a policy to make it available to the update service and the rule engine

runtime. The wizard deploys a policy by marking it as “deployed” in the database. • Undeploy a version of a policy from a production SQL rule store to make the policy

unavailable for use by a rule-based application.

37

Business-to-Business Integration

BizTalk Server 2009 enables you to integrate trading partners into your existing business processes. A trading partner can be an external company or even a department within your own organization. BizTalk Server includes a number of capabilities to simplify the integration of your business processes with your trading partners and the management of trading partner relationships:

• Native support for EDI and AS2 protocols. BizTalk Server provides data exchange options including a native engine that provides integrated support for Electronic Data Interchange (EDI) data (including both X12, EDIFACT and HIPAA support) and Applicability Statement 2 (AS2) data for EDI over the Internet.

• Trading Partner Management. BizTalk Server provides a common storage database to store and manage all trading partner information. This information is maintained using the BizTalk Server Administration console, through the TPM Web Services (TPMgmtWS, TPMPubWS).

• BizTalk Server accelerators. Accelerators are used to integrate BizTalk Server with a broad spectrum of business scenarios and industries. Accelerators include specific BizTalk Server components, samples, and guides that help to reduce the time, effort, and costs associated with developing an industry-specific solution.

EDI and AS2 Processing

The EDI capabilities have been completely rewritten in BizTalk Server 2009. The BizTalk Server Administration console has a new EDI section for managing trading partner configuration and reporting and auditing of EDI message activity. The EDI parser/serializer uses the existing BizTalk pipeline architecture and supports both EDI and AS2 transactions.

Both inbound de-batching and outbound batching is supported. Batches can include different transaction sets and would be processed properly as long as the corresponding schemas are deployed. Batches can be initiated based on a schedule, based on number of messages received, or based on some kind of external trigger (for example, contents of a message).

BizTalk EDI and AS2 receive processing capabilities include:

• Parses the EDI interchange, processing batched transaction if configured

• Performs HIPAA document splitting

• Validates the message

• Generates the acknowledgment or acknowledgments

• Receives EDIINT/AS2 encoded messages over an HTTP/HTTPS transport.

• Re-assembles the interchange if the batch is to be preserved

BizTalk EDI and AS2 send processing capabilities include:

• Serializes the EDI interchange, batching transaction sets if configured

• Validates the message to be sent

• Sends EDIINT/AS2 encoded messages over an HTTP/HTTPS transport.

• Processes a received acknowledgment or acknowledgments to the message

Other functionality

• Provides the capability to set processing properties for parties engaging in EDI document exchange and AS2 document transport

38

• Provides a comprehensive status of EDI document exchange transactions through a list of EDI interchanges and their correlated acknowledgments

• Provides the capability to validate schemas, validate instances, and generate instances at design time

• Enables migration of BizTalk Server 2004 XSD “Attributes” in BizTalk Server 2004 XSD schemas (to “Elements”), BizTalk Server 2004 .xml, and BizTalk Server 2004 port-based party properties.

EDI Parties

A party is an entity outside of BizTalk Server that interacts with a BizTalk process. For example, each trading partner that you need to integrate with can be configured as a separate party with its own unique communication parameters. You must set properties for how BizTalk Server will receive an EDI message from, and send an EDI message to, the party. On its end, the party must do the same, and to exchange messages, the configurations must be compatible. Party properties determine the following specific processing:

• EDI envelope processing and generation

• ACK processing and generation

• Validation of incoming and outgoing EDI messages

• Batch creation

• Status reporting

You define a party in the BizTalk Server Administration console. You must define the following sets of properties for a party for EDI communications:

• Party properties that define general aspects of the party, such as name and aliases, send

ports, and the signing certificate.

• EDI properties that define how BizTalk Server will process an incoming message from the

party and how it will generate an outgoing message bound to the party.

• AS2 properties that define how BizTalk Server will perform AS2 communications, both

incoming and outgoing. These properties affect EDI communications only when an EDI

message is sent over AS2.

Any time BizTalk Server receives an EDI message, it attempts to determine the party that sent the message. It attempts to resolve the party by making a match between the message and the party using the sender qualifier, sender identifier, receiver qualifier, and receiver identifier. Any time BizTalk Server generates an EDI message to send, it attempts to determine the party that it is going to send the message to. It attempts to resolve the party by making a match between the message and the party using the context property DestinationPartyName, or the sender qualifiers and identifiers, and receiver qualifiers and identifiers, or the send port name.

Trading Partner Management

Maintaining the information required to manage trading partner relationships can be unwieldy when many organizations are involved or when the parties change frequently. BizTalk Server includes several services and tools to simplify the integration of your business processes with your trading partners and the management of trading partner relationships.

Some examples of trading partner integration scenarios:

• Modifying the business process criteria. Consider three separate shipping companies that ship orders depending on the order destination and the total order amount. You can integrate these three shippers into an orchestration by using role links, and configure the orchestration to dynamically select the correct shipper configuration based on the order criteria. To change the

39

criteria when a particular business process should use a specific shipper or add a fourth shipper, you can use the BizTalk Trading Partner Management (TPM) functionality without modifying the existing business process.

• Adding new partners. Consider a cosmetics company with hundreds of trading partners that supply necessary ingredients for its products. To add a new supplier, an information worker can easily input the required information and specify the agreements and rules for interacting with the supplier. You can make such changes in a central location instead of the hard coding the rules into the business processes.

• Processing student loans. Consider a company that processes student loans for thousands of universities that use its services. Each university is different and has different rules for interacting with the student loan processing company. In this situation, you can use TPM to configure and maintain trading partners.

Trading partner management and integration is important to the success of the Northwind Traders

BizTalk Server implementation. Each store, distribution center, and supplier has specific rules for how

they fit into the business process. For example, although each store places their orders through a

single SharePoint site, the order acknowledgments are sent via e-mail to each store's manager. During

the same instance of the process, an ASN is sent to the store, but this is a direct update to the

inventory system. Within this process each store must be treated like a separate entity with

communication that applies exclusively to them. The same is true of the distribution centers and

suppliers.

BizTalk Server Accelerators

Accelerators are used to support a broad spectrum of business scenarios and industries, from high tech to healthcare. Accelerators for BizTalk Server add additional functionality and vastly reduce the time, effort, and costs associated with solution development, deployment, and management. BizTalk Server accelerators include a powerful combination of product enhancements, simple-to-use tools, documentation, and samples that are developed in concert to ensure they work well together. This translates into rapid deployments, a lower overall cost of ownership, and improved efficiency.

Microsoft offers these BizTalk Server accelerators:

Accelerator Description

BizTalk Accelerator

for HL7

Provides a comprehensive HL7 messaging solution that enables sharing of patient

information within and between healthcare systems and organizations. For more

information, got to: http://go.microsoft.com/fwlink/?LinkId=140030.

BizTalk Accelerator

for SWIFT

Extends the capabilities of BizTalk Server for the financial services industry by

delivering cost-effective, reliable, and secure SWIFT messaging capabilities,

including message schemas and network connectivity. It will provide financial

institutions and corporate treasury departments with a single extensible infrastructure

for integration, both within the organization and also for integration with

counterparties and external service providers. For more information, go to:

http://go.microsoft.com/fwlink/?LinkID=79657.

40

Accelerator Description

BizTalk Accelerator

for RosettaNet

Helps you implement RosettaNet for your business. RosettaNet is a consortium of

major companies working to create an industry-wide approach to process standards

for open electronic business. These standards form a common business language

that helps to align the processes of supply chain partners on a global basis.

Implement a BizTalk Server environment customized to RosettaNet standards with

this accelerator. For more information, go to:

http://go.microsoft.com/fwlink/?LinkId=140031.

41

Building RFID Solutions

Radio Frequency Identification (RFID) is a data collection system based on tiny microchip tags attached to a box, pallet, or individual item that communicates with other devices by using radio waves. Device readers capture data from the tags and, in some cases, write to them as well. A software application then collects, organizes, and distributes this data. The combination of these RFID tags, sensors, and software technology can vastly improve supply chain operations and can dramatically improve operational efficiencies and customer service.

RFID offers many potential business and technology benefits:

• Supply chain visibility to shorten the order-to-cash cycle, prevent out-of-stock situations, and minimize inventory and safety stock levels

• Real-time visibility to support vendor-managed inventory programs and minimize counterfeiting by making it easier to identify fake products

• Lower cost of ownership

• Simplified end-to-end integration from the device level to back-end applications

• Conversion of data into actionable information

By using RFID, you can achieve greater control over inventory, gather more accurate production forecasting, reduce losses from counterfeiting and theft, and achieve more timely order fulfillment. The advantages that RFID offers over bar code systems include:

• Does not require the tags to be in the line of sight of the reader

• Enables the tags to be read in bulk almost simultaneously

• Makes tags carry more data than a bar code

• Enables automated reading

• Ensures extremely high data accuracy

• Identifies individual items whereas bar codes only identify classes of objects

• Makes the data more granular due to the potential for more frequent collection

• Enables an automatic count of tagged objects

• Makes the read/write tags receive new information throughout an item’s life cycle

Northwind Traders uses BizTalk RFID to reconcile incoming shipments to the advance ship notices

(ASNs) generated by its suppliers. The suppliers package their shipments in boxes with unique RFID

tags attached. When the box is shipped, the supplier sends an electronic ASN to Northwind Traders.

When the shipments are received at the Northwind Traders distribution center, the RFID tags are read

and a BizTalk RFID process correlates the tag information against the ASN to confirm that the shipment

has been received.

BizTalk RFID Infrastructure

BizTalk RFID enables the integration of disparate RFID devices from multiple vendors, the capacity to filter and manage collected data, and the ability to use collected data inside a business process. To encourage the widespread adoption of RFID technology, Microsoft has developed a layered RFID infrastructure and platform.

42

The BizTalk RFID infrastructure consists of several tightly integrated layers that interconnect to provide full integration between hardware devices and applications.

The following table describes the different layers in the RFID infrastructure:

Layer Description

Devices This is the bottom layer that consists of hardware such as RFID readers, printers,

sensors, barcode scanners, 802.1X access points for wireless Local Area Networks

(LANs), handheld terminals, and Pocket PCs, which are provided by partners.

Data collection and

management

This layer provides Device Service Provider Interface (DSPI) that provides a

consistent way for devices from multiple hardware vendors to expose their device

services to the Microsoft platform. DSPI provides a scalable, extensible

infrastructure that allows customers to read data through any standards- or non-

standards-based sensor, regardless of the format, and therefore reduces

dependency on a specific technology and protects long-term RFID investments.

Event processing

engine

This layer includes event and workflow management, messaging, and can use the

Business Rule Engine (BRE). It enables context- or rules-based processing of RFID

data to provide information directly to LOB applications or to business processes

that span applications via Web services integration or BizTalk orchestrations. In

addition, this layer provides the structure for integration across multiple facilities and

partners as well as includes device management to convert data into business-

process-relevant information.

Services This layer includes product-information-resolution lookup, business-process

management, analytics, reports, notifications, and enterprise content solutions.

Application solutions This uppermost layer relies on services, data, and tools from the lower layers to

implement application solutions that drive business processes for the end user.

Microsoft relies on its partners to build solutions that are divided between two

classes of applications, real-time enterprise/point apps and batch-oriented

enterprise apps.

43

Monitoring Business Activity

The most critical factor for the success of any business is getting the right information at the right time. The ease of information access can determine the fate of business deals and partnerships. One of the major incentives driving growth and demand for a new generation of integration solutions is the ability to provide both technical and non-technical users with end-to-end visibility into the business process on a near real-time basis. This improved visibility enables organizations to make timely and well-formed decisions to improve business agility and customer satisfaction.

Business Activity Monitoring (BAM)

Business Activity Monitoring (BAM) in BizTalk Server 2009 allows business analysts, developers, and information workers to monitor and analyze data from business process information sources in real time. By using BAM, users can get information about business state, trends, and critical conditions.

Additionally, the BAM application programming interface (API) enables developers to expose visibility into data that is external to BizTalk processes, such as archival data or other non-BizTalk processes and systems. Developers, administrators, business analysts, and end users can use the BAM Portal to view, aggregate, search, or create alerts based on the data collected by BAM.

Figure 10: Microsoft Office SharePoint can display KPIs and charts based on BizTalk BAM data

44

The Northwind Traders solution uses BAM to collect information from the restock process to display

up-to-the-hour information about the progress of orders and shipments. This information is displayed

using a custom ASP.NET Web site portal that can be accessed by store managers, warehouse managers,

sales staff, and business analysts at the corporate office.

BAM Activities

To gather the necessary business data, you need to create one or more BAM activities. A BAM activity represents a unit of work in a business, such as a purchase order or a loan application. BAM activities can also contain milestones, which are a date/time measure, throughout the business process that allow business analysts to see the overall progress of the business process and investigate each individual step of the process. BAM activities are independent of the actual implementation of your BizTalk solution. Think of a BAM activity as a record in an SQL table that you keep in synchronization with the actual unit of work.

You can relate multiple BAM activities together as well. An example of relating activities is a situation where a single purchase order contains multiple shipments. Properly configured BAM activities can allow you to view the shipping information for each item in the purchase order. You can use the BAM API to create BAM activities that span multiple disparate systems. For example, if one step in a loan process is to execute a Web service or make a call to a mainframe system, business analysts can include this data in their analysis.

Note: For more information on BAM activities, refer to “Using Business Activity Monitoring” and “About BAM Activities” in the BizTalk Server 2009 documentation.

BAM Views

Once you have defined the information that you want BAM to collect, you need to define how the data will be displayed or viewed. To do this you define a BAM view; the view defines the context for the information being collected.

In the Northwind Traders scenario the product ID and quantity for each product ordered is collected

from each restock order. This single piece of data has dozens of different meanings or contexts.

Northwind Traders uses different BAM views to display the information in the appropriate context for

each user to ensure that the data is meaningful to them.

For example:

The corporate analyst uses this information to evaluate the effectiveness of promotions and to

determine the popularity of a product based on geography.

Store mangers use this information to monitor ordering habits over time in order to be proactive about

ordering decisions, especially for popular products to ensure they are always in stock.

Warehouse managers use this to determine the amount of warehouse space required for incoming

shipments in order to optimize and to stream-line shipping logistics.

Because there are business processes that several different users will have an interest in it's important to be able to provide different contexts for the same data based on the user accessing it. A single BAM activity can have multiple views; there could be a Store Manager view, a Sales Associate view, and a Supplier view. While that actual data displayed is the same for all three views the way that it is displayed can change based on how the user wants to consume it.

45

Aggregating and Filtering Data

BAM provides interceptors that are used to gather data from incoming messages and from any point within the business process such as an orchestration, pipeline, or message type. By aggregating data, BAM provides an overview of business trends. You can also use BAM to link together various messages as they travel through the system to create a unified BAM view, which shows the entire business process and spans multiple orchestrations and data external to BizTalk Server. This visibility is tremendously beneficial for users making critical business decisions.

You can also filter the data received from BAM. This can be useful, for example, if you want to see loans that were processed from a certain state or by a certain loan officer or between two dates. Filtering allows business users to focus on only the data they require to make business decisions such as:

• How long did it take for this process to be approved?

• How quickly was this order filled after it was received?

• How many process cycles occurred in the last month? In the last year?

• How many purchase orders were processed last week?

• How much is our total revenue this year so far?

Note: You can share BAM databases across BizTalk groups to present an aggregated view of a

business process.

Gaining Better Visibility in SOA Solutions

A major goal for SOA is to provide total process visibility. In some cases, there are dozens or hundreds of different services, each of which plays its own part in a larger business process. Each one of these services is likely to have its own tracking or monitoring mechanism. By using BAM, a developer could create a single activity that spans across all of these processes and gathers the relevant information about each process. This kind of implementation provides a single view for information about the process as a whole rather than using different tools to look at disparate data stores to view information about a single service.

46

Management and Operations

After a BizTalk application has been developed and tested, it is typically handed off to the site administrator who deploys and manages the application in the production environment. BizTalk Server 2009 comes with improved tools to help you manage and monitor your BizTalk Server environment. These tools provide powerful query and reporting capabilities to assist in identifying the overall health of a BizTalk Server system.

The BizTalk Server 2009 Administration console is a Microsoft Management Console (MMC) snap-in which helps you create, configure, deploy, and manage applications as well as create and configure send ports, receive ports, and orchestrations. The administration console also contains the Group Hub, which provides a complete view into the health of a running BizTalk Server system. You can use the Group Hub to view live BizTalk Server instances, which can be in an active, dehydrated, or suspended state. You can also use a command-line option, BTSTask, to perform the same tasks as the BizTalk Server 2009 Administration console.

Deploying a BizTalk Application

A complete BizTalk Server 2009 solution typically contains various components such as:

• BizTalk artifacts such as orchestrations, schemas, maps, and pipelines

• Messaging components such as receive ports, receive locations, and send ports

• Supporting .NET assemblies such as helper applications and test harnesses

• Other related items such as security certificates, business rules policies, BAM configuration files, bindings, and scripts required by the application

Artifacts Managed as an Application

As the number of these components increases, managing them becomes cumbersome. To solve some of the management and deployment issues with these components, BizTalk Server 2009 formalizes the notion of a BizTalk application. A BizTalk application acts as a logical grouping of business solution components, simplifying their management and deployment. You can create one or more BizTalk applications to:

• Manage the artifacts and supporting configurations for separate business processes

• Isolate the applications for security and manageability

• Scale specific parts of the application to multiple BizTalk Server computers

The newly designed administration and monitoring tools in BizTalk Server 2009 enable you to be more efficient by managing and configuring BizTalk solutions per application and not just at the individual artifact level. You can view and manage all the artifacts and supporting configurations in an application as a single entity from within the BizTalk Server Administration console. This console provides a consolidated toolset of administration features that were previously contained in separate tools in the earlier versions.

47

MSI Packages

Deploying a BizTalk application involves deploying BizTalk assemblies into the BizTalk Configuration database and installing the assemblies into the global assembly cache (GAC) on each BizTalk Server computer. Developers usually deploy these assemblies from their own development environments to the development server, and then the site administrators deploy them from the development to test environments and further to production environments.

BizTalk Server 2009 simplifies the process of deploying BizTalk assemblies and moving deployed applications from one physical environment to another by enabling you to generate standard MSI packages by using the Microsoft Windows Installer technology. Using MSI packages eases the process of deployment upgrade and removal.

When you export a BizTalk application, it generates an MSI file that contains the application and some or all of its artifacts. You can specify the artifacts that should be exported. You can later import this MSI file into another BizTalk group to add the artifacts to an existing application in the new group, update the artifacts in an existing application, or create a new application in the group that contains the artifacts being imported.

A BizTalk application MSI package can contain:

• The BizTalk assemblies that make up the application, including schemas, maps, pipelines, and orchestrations

• The configuration information or bindings required to start the application on a computer

• Any non-BizTalk items, such as .NET assemblies

• Business rule policies to be processed by the BRE

• Any Web resource information required to enable Web-based applications

Using an MSI package to install a BizTalk application is beneficial for developers because they can control its contents and easily hand it off to administrators.

All servers in a BizTalk group share the BizTalk Configuration database, so when you deploy an MSI package to multiple physical servers, the assemblies are added only once to the Configuration database. However, all assemblies for a BizTalk application must also be added to a physical location on each server as well as to each server’s GAC.

You must execute the MSI on each server to facilitate this installation. Once an MSI package is installed, it creates an entry in the Add or Remove Programs list on the BizTalk Server computer. This allows administrators to manage BizTalk applications in the same way as other applications that have been installed on the computer.

Tracking and Debugging BizTalk Processes Using the Group Hub Page

The BizTalk Group Hub page is an operations management tool used for monitoring the running processes in a BizTalk group. As a fully integrated part of the BizTalk Server Administration console the Group Hub page is the first place that an administrator or business user should look to track or debug a running BizTalk process. The Group Hub page provides a view into the activity within the MessageBox database.

Figure 11. The BizTalk Group Hub

As shown in Figure 11, the Group Hub is divided into three sections that provide an overview of thehealth of your BizTalk Server system

• Configuration Overview. This sectionBizTalk group by displaying the number and state of Applications, Host Instances, and Adapter Handlers within the BizTalk of the health of these different

• Work in Progress and Suspended Items.

dehydrated orchestrations, retrying and idle ports, and running, reasuspended service instances.

• Grouped Suspended Service Instances.

displays summary links for viewing suspended service instances grouped in various ways. This grouping assists in managin

48

. The BizTalk Group Hub page

, the Group Hub is divided into three sections that provide an overview of thehealth of your BizTalk Server system:

This section provides information about the overall health of the BizTalk group by displaying the number and state of Applications, Host Instances, and Adapter Handlers within the BizTalk group. This section shows a red light or green light view

different items.

Work in Progress and Suspended Items. This section displays the summary links for viewing dehydrated orchestrations, retrying and idle ports, and running, ready, scheduled, and suspended service instances.

Grouped Suspended Service Instances. This section, the bottom section of the Hub page, displays summary links for viewing suspended service instances grouped in various ways. This grouping assists in managing suspended instances in a batch fashion. For instance, if

, the Group Hub is divided into three sections that provide an overview of the

the overall health of the BizTalk group by displaying the number and state of Applications, Host Instances, and

roup. This section shows a red light or green light view

This section displays the summary links for viewing dy, scheduled, and

This section, the bottom section of the Hub page, displays summary links for viewing suspended service instances grouped in various ways.

g suspended instances in a batch fashion. For instance, if

49

several messages are caused by the same error, they are grouped together here by the Error Code. Once the error has been resolved, all instances can be resumed (or terminated) at the same time.

An administrator of the Northwind Traders BizTalk solution uses the Group Hub page to manage the

BizTalk processing environment. For example, occasionally a message is received from a supplier that

cannot be processed by the receive location's pipeline. The message is suspended and waiting for

administrative action. Using the Group Hub, the administrator can be notified of the condition and

investigate the error. For example, the error message might indicate that a schema matching the

incoming message cannot be found. In the case of Northwind Traders, this usually happens when a new

supplier sends a message using the wrong schema format. The administrator can then notify the

supplier and terminate the message instance.

With many of out-of-the-box queries to help you quickly find the service instance you're looking for and the ability to create your own custom queries, the Group Hub page is a tool that both Administrators and Developers can't live without. Functionality of the Group Hub page includes:

• Queries. The Group Hub page provides you with several default queries that allow you to quickly view running, dehydrated, and suspended service instances. You can also create your own queries to further refine your tracking.

• Service instance visibility. Often times your queries will return suspended service instances, a service could be suspended for any number of reasons. You can use the Group Hub page to view errors, messages (including the message bodies and contexts), and running orchestrations.

• Orchestration Debugger. The Orchestration Debugger provides the same graphical view of the orchestration as the Visual Studio design experience but includes information on the execution of the orchestration shapes. This allows you to see the process flow of the individual instance such as which branch in a decision shape the instance processed. Additionally, you can set breakpoints on orchestration shapes. Once a breakpoint is set each instance of that orchestration will be caught in the breakpoint. You can attach to orchestration instances in breakpoints to see the specific variables and message data associated with the orchestration instance.

Monitoring BizTalk Applications Using MOM

The Microsoft BizTalk Server Management Pack for Microsoft Operations Manager (MOM) provides options for proactive and reactive monitoring of BizTalk Server computers. This management pack is provided as a download for users of BizTalk Server 2009.

The BizTalk Server management pack provides for comprehensive monitoring of BizTalk events and performance counters to provide a centralized management and monitoring experience for a BizTalk Server environment. MOM enables you to monitor BizTalk Server events, collect BizTalk Server-specific performance counters in one central location, and raise alerts that require operator intervention.

The BizTalk Server management pack includes the following features:

• Event and performance-based alerts for all major BizTalk Server components. All alerting rules contain product knowledge.

• Streamlined alerts. Redundant alerts are minimized and alert suppression is typically based on end points, especially for suspended messages in BizTalk Server.

• Rich views specific to the BizTalk Server 2009 management pack.

50

• State Monitoring view. The pack provides Green/Yellow/Red states for important health aspects of BizTalk Server and indicates overall health.

• BizTalk Server MOM tasks to execute common operational tasks.

• Additional MOM tasks to transition directly from a MOM alert to BizTalk Server Administration console.

• Comprehensive monitoring for the health of BizTalk Server MessageBoxes and hosts.

Implementing Load Balancing

BizTalk Server 2009 provides inherent load balancing through the use of hosts and host instances. A host in BizTalk Server is a named container for services which can be a receive location, an orchestration, or a send port. A host instance is the physical installation of a host on a BizTalk Server computer.

When you create a host instance on a BizTalk Server computer, it creates an NT service that is used to run the different processes associated with that host. Host instances can exist on multiple different BizTalk Server computers, providing the ability to easily scale the available resources for a process, as well as creating logical and physical boundaries for the runtime processes.

All servers that belong to a single BizTalk Server 2009 group (a group by definition is a set of servers which share a common BizTalk Server 2009 configuration and MessageBox database) can run an instance of each named host.

Northwind Traders has a total of four computers running BizTalk Server 2009 in their environment. To

take advantage of the load balancing capabilities in BizTalk Server, Northwind Traders created one

host for receive location processing, a separate host for send port processing, and another host for

orchestration processing. Instances of each host only run on certain BizTalk Server computers. For

example, the send host and the receive host run on the same BizTalk Server computer. However, since

the orchestration processing requires a greater amount of resources, the orchestration host runs on

three dedicated servers. If any one of these servers malfunctions, an instance of the appropriate host

can be created on one of the other servers. If additional processing resources are required, another

computer can be added to the BizTalk group and host instances can be created on it.

For improved performance and isolation, the BizTalk databases should be hosted on a dedicated server running SQL Server 2008. Other services, such as Windows SharePoint Services and Web services, should also run on dedicated computers. You can install the BizTalk Server administration tools on multiple workstations for distributed administration.

Securing Applications

BizTalk Server provides a standard gateway for sending and receiving documents both within the intranet and over the Internet. Most of these documents contain business-critical messages and therefore it is important to consider measures for securing these documents, both when they are being transmitted and when they are being processed and stored in the BizTalk Server environment.

These security measures include:

• Encryption and authentication. BizTalk Server 2009 supports XML industry standards as well as the use of Kerberos authentication and Public Key Infrastructure (PKI). In addition, several XML technologies have recently been enhanced to provide features such as authentication, encryption, and signing of messages. You can also secure network communication by using industry-standard Secure Sockets Layer (SSL) or Transport Layer Security (TLS).

• Logon credentials security. Enterprise Single Sign-On (SSO) allows the seamless integration of user information between disassociated security providers such as an IBM mainframe, UNIX computer, or some other system. To use SSO, an administrator defines an affiliate

51

application for the external system and defines an encrypted mapping of the credentials to a Windows ID. SSO helps solve the problems that administrators and users face with a proliferation of security credentials.

• SQL Server security. BizTalk Server is dependent on SQL Server 2008 or SQL Server 2005 for the messaging tracking database as well as other databases. You must plan the SQL Server implementation in a way that supports security for the BizTalk Server databases. The most sensitive information, such as credential information containing details of database connection strings and user names and passwords related to the BizTalk adapters, is stored in an encrypted format in the Single Sign-On (SSO) database.

52

Summary

BizTalk Server 2009 is now implemented by over 8,500 organizations worldwide that trust BizTalk Server for their mission-critical integration and business process automation solutions. BizTalk Server is designed for high volume, high availability, and scalability to an unlimited number of CPUs and can automate processes spanning many different vendor platforms.

BizTalk Server 2009 continues to build on the investments made to address service-oriented architecture and business process management solutions. It offers significant advancements in areas of connecting applications and businesses through enhanced support for Web services, integrated support for EDI protocols, enhanced business activity monitoring, and a comprehensive RFID platform infrastructure.