20131028 BTUG.be - BizTalk Tracking
-
Upload
btugbe -
Category
Technology
-
view
1.134 -
download
0
description
Transcript of 20131028 BTUG.be - BizTalk Tracking
![Page 1: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/1.jpg)
BTUG – BizTalk TrackingToon Vanhoutte – Integration Architect
![Page 2: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/2.jpg)
2
Configuration
Under the hood
Best Practices
Demo Visualization
Tools
Agenda
![Page 3: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/3.jpg)
3
SetupDemo
Specify unique reference for messages #search
Specify simulated response times of backend systems #sla
Specify customer and transporter names #inspiration
![Page 4: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/4.jpg)
4
Configuration
![Page 5: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/5.jpg)
5
| Port Tracking Configuration
Configuration
Only track what’s really required #performance
![Page 6: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/6.jpg)
6
| Pipeline Tracking Configuration
Configuration
Bug BTS2010 CU2 CU5 #solved #CU6 #kb2728652
![Page 7: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/7.jpg)
7
| Orchestration Tracking Configuration
Configuration
Avoid Shape start and end #performance #orchestrationdebug
![Page 8: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/8.jpg)
8
| Property Schema Tracking Configuration
Configuration
Make context properties searchable #troubleshooting
Global + specific property schema #generic #consistent
![Page 9: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/9.jpg)
9
| Disable Global BizTalk Tracking
Configuration
![Page 10: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/10.jpg)
10
Under the hood
![Page 11: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/11.jpg)
11
| BizTalk Database Structure
Under the hood
11
SQL Agent
Host Instance processes message
![Page 12: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/12.jpg)
12
| BizTalk Database Structure
Under the hood
12
SQL Agent
Host Instance processes message
Message stored in MessageBox
![Page 13: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/13.jpg)
13
| BizTalk Database Structure
Under the hood
13
SQL Agent
Host Instance processes message
Message stored in MessageBox
TDDS copies events to DTA + BAM
![Page 14: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/14.jpg)
14
| BizTalk Database Structure
Under the hood
14
SQL Agent
Host Instance processes message
Message stored in MessageBox
TDDS copies events to DTA + BAM
SQL Agent copies bodies to DTA
![Page 15: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/15.jpg)
15
| Store messages in MessageBox database| Physical runtime message bodies are stored in:
| Spool| MessageParts| Parts| Fragments
Under the hood
15
Fragmentation controlled by large message size
![Page 16: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/16.jpg)
16
| Process messages in MessageBox database| Per Host, these tables are created:
| HostNameQ: Main queue: each host instance polls this queue table to collect messages that are pending processing
| HostNameQ_Suspended: Suspended queue: queue table that holds supended messages
| HostNameQ_Scheduled: Scheduled queue: not used| InstanceStateMessageReferences_Host: State queue:
holds list of messages that have been processed but are required later
Under the hood
16
Stores only reference to physical message in Spool table
![Page 17: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/17.jpg)
17
| Store messages in MessageBox database
Under the hood
17
#probts2006
![Page 18: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/18.jpg)
18
| Store tracking data in MessageBox database| Host Instance serializes data and stores it in the
MessageBox| Message events:
| TrackingData_0_x [x: 0 - 3]| TrackingData_1_x [x: 0 - 3]
| Message bodies:| Spool| MessageParts| Parts| Fragments
Under the hood
18
#probts2006
![Page 19: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/19.jpg)
19
| Copy Events from MessageBox to DTA database
| Tracked Message Events copied by TDDS Service:| Tracking Data Decode Service| Runs in each host instance with tracking enabled
Under the hood
19
Create a dedicated tracking host
![Page 20: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/20.jpg)
20
| Copy Events from MessageBox to DTA database
Under the hood
20
BizTalk MessageBoxTrackingData_0_0TrackingData_0_1TrackingData_0_2TrackingData_0_3
TrackingData_1_0TrackingData_1_1TrackingData_1_2TrackingData_1_3
BAM Primary ImportActivity_ActiveActivity_ActiveRelationshipsActivity_CompletedActivity_CompletedRelantionshipsActivity_Continuations
BizTalk DTA DatabaseDta_MessageInOutEventsDta_ServiceInstances...
TDDS
![Page 21: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/21.jpg)
21
| Copy Events from MessageBox to DTA database
Under the hood
21
BizTalk MessageBoxTrackingData_x_x
BAM Primary ImportTDDS_StreamStatus TDDS_FailedTrackingData
BizTalk DTA DatabaseTDDS_StreamStatus TDDS_FailedTrackingData
TDDS
TDDS_StreamStatus contains last copied sequence n°
TDDS moves events with > sequence n° from TrackingData_x_x to DTA
TDDS_FailedTrackingData contains tracking failures
![Page 22: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/22.jpg)
22
| Copy Bodies from MessageBox to DTA database
| Tracked Message Bodies copied by SQL Agent Job:| TrackedMessages_Copy_BizTalkMsgBoxDb| Scheduled by default each minute (development: 10
seconds)
Under the hood
22
Monitor the SQL Agent Jobs #SCOM #blog
![Page 23: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/23.jpg)
23
| Copy Bodies from MessageBox to DTA database
Under the hood
23
BizTalk MessageBoxSpoolMessagePartsPartsFragments
BizTalk DTA DatabaseTracking_Spoolx Tracking_Partsx Tracking_Fragmentsx
SQLAgent
[x: 1 - 2]:Always one online / one offline
![Page 24: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/24.jpg)
24
| Remove data from MessageBox| This is performed by SQL Agent Jobs:
| MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb:| Runs continuously (due to former SQL limitation of 1 min job
schedule)| This jobs checks weather a message is still referenced by instances
| MessageBox_Message_Cleanup_BizTalkMsgBoxDb| Is disabled (keep it like this)| It’s called from with the infinite loop of the ManageRefCount Job| This job deletes physical messages from MessageBox that have no
outstanding references anymore
Under the hood
24
![Page 25: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/25.jpg)
25
| Remove data from DTA| Enable + Configure the DTA Purge and Archive Job
#avoidlargeDTA
| Remove data from BAM| SQL Agent Job BAM_DM_Activity moves data to BAMArchive
Under the hood
25
![Page 26: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/26.jpg)
26
| BizTalkDTADb Database model| No foreign keys #performance
| Designed for fast insert| Results in slower query
| No single chain ID| Difficult to reproduce the “route” within BizTalk
Under the hood
26
![Page 27: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/27.jpg)
27
| BizTalkDTADb Database model| No foreign keys #performance
| Designed for fast insert| Results in slower query
| No single chain ID| Difficult to reproduce the route within BizTalk
Under the hood
27
![Page 28: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/28.jpg)
| Create a chain of messages
Under the hood
![Page 29: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/29.jpg)
29
| Create a chain of messages
Under the hood
29
SI = 1MI = A
SI = 1MI = B
SI = 2MI = B
SI = 2MI = C
SI = 3MI = C
SI = 3MI = D
dta_MessageInOutEvents Service Instance ID Message Instance ID
![Page 30: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/30.jpg)
30
Tools & SDK
30
![Page 31: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/31.jpg)
31
| BizTalk Administration Console #technical #notsexy
Tools & SDK
31
![Page 32: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/32.jpg)
32
| BizTalk MessageBox Viewer #troubleshooting
Tools & SDK
32
![Page 33: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/33.jpg)
33
| BizTalk Terminator #troubleshooting
Tools & SDK
33
Import MBV scripts
![Page 34: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/34.jpg)
34
| Performance Counters #troubleshooting| Throttling state find reason on MSDN| TDDS
Tools & SDK
34
![Page 35: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/35.jpg)
35
| Retrieve tracked message body #3ways| Microsoft.BizTalk.Operations.dll
Tools & SDK
35
![Page 36: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/36.jpg)
36
Visualization
36
![Page 37: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/37.jpg)
37
Visualization
37
| Bad practice| Develop a dashboard directly on BizTalkDTADb
#performance| No control on user queries| Impacts directly the BizTalk runtime| Bad dashboard performance
![Page 38: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/38.jpg)
38
Visualization
38
| Bad practice| Develop custom tracking mechanism
![Page 39: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/39.jpg)
39
Visualization
39
| Good practice| Use default BizTalk Tracking #CID
![Page 40: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/40.jpg)
40
Visualization
40
| Good practice| Use BizTalk Tracking #CID
![Page 41: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/41.jpg)
41
Visualization
41
| Business Activity Monitoring? #notideal| Development impact
| Tracking Profile Editor is limited| BAM API is rather complex
| SQL Server Analysis Services| 2008 R2 required, when using SQL 2012
| Deprecated BAM portal| Impact on operations| Not ideal OLAP cube structure| Termination of activities not straightforward
![Page 42: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/42.jpg)
42
Visualization
42
| Use case #1: Message Search| BizTalk Administration Console
| Codit Integration Dashboard
![Page 43: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/43.jpg)
43
Visualization
43
| Use case #1: Message Search| Search on context properties:
| Application| Flow| Sender| Receiver| Reference
Demo
![Page 44: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/44.jpg)
44
Visualization
44
| Use case #1: Message Search
Demo
![Page 45: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/45.jpg)
45
Visualization
45
| Use case #2: Operational reports | Message Load| Message Size| Database Size
#managedservices
![Page 46: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/46.jpg)
46
Visualization
46
| Use case #3: Pro-Active Monitoring| Custom SCOM alert: monitoring non-events #vbs
| E.g.: Just-In-Time ordering| Daily check on expected amount of received orders| Query on BizTAlkDTADb| Automated email alert / support ticket
#managedservices
![Page 47: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/47.jpg)
47
Visualization
47
| Use case #4: SLA Reporting| Processing times of web services
| Total processing time| Processing time of every backend system| Processing time in BizTalk
![Page 48: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/48.jpg)
48
Visualization
48
| Use case #4: SLA Reporting
Demo
rp_MasterDataService.WCF
sp_CRM_GetCustomer.WCF
sp_SAP_GetItem.WCF
sp_AX_GetEmployee.WCF
ORCH
GetCustomerGetCustomerItemList
GetItemGetEmployee
![Page 49: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/49.jpg)
49
Visualization
49
| Use case #4: SLA Reporting
Demo
rp_MasterDataService.WCF sp_SAP_GetItem.WCF
Tota
l Pro
cess
ing
Tim
e SAP Processing Time
BizTalk Processing Time = Total Processing Time – SAP Processing Time
1 2 3 4
8 7 6 5
Same MessageID
Same MessageID
![Page 50: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/50.jpg)
50
Visualization
50
| Use case #4: SLA Reporting| SQL Agent Job
| BizTalkDTADB Custom database #chaining
Demo
![Page 51: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/51.jpg)
51
Visualization
51
| Use case #4: SLA Reporting
Demo
![Page 52: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/52.jpg)
52
Visualization
52
| Use case #4: SLA Reporting
Demo
![Page 53: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/53.jpg)
53
Visualization
53
| Use case #5: Process Overview| Visualization of related integration flows
![Page 54: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/54.jpg)
54
Visualization
54
| Use case #5: Process Overview| Example: EDI Order Processing
Demo
![Page 55: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/55.jpg)
55
Visualization
55
| Use case #5: Process Overview
Demo
![Page 56: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/56.jpg)
56
Visualization
56
| Use case #6: Dashboards| BizTalk 360: Operational focus| AIMS for BizTalk: Behavior / trend analysis| Codit Integration Dashboard: Advanced message
handling| Minotaur| Frends Helium| Moesion| …
![Page 57: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/57.jpg)
57
Visualization
57
| Use case #6: Dashboards| Codit Integration Dashboard
| Friendly Naming of technical artifacts| Message Search| Message Resume / Resubmit
| Editing| Multi selection
Demo
![Page 58: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/58.jpg)
58
Visualization
58
| Use case #6: Dashboards| Codit Integration Dashboard: resubmit
Demo
![Page 59: 20131028 BTUG.be - BizTalk Tracking](https://reader035.fdocuments.net/reader035/viewer/2022062303/554c4900b4c90530668b559f/html5/thumbnails/59.jpg)
59
Questions?
59
You
BTUG
Microsoft