Enterprise Data Validation Architecture...
Transcript of Enterprise Data Validation Architecture...
© 2007 The MITRE Corporation. All rights reserved
Enterprise Data Validation Architecture (EDVA):
Fixing Data Quality for Enterprise Interoperability
Cambridge, Massachusetts, USAJuly 18-19, 2007
Bill McMullen
The MIT Information Quality Industry Symposium, 2007
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 798
2
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Outline•
Purpose•
Nature of the Problem•
Elements of a Solution•
Business Case Considerations•
Q&A
Department of Defense
Enterprise Architecture Federation Strategy
Draft Version 1.0
18 September 2006
Prepared by the DoD CIO
Department of Defense
Enterprise Architecture Federation Strategy
Draft Version 1.0
18 September 2006
Prepared by the DoD CIO
Business Enterprise Architecture (BEA) & Enterprise Transition Plan (ETP)
September 2006
An Objective – “Transform the Department’s supply chain information environment by: 1) improving data integrity and visibility by
defining, managing, and utilizing item, customer, and vendor master data; and 2) reducing
complexity and minimizing variability on the supply chain business transactions by adopting standardized transaction and business rules.”
Business Enterprise Architecture (BEA) Business Enterprise Architecture (BEA) & Enterprise Transition Plan (ETP)& Enterprise Transition Plan (ETP)
September 2006September 2006
An Objective An Objective –– ““Transform the DepartmentTransform the Department’’s s supply chain information environment by: supply chain information environment by: 1) 1) improving data integrity and visibility by improving data integrity and visibility by
defining, managing, and utilizing item, customer, defining, managing, and utilizing item, customer, and vendor master dataand vendor master data; and 2) reducing ; and 2) reducing
complexity and minimizing variability on the complexity and minimizing variability on the supply chain business transactions by adopting supply chain business transactions by adopting standardized transaction and business rules.standardized transaction and business rules.””
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 799
3
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Purpose
•
Purpose of BriefingProvides an overview of an Enterprise Data Validation Engine
(This may be an important management option for certain business cases.)
•
Purpose of an Enterprise Data Validation Engine–
Aligns data across previously independent legacy system sources –
Eliminates routine manual reconciliation efforts previously needed to coordinate legacy data sources
–
Improves decision making at the domain and/or enterprise levels through enhanced legacy data quality
–
Can be applied in many ways across the enterprise
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 800
4
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Decision Making Capability for the Next Decade
•
ERP promises highly accurate data and superior decision making ability, but a wait is involved
–
ECSS is expected to be deployed about 2011+•
It is not necessary to wait to increase data and decision integrity–
Selected “legacy”
systems, with minimal effort, can provide more integrity in the
meantime
–
Global Logistics Support Capability (GLSC) requires reengineering both the business processes and their supporting systems
•
It is possible to make improvements without distracting from ECSS-ERP–
The proposed approach will actually facilitate the migration of valid data to ECSS
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 801
5
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Nature of the Problem Regarding Enterprise Data Validation •
The Business Problem–
Data from separate legacy systems frequently can not reliably be
combined at domain or enterprise levels
–
When combined, may not yield trustworthy results for quality enterprise decision making
–
Examples •
Case #1: 29% mismatch among two requisition systems (re GLSC)–
“Almost 30 percent of Air Force backorder data is inaccurate.”
–
“Forty-two percent of in-transit records … were invalid. This equates to 4,627 transactions, valued at $325M.”
•
Case #2: 98% failure to integrate databases across domain•
Case #3: 25% mismatch (est.) among several vehicle systems
–
Manual reconciliation to compensate is not satisfactory in today's environment (AFSO21)
•
Airman resources no longer will be available to “fix” data (re GLSC)
•
The Technical Issues–
Synchronization of multiple source system data–
Association of multiple source data
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 802
6
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
The Systems Gap: Misaligned Data Leads to Questions Integration blends the data from the
multiple source systems into a global “mismatched” view.
Issue – Time and Instance Integrity Across SystemsProblem – Reporting Produces Inconsistencies(Q: How much can we trust this?)
Source systems often describe different aspects of the same asset.
D3
E3
F3
G3
H3
I3
J3
K3
L3
M3
B2
C2
D2
E2
G2
H2
I2
J2
K2
L2
A1
B1
C1
D1
I1
J1
K1
L1
M1
A
B
C
D
E
F
G
H
I
J
K
L
M
Note: The Synchronization Issue shows Note: The Synchronization Issue shows up in a database when data from multiple up in a database when data from multiple source systems does not correspond source systems does not correspond across the sources in a logical manner, across the sources in a logical manner, or as expected, given the business rules.or as expected, given the business rules.
Data for Vehicles Within 3 Systems
Maintenance Activity
Parts Inventory
Vehicle Inventory
OLVIMS
SBSSAFEMS
Maintenance Activity
Parts Inventory
Vehicle Inventory
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 803
7
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Enforce Business Rules & Link Records
•
Check against enterprise business rules•
Make corrections at this point or they usually won’t (can’t) get done later
D3
E3
F3
G3
H3
I3
J3
K3
L3
M3
B2
C2
D2
E2
G2
H2
I2
J2
K2
L2
A1
B1
C1
D1
I1
J1
K1
L1
M1
A1
B1 B2
C1 C2
D1 D2 D3
E2 E3
F3
G2 G3
H2 H3
I1 I2 I3
J1 J2 J3
K1 K2 K3
L1 L2 L3
M1 M3
Cross-Reference Information
(GFM-DI)
Note: The Association Issue shows up in a federated database wheNote: The Association Issue shows up in a federated database when n invalid associations among object is erroneously permitted. invalid associations among object is erroneously permitted. Validation against business rules must be performed Validation against business rules must be performed prior to a source prior to a source system allowingsystem allowing such a transaction to update its own database. such a transaction to update its own database.
Aircraft Information
(ForceTab)
Organizational Information
(MPES)
Equipment Information (SBSS & AFEMS)
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 804
8
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Problems Compound as More Systems Are Considered
Compounding of Missing, Mismatched, or Unsynchronized Data
(Due to improperly associated and serialized enterprise-wide data and enforcement of enterprise-wide business rules)
Large
Ente
rpris
eD
ecis
ion
Mak
er’s
Abi
lity
to U
se D
ata
MediumSmall
Good
Very Good
Very Poor
Poor Notional
Notional
CAUTION
* * Only the topic of Object Only the topic of Object Association is addressed Association is addressed hereafter, because the hereafter, because the solution to this single solution to this single problem facilitates the problem facilitates the solution of Attribute solution of Attribute Synchronization (even Synchronization (even though its solution can be though its solution can be attempted separately).attempted separately).
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 805
9
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Some Elements of a Solution for Using an Enterprise Data Validation Engine
Validation Engine
System A System B
Object X
Object Y
Associative Cross-
Reference
New business rules must be enforced by
Systems A and B.
Determine how different objects of
two systems are associated.
Note: EII usually requires that Note: EII usually requires that objects in separate database (i.e., objects in separate database (i.e., Objects X and Y above) be Objects X and Y above) be logically related according to logically related according to certain enterprise business rules. certain enterprise business rules.
Associations Across Systems
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 806
10
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Classified / SIPRNetUnclassified / NIPRNet
AFPortal
Data Services
Exchange
Tools
Enterprise Data Warehouse
AFPortal
Guard
GCSS-AF Infrastructure
BI Tools
AF GLSC Archive
Metadata
AF GLSC
Discovery
GLSC Data Translator
(Up to AOC and Up from Legacy)
Authoritative Sources (Unclassified)
ES-S
AFEMS
Others
...
Data Services
Enterprise Data Warehouse
BI Tools
Metadata
AF GLSC
Authoritative Sources (Classified)
ES-S
AFEMS
Others
... Joint StaffAOC
GLSC
GCSS-AF InfrastructureFramework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)Framework & Enterprise Services Bus (EAI, Web Services, XML, etc.)
Joint StaffAOC
GLSC
BoM
GLSC Data Loading
Exchange
ToolsDiscovery
GLSC Data Translator
(Up to AOC and Up from Legacy)
GLSC Data Loading
HAFMAJCOM
NAF NAFWING WING
SG
SQN
OG
SQNFLT FLT
CREW CREW
BILLET BILLET
HAFMAJCOM
NAF NAFWING WING
SG
SQN
OG
SQNFLT FLT
CREW CREW
BILLET BILLET
BoM
AF GLSC AF GLSCAF GLSC Archive
Viewers (e.g., FSC) Viewers (e.g., FSC)
Leveraging the Air Force Architecture
Framework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)
ES-S AFEMS etc.
Adapter MAdapter M Adapter AAdapter A Adapter etc.Adapter etc.
Change Change Identification Identification & Notification & Notification
for Mfor M
Change Change Identification Identification & Notification & Notification
for Afor A
Change Change Identification Identification & Notification & Notification
for etc.for etc.
Enterprise Enterprise Data Data
Validation Validation Engine for Engine for
MM
Enterprise Enterprise Data Data
Validation Validation Engine for Engine for
AA
Enterprise Enterprise Data Data
Validation Validation Engine for Engine for
etc.etc.
CrossCross--Reference Reference for Associationsfor Associations
Previous Snapshot for Previous Snapshot for NonNon--Intrusion SystemsIntrusion Systems
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 807
11
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
•
Leads to greatly enhanced data agreement across systems that currently lack common standards or interoperability
•
Enterprise Data Validation Engine ensures data integrity as it trickles in from the various systems
•
Enterprise Data Validation Architecture –
Components include•
Cross-Reference for Associations•
Enterprise Data Validation Engine•
Change Identifier and Notifying•
Interoperability Protocols (7 possible choices)[Database Snapshot for Non-Intrusion*]
* Can be non-intrusive to source system, but with critical tradeoffs
•
Performance•
Data Quality
Source System Architecture Using the Enterprise Data Validation Engine
Framework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)
ES-S AFEMS etc.
Adapter MAdapter M Adapter AAdapter A Adapter etc.Adapter etc.
Change Change Identification Identification & Notification & Notification
for Mfor M
Change Change Identification Identification & Notification & Notification
for Afor A
Change Change Identification Identification & Notification & Notification
for etc.for etc.
Enterprise Enterprise Data Data
Validation Validation Engine for Engine for
MM
Enterprise Enterprise Data Data
Validation Validation Engine for Engine for
AA
Enterprise Enterprise Data Data
Validation Validation Engine for Engine for
etc.etc.
This architecture is an adaptation from “Protocols for Integrity Constraint
Checking in Federated Databases”, P. Grefen
and J. Widom, 1997
CrossCross--Reference Reference for Associationsfor Associations
Previous Snapshot for Previous Snapshot for NonNon--Intrusion SystemsIntrusion Systems*
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 808
12
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Data Entry Scenario Where Legacy Modifications Allowed
IDID modelmodel
serial numberserial number
xxxxxx--xxxxxx
AddressAddressLine 1Line 1Line 2Line 2Line 3Line 3
zzzzzz--zzzzzz
yyyyyy--yyyyyy
Current Legacy Data Entry Process
Improved Legacy Data Entry Process
Local Store
IDID -- -- -- -- --
-- -- -- -- -- -- -- -- -- -- --
-- -- -- -- --
-- -- -- -- -- -- ---- -- -- -- ---- -- -- -- ---- -- -- -- --
-- -- -- -- --
-- -- -- -- --
Cross- Reference
Check for Check for existenceexistence
IDID modelmodel
serial numberserial number
-- -- -- -- --
AddressAddressLine 1Line 1Line 2Line 2Line 3Line 3
-- -- -- -- --
-- -- -- -- --
If already existing: If already existing: populate formpopulate form
IDID modelmodel
serial numberserial number
xxxxxx--xxxxxx
AddressAddressLine 1Line 1Line 2Line 2Line 3Line 3
zzzzzz--zzzzzz
yyyyyy--yyyyyy
Local Store
Finish Finish data entrydata entry
If not already existing: If not already existing: continue as beforecontinue as before
1. Initial Entry Screen1. Initial Entry Screen 2. Pre2. Pre--Populate ScreenPopulate Screen 3. Remaining Data Entry3. Remaining Data Entry
ERPERP--newnew
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 809
13
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Business Case Considerations
•
Program-Centric Aspects–
Development Effort and Time [Investment]•
Added Effort–
Systems Aspects (enhanced data integrity capabilities vs. system revisions)–
Operational Aspects (elimination of reconciliation activities vs. data entry revisions)
•
Coordination Among Other Stakeholders (via a COI?)
–
Lifecycle Benefits (may accrue over time) [Return]•
Improved Data Integrity•
Improved Data Timeliness•
Eliminate Redundant Data Entry•
Eliminate Reconciliation Effort
•
Enterprise-Centric Aspects (in addition to above)–
Generic Solution Template–
Prerequisite to ERPs (DEAMS, ECSS, DIMHRS) Migration–
Improved Integrity of Enterprise-Level Decision Making
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 810
14
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Considerations for Use of Enterprise Data ValidationWhere would the Enterprise Data Validation Engine be considered for use?
After conducting Enterprise-Level Cross-System Data Quality assessments for given products, then perform the following Data Fixing Strategy as applicable.
RuleSituation
1 2 3 4 5 6 7 8
Can Data Be “Abandoned”?
Yes No No No No No No
Is Current Quality OK? - Yes No No No No No
Feasibly to Re-Entered?
- - Yes No No No No
Feasible to Reconcile? - - - Manually Auto-
matically Auto-
matically Auto-
matically
Can Apply Fix at Migration Time?
- - - - Yes No No
Fixing Sources Cost Acceptable?
- - - - - Yes* No
Data Fixing Strategy
Do Nothing
Do Nothing
Manually Re-Entry
Manually Reconcile
Do Data Cleansing
Enforce Enterprise Data
Validation???
This issue is similar to the Y2K problem in that you can’t wait until the last minute to think about or act upon. It has the additional characteristic that the sooner it is addressed the better the data quality gets over time.
This issue is This issue is similar to the similar to the Y2K problem in Y2K problem in that you canthat you can’’t t wait until the wait until the last minute to last minute to think about or think about or act upon. It has act upon. It has the additional the additional characteristic characteristic that the sooner that the sooner it is addressed it is addressed the better the the better the data quality gets data quality gets over time.over time.
Indicates Inherent Indicates Inherent Government Government ResponsibilitiesResponsibilities
Not
e: T
he te
rm
Not
e: T
he te
rm ““
data
data
””he
rein
incl
udes
any
dat
a ag
greg
atio
n he
rein
incl
udes
any
dat
a ag
greg
atio
n (e
lem
ent,
reco
rd, f
ile, c
ross
(ele
men
t, re
cord
, file
, cro
ss-- s
yste
m c
ompo
site
, etc
.).sy
stem
com
posi
te, e
tc.).
Data Quality ImprovementsData Quality Improvements
* If interfaces are * If interfaces are being changed being changed anyway, then anyway, then these costs may these costs may be negligible to be negligible to use EDVA use EDVA template.template.
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 811
15
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Questions and Answers
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 812
16
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Backup Slides
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 813
17
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Associations Between Objects
Validation Engine
System A System B
Object X
Object Y
Associative Cross-
Reference
Thus, new synchronization business rules must
be enforced on Systems A and B.
Tx means transaction.
Federating among systems can require that different objects
of two systems be associated.
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 814
18
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Classified / SIPRNetUnclassified / NIPRNet
AFPortal
Data Services
Exchange
Tools
Enterprise Data Warehouse
AFPortal
Guard
GCSS-AF Infrastructure
BI Tools
AF GLSC Archive
Metadata
AF GLSC
Discovery
GLSC Data Translator
(Up to AOC and Up from Legacy)
Authoritative Sources (Unclassified)
ES-S
AFEMS
Others
...
Data Services
Enterprise Data Warehouse
BI Tools
Metadata
AF GLSC
Authoritative Sources (Classified)
ES-S
AFEMS
Others
... Joint StaffAOC
GLSC
GCSS-AF InfrastructureFramework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)Framework & Enterprise Services Bus (EAI, Web Services, XML, etc.)
Joint StaffAOC
GLSC
BoM
GLSC Data Loading
Exchange
ToolsDiscovery
GLSC Data Translator
(Up to AOC and Up from Legacy)
GLSC Data Loading
HAFMAJCOM
NAF NAFWING WING
SG
SQN
OG
SQNFLT FLT
CREW CREW
BILLET BILLET
HAFMAJCOM
NAF NAFWING WING
SG
SQN
OG
SQNFLT FLT
CREW CREW
BILLET BILLET
BoM
AF GLSC AF GLSCAF GLSC Archive
Viewers (e.g., FSC) Viewers (e.g., FSC)
Vision Architecture
(using GLSC example)
AF GLSC Data Adaptors connect the authoritative sources to the ESB and provide various mediation services
including format and protocol translation, mapping and translation.
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 815
19
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Source System Architecture
Authoritative Sources
ES-S
AFEMS
Others
...
Framework & Enterprise Services Bus (EAI, Web Services, XML, etc.)
Framework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)
ES-S AFEMS etc.
Adapter MAdapter M Adapter AAdapter A Adapter etc.Adapter etc.
Change Change Identification Identification & Notification & Notification
for Mfor M
Change Change Identification Identification & Notification & Notification
for Afor A
Change Change Identification Identification & Notification & Notification
for etc.for etc.
Enterprise Enterprise Data Data
Validation Validation for Mfor M
Enterprise Enterprise Data Data
Validation Validation for Afor A
Enterprise Enterprise Data Data
Validation Validation for etc.for etc.
CrossCross--Reference Reference for Associationsfor Associations
Previous Snapshot for Previous Snapshot for NonNon--Intrusion SystemsIntrusion Systems
CrossCross-- Enterprise Enterprise (functional) (functional)
SMEsSMEs (COI?)(COI?)
andandSource Source SystemSystemSMEsSMEs
Source Source SystemSystemSMEsSMEs
GCSSGCSS--AFAF(technical) (technical)
SMEsSMEs(with AFKS (with AFKS mappings)mappings)
(expertise)
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 816
20
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Source System Architecture
Authoritative Sources
ES-S
AFEMS
Others
...
Framework & Enterprise Services Bus (EAI, Web Services, XML, etc.)
Framework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)
ES-S AFEMS etc.
Adapter MAdapter M Adapter AAdapter A Adapter etc.Adapter etc.
Change Change Identification Identification & Notification & Notification
for Mfor M
Change Change Identification Identification & Notification & Notification
for Afor A
Change Change Identification Identification & Notification & Notification
for etc.for etc.
Enterprise Enterprise Data Data
Validation Validation for Mfor M
Enterprise Enterprise Data Data
Validation Validation for Afor A
Enterprise Enterprise Data Data
Validation Validation for etc.for etc.
CrossCross--Reference Reference for Associationsfor Associations
Previous Snapshot for Previous Snapshot for NonNon--Intrusion SystemsIntrusion Systems
Conceptual ModelConceptual Model of of ““CommonCommon””
Objects and AttributesObjects and Attributes
EnterpriseEnterprise-- Level Business Level Business
Rules for Rules for How Source How Source
System System Relates to Relates to
““CommonCommon”” Objects and Objects and AttributesAttributes
Source System Source System Objects and Objects and
Attributes Related to Attributes Related to ““CommonCommon”” OnesOnes
Common Data ModelCommon Data Model
Business Rules Business Rules (i.e., Constraints)(i.e., Constraints)
Data Mappings to Data Mappings to Common Model (above)Common Model (above)
(some needed pre-requisites)
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 817
21
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Source System Architecture Variant Where Source System A Is ESB-Enabled
Authoritative Sources
ES-S
AFEMS
Others
...
Framework & Enterprise Services Bus (EAI, Web Services, XML, etc.)
Framework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)
ES-S AFEMS etc.
Adapter MAdapter M Adapter AAdapter A Adapter etc.Adapter etc.
Change Change Identification Identification & Notification & Notification
for Mfor M
Change Change Identification Identification & Notification & Notification
for Afor A
Change Change Identification Identification & Notification & Notification
for etc.for etc.
Enterprise Enterprise Data Data
Validation Validation for Mfor M
Enterprise Enterprise Data Data
Validation Validation for Afor A
Enterprise Enterprise Data Data
Validation Validation for etc.for etc.
CrossCross--Reference Reference for Associationsfor Associations
Previous Snapshot for Previous Snapshot for NonNon--Intrusion SystemsIntrusion Systems
However, existing However, existing messages from source messages from source system A to ESB must be system A to ESB must be sufficient for enterprisesufficient for enterprise-- wide needs, both in its wide needs, both in its timing and in its elements.timing and in its elements.
If Already If Already PublishingPublishing
Then Then SubscribeSubscribe
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 818
22
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Not part of these discussions.Not part of these discussions.
Data Services
System A Database
Framework & Enterprise Services Bus (EAI, Web Services, XML, etc.)
Framework &
Enterprise Services Bus (EAI, Web Services, XML, etc.)
ES-S AFEMS etc.
Adapter MAdapter M Adapter AAdapter A Adapter etc.Adapter etc.
Change Change Identification Identification & Notification & Notification
for Mfor M
Change Change Identification Identification & Notification & Notification
for Afor A
Change Change Identification Identification & Notification & Notification
for etc.for etc.
Enterprise Enterprise Data Data
Validation Validation for Mfor M
Enterprise Enterprise Data Data
Validation Validation for Afor A
Enterprise Enterprise Data Data
Validation Validation for etc.for etc.
CrossCross--Reference Reference for Associationsfor Associations
Previous Snapshot for Previous Snapshot for NonNon--Intrusion SystemsIntrusion Systems
However, existing However, existing messages from messages from source system A source system A to ESB must be to ESB must be sufficient for sufficient for enterpriseenterprise--wide wide needs, both in its needs, both in its timing and in its timing and in its elements.elements.
Change Change Identification Identification & Notification & Notification
for Afor A
These are These are either/or, either/or, not both.not both.
Enterprise Data Warehouse
BI Tools
Metadata
AF GLSC
Exchange
ToolsDiscovery
GLSC Data Translator
(Up to JS and Up from Legacy)
GLSC Data Loading
Viewers (e.g., FSC)
System A Database
Enterprise Data Warehouse
AF GLSCAF GLSC Archive
Authoritative Sources
ES-S
AFEMS
Others
...
If Already If Already PublishingPublishing
Or Or SubscribeSubscribe
Either Either ““TrapTrap””
Changes Changes or or
SubscribeSubscribe
Source System Architecture Variant where Source System A Is AFKS-Enabled
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 819
•About 200 Sites•Distributed Data•Stand Alone Operations
System of Record for Equipment Management
System of Record for Inventory Financial Data
Vehicle OperationsManagement Data
AFEMS
OLVIMS SBSS
Air Force Equipment Management SystemRejected
TransactionsRequire HumanIntervention
Standard Base Supply System On-Line Vehicle Information
Management System
Rejected Transactions
Require HumanIntervention
Summary Financial Reporting
Tx
Types of Financial Interest and
Equipment Management Interest
$$$Systems
Rejected TransactionsRequire HumanIntervention
Tx Types o
f Equipment
Management Interest
Tx Types of Financial Interest
and Equipment M
anagement Interest
Vehicle Records
Vehicle Records
Vehicle Records
1
2
3
Rejected tx
have limited notification processes and must each be cleared by workers
1
2
3
Core vehicle data copieswith additional domain data
Tx
are edited for syntax by SBSS
Interfaces with limited automation; mostly abandoned in favor of dual data entry
Tx
are edited for syntax and semantics by AFEMS
Some tx
are simply forwarded after syntax check (no other processing done by SBSS)
AS-IS Data Sharing Example from Vehicle Domain
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 820
24
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Vehicle Life Cycle
Need: One PickupGO
Allowed?
Authorized?
Priority Buy List$ APPROVED $
Acquire
Wait for
Turn - In
Receive
DRMO
Change Highlights Generating Parties Interested Parties Participating Systems
Wait for (SPR) Acquisition Vehicle Management, Equipment Management, Inventory Management
OLVIMS, AFEMS, SBSS
Receive (REC) Vehicle Management
Vehicle Management, Equipment Management, Inventory Management
OLVIMS, AFEMS, SBSS
Operate (mission capable)
Vehicle Operations Vehicle Management, Equipment Management
OLVIMS, AFEMS
Maintain (non-mission capable)
Vehicle Maintenance
Vehicle Management, Equipment Management
OLVIMS, AFEMS
Maintain (awaiting parts) Vehicle Maintenance
Vehicle Management OLVIMS
Turn-In (salvage) Vehicle Management
Vehicle Management, Equipment Management
OLVIMS, AFEMS, SBSS
• Mission Capable• Mission in Progress• Needs Preventive Maintenance• Work Order in Progress
Operate and Maintain
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 821
25
The MIT Information Quality Industry Symposium, 2007
© 2007 The MITRE Corporation. All rights reserved
Characteristics of The Situations
Constraint Mgr.
System A System B
Object X piece 1
Object X piece 2
Associative Cross-
Reference
System C
Object X piece 3
••
Object creation is System A responsibilityObject creation is System A responsibility••
CrossCross--Reference containsReference contains−−
Object Identifiers Object Identifiers −−
Data common to the systemsData common to the systems••
Systems canSystems can−−
Check object validity (B & C)Check object validity (B & C)−−
Eliminate conflicting dataEliminate conflicting data−−
Eliminate redundant data entryEliminate redundant data entry
Constraint Mgr.
System A System B
Object X Object Y
Associative Cross-
Reference
System C
Object Z
••
Multiple object creation responsibilityMultiple object creation responsibility••
CrossCross--Reference containsReference contains−−
Object identifiers (with association rules) Object identifiers (with association rules) −−
Data common to the systemsData common to the systems••
Systems canSystems can−−
Check object association validityCheck object association validity−−
Eliminate crossEliminate cross--system data conflicts system data conflicts −−
Reduce redundant data entryReduce redundant data entry
Attribute Synchronization Across an Object
Associations Among Objects
Account Information
Branch Information
Customer Information
Acquire Data Finance Data Inventory Data
Proceedings of the MIT 2007 Information Quality Industry Symposium
PG 822