Limited Distribution Release Open Information Interoperability Tool Suite Dr. Len Seligman, Dr. Ken...

Limited Distribution Release

Open Information Interoperability Tool SuiteDr. Len Seligman, Dr. Ken Smith,

Catherine Macheret, Chris [email protected],

[email protected], [email protected],

[email protected]

mailto:[email protected]




© 2009 The MITRE Corporation. All rights reserved

2


Outline

Background: Interoperability Challenges

OpenII overview

Demo

Status and Next Steps


3


Challenges and Opportunities

Resource-Intensive Interoperability Tasks:

– Enterprise challenge: Developing, maintaining, discovering and harmonizing data standards

– Challenge for programs: Adopting, translating and migrating standards to implement data exchanges

Interoperability tools reduce effort

– Improving stewardship of standards

– Increasing adoption of standards

– Easing data translation among standards

– Increasing number of data exchanges


4


Mapping Needs for Standards Based Exchanges

TargetDatabase

SourceDatabase

<xsd:schema xmlns:iepd-exchange = "urn:dhs.cbp.sbinet.p25_agent_location.exchange“ … </xsd:schema>

Exchange Schema

Map

Map

NIEM

Other Standards:Geospatial, Health, …

Map

Map


5


Outline


OpenII overview

Demo



6


Problem

It takes too long to productively share data today

Many available tools are vendor-specific and costly

– Interoperability tools don’t interoperate!

More functionality needed

– e.g., Find my sharing opportunities?IB

M

Ora

cle

Mic

roso

ft


7


OpenII

Collaboration with industry and academic leaders to build an open source information interoperability toolkit

Extensible, plug-in architecture

Rich & growing tool suite

Open source advantages:

– Free!

– Anyone can customize it, incorporate it into contractor tools

OpenIntegration.org


8


Demo


9


Step 1: Manage Schemas

Step 3: Generate Mapping Between Schemas

Step 2: Identify Associations Among Schemas

The Schema Integration Process

Step 4: Use Results of Schema Mappings

Schemas

DDL

XMLOWL


10


Importing Schemas


11


Searching for Schemas


12


Finding Schema Associations


13


Analyzing Schema Relatedness


14


Matching Schemas


15


Use Results of Schema Mappings

CREATE TABLE "SK260" ( "skid" int GENERATED ALWAYS AS IDENTITY ,"V0" String, "V1" String, "V2" int, "V3" int, "V4" String, "V5" String );INSERT INTO "SK260" ( "V0", "V1", "V2", "V3", "V4", "V5" ) SELECT DISTINCT "T0"."title","T0"."authorFName","T0"."ISBN","T0"."productID","T0"."authorLName","T0"."publisher" FROM "Books" AS "T0", "Data" AS "T1" WHERE "T0"."" = "T1"."ID";CREATE TABLE "SK243" ( "skid" int GENERATED ALWAYS AS IDENTITY ,"V0" String, "V1" String, "V2" int, "V3" int, "V4" String, "V5" String );INSERT INTO "SK243" ( "V0", "V1", "V2", "V3", "V4", "V5" ) SELECT DISTINCT "T0"."title","T0"."authorFName","T0"."ISBN","T0"."productID","T0"."authorLName","T0"."publisher" FROM "Books" AS "T0", "Data" AS "T1" WHERE "T0"."" = "T1"."ID";INSERT INTO "Data" ("ID","") SELECT DISTINCT ( SELECT "skid" FROM "SK260" WHERE "V0" = "T0"."title" AND "V1" = "T0"."authorFName" AND "V2" = "T0"."ISBN" AND "V3" = "T0"."productID" AND "V4" = "T0"."authorLName" AND "V5" = "T0"."publisher"), ( SELECT "skid" FROM "SK260" WHERE "V0" = "T0"."title" AND "V1" = "T0"."authorFName" AND "V2" = "T0"."ISBN" AND "V3" = "T0"."productID" AND "V4" = "T0"."authorLName" AND "V5" = "T0"."publisher") FROM "Books" AS "T0", "Data" AS "T1" WHERE "T0"."" = "T1"."ID";INSERT INTO "Books" ("author","description","format","ID","ISBN","length","productId","publisher","supplierID","title") SELECT DISTINCT "T0"."authorLName",1,1, ( SELECT "skid" FROM "SK243" WHERE "V0" = "T0"."title" AND "V1" = "T0"."authorFName" AND "V2" = "T0"."ISBN" AND "V3" = "T0"."productID" AND "V4" = "T0"."authorLName" AND "V5" = "T0"."publisher"),"T0"."ISBN",1,"T0"."productID","T0"."publisher",1,"T0"."title" FROM "Books" AS "T0", "Data" AS "T1" WHERE "T0"."" = "T1"."ID";DROP TABLE "SK260";DROP TABLE "SK243";

SQL Data Transform


16


RMAP

OpenII Architecture & Toolkit

OpenII SchemaStore Metadata Repository (M3 Metamodel)OpenII SchemaStore Metadata Repository (M3 Metamodel)

Importers, Exporters, Schema Search, & Display, Admin functions Importers, Exporters, Schema Search, & Display, Admin functions

Eclipse F

rame

work

Eclipse F

rame

work

WS

DL

, Repository S

electionW

SD

L, R

epository Selection

Affinity Overlap Harmony UnityXMAP

Common Ground Workbench


17


Outline


OpenII overview

Demo



18


Selected Customer Applications

Army: Distributed Common Ground Station (DCGS-A)

National Geospatial-intelligence Agency (NGA) and DHS: aligning infrastructure protection models

Air Force: aligning multiple large code lists; also data models with architecture products

Army/G8/Force Development/Warfighter Analysis Office


19


Status

Most OpenII tools are mature prototypes

– Proven useful on real problems

– Innovative

– Not shrink-wrapped for NIEM applications, however

Proposed next step: Customize OpenII to better support NIEM workflows

– Tune matching algorithms for known NIEM schema patterns

– Create bridge to metadata repositories to maximize reuse

– Integrate with IEPD production tools

– Apply to DHS pilots


20


Proposed NIEM Use Cases

NIEM Domain and Core Vocabulary Stewardship

– Creating new core vocabularies

– Harmonizing newly introduced NIEM domains into existing set

– Mapping enterprise conceptual models to core vocabularies and data assets

Data Exchange Implementation

– Searching IEPD repositories for relevant IEP schemas to leverage

– Mapping business data model to NIEM models

– Mapping source and/or target schemas to NIEM exchange schema and auto-generating data translation code


21


Summary

Open architecture, open source

– Contributions from industry, academic leaders

– Free to incorporate into COTS or GOTS

– Customizable, extensible

Benefits

– Improved stewardship and harmonization of standards

– Easier adoption of standards

– Quicker standards-based interoperability

– [email protected], [email protected], [email protected], [email protected]

We are ready to begin pilots and to customize tools for DHS needs






22


Backup


23


Domain Overlaps in Maritime and International Trade


24


Developer View of Gaps

No easy way to browse across program and application schemas

No automatic way to find common data across the enterprise

Metadata harmonization process is manual and time consuming

No automatic way to find gaps between the enterprise logical model and the various enterprise data stores

Building Data Exchanges is Time Consuming

Programs need tools to ease adoption of data standards


25


Developer View of Gaps II

Progress: NIEM, Data Asset Catalogs and enterprise data models are positive steps

Continuing Needs:

– Visibility at data element level, to identify schema reuse and harmonization opportunities

– Data asset clustering, to speed definition of new NIEM domains and IEPDs

– Matching, to identify correspondences between IEPDs and existing systems

– Mapping and CodeGen, to speed generation of data exchange code


26


OpenII Framework and Tools

SchemaStore

Schemas ExtensionsMappings

Harmony

Identifyschema

correspondences

API

Transfer data via SQL

Generate new domain models

/ exchange schemas

Transfer data via XQuery

Cluster schemas around themes

Affinity RMap XMapUnity

XSD

OWL

DDL

Import / Export

Search

Examine degree of schema relatedness

Overlap

Common Ground Toolkit

Limited Distribution Release Open Information Interoperability Tool Suite Dr. Len Seligman, Dr. Ken...

Documents

Transcript of Limited Distribution Release Open Information Interoperability Tool Suite Dr. Len Seligman, Dr. Ken...