Oracle Data Integration Management (OEMM) Overview

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Data Integration Enterprise Metadata Management (OEMM) Overview Dr. Holger Dresing Oracle Data Integration Solutions (DIS) EMEA

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Agenda

4

Datenintegration und Data Governance

OEMM Grundlagen

Schnittstellen/Connectoren

Demo/Powerpoint Screenshots

Q&A

1

2

3

4

5


Comprehensive Data Integration & Governance Capabilities

5

Real-Time Data Movement – Low impact capture, stage in

Hadoop – Continuous data availability

Data Transformation – Bulk data movement – Pushdown data processing

Data Federation – Virtualized Data Services

Data Quality & Verification – Fix quality at the source – Verify data consistency

Metadata Management – Lineage and Impact Analysis – Business Glossary Semantics

Data Governance Foundation

Oracle Data Integrator (Transformation)

Enterprise Data Quality (Profile, Cleanse, Match and De-duplicate)

Fast Load

Oracle GoldenGate (Movement)

Enterprise Metadata Management & Business Glossary (Business Glossary, Data Lineage, Impact Analysis and Data Provenance)

Data Service Integrator (Federation)

GoldenGate Veridata (Online Data Verification)

ELT Processing on Hadoop or

SQL

Continuous Availability

Big Data Preparation Service


Data Governance Foundation

Differentiated Technical Approach

Oracle OpenWorld 2014 6

Dynamic Data Movement – Real-time CDC is by default, not ETL – Least invasive on sources – Proven best performance – Integrated Oracle capture/apply

No ETL Engines – Take the processing to the data;

don’t move the data to the process – Leverage your data engines for the

workloads (Hadoop or SQL)

Most Heterogeneous – Leverage open source Hadoop, not

proprietary distributions – Hadoop is the Hub, not ETL tools – Open metadata standards

Oracle Data Integrator * (Transformation)

Enterprise Data Quality (Profile, Cleanse, Match and De-duplicate)

Fast Load

Oracle GoldenGate (Movement)

Enterprise Metadata Management & Business Glossary (Business Glossary, Data Lineage, Impact Analysis and Data Provenance)

Data Service Integrator (Federation)

GoldenGate Veridata (Online Data Verification)

ELT Processing on Hadoop or SQL

Continuous Availability

Big Data Preparation Service


Oracle Enterprise Metadata Management Trust your data – BCBS 239

Executives Dashboards / Reports

!=

!!! ???

•Dashboard does not match report? •Where did this data come from? •Can I trust my other dashboards and reports?


Oracle Enterprise Metadata Management Data Lineage Impact Analysis

DBA’s / ETL / BI Developers Data Mgmnt. / Development Env.

•What impact on does my column change have? •What impact does my ETL mapping have? •I want to create new assets from my metadata!!

?? !! ??


Value of Enterprise Metadata Management

Oracle Confidential, under Non-Disclosure 9

ETL

BI Dashboards

App

ETL

ETL

How was sales figure calculated?

What will happen if I change this

table?

What reports use the mainframe

data? Sys Admin

Executive

BI Developer

Where did this data

come from?

Application User

Which reports use this

customer data?

CDC

Data Reservoir

Data Steward

Can I trust the sources of this

customer data?

ETL

Developer

Solves a significant pain point for a wide variety of business consumers and technical staff

I want to design an experiment to measure the

success of a signup page. What data do I have?

Data Scientist

GG

http://hadoop.apache.org/


10

Vertical Lineage Links a business friendly set of terms to the

IT metadata and operational assets Capture Business Glossary, Taxonomy,

Ontology, Conceptual Models

Horizontal Column Level Links the data fields from Business Intelligence

Dashboards or Reports back to the Source Columns Schemas, BI View Layers, ETL Transformations,

Calculations, etc.

Ve

rtic

al L

ine

age

Horizontal Lineage

“NE_SALES”

“SALES”

“NAME” “ACCT_NAME”

“NORTH”

“AGG_TOTAL”

BI Fields to Source Columns

“FNAME|LNAME”

“Customer”

Biz Terms to IT

Two Crucial Styles of Metadata Management



Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 11

Introducing: Oracle Enterprise Metadata Management

1. Business Glossary – simple tools to catalog, link and collaborate on business terms

2. Metadata Management – horizontal and semantic data lineage for all big data sources

Business/Metadata Catalog Report to Source Lineage

Impact Analysis

Audit, Versioning & Diff Reports

Social/Collaboration Features

Annotations and Tagging

Automatic Stitching

Comprehensive Harvesting 3rd Party BI Metadata

3rd Party ETL Metadata

3rd Party DB Metadata

3rd Party Modeling Tools

Hadoop & NoSQL Metadata

Metadata Standards


Oracle Metadata Management Logical Architecture

14

Actionable Information

Data Lin

eage

Imp

act

An

alys

is

ETL Data Quality

Data Integration

Replication Messaging Data Federation

File Based Cloud

Data Sources

Big Data Relational Applications

Metadata Management

Business Glossary

Metadata Repository

Semantic Linking

Data Discovery Analytics




15


Data Lin

eage

Imp

act

An

alys

is

ETL Data Quality

Data Integration

Replication Messaging Data Federation

File Based Cloud

Data Sources


Business Glossary

Metadata Repository

Semantic Linking




16


Data Lin

eage

Imp

act

An

alys

is

ETL Data Quality

Data Integration

Replication Messaging

Data Federation

File Based Cloud

Data Sources


Business Glossary

Metadata Repository

Semantic Linking


Hadoop HBase Hadoop Hive/Flume HP Enscribe HP NonStop HP Neoview Hypersonic SQL IBM DB2 i Series IBM DB2 UDB IBM DB2 z Series IBM Informix IBM Netezza JMS / MQ Microsoft Access Microsoft SQLServer MySQL Pivotal Greenplum PostgreSQL Salesforce.com SAP BW / BI SAP ERP / ECC SAS SQL/MP SQL/MX Sybase ASE Sybase IQ Teradata

Adaptive Altova Apache Hcatalog Apache Hive/HQL Borland CA ERwin Cloudera Impala COBOL Copybook DataStax Embarcadero EMC ProActivity GentleWare Google BigQuery Grandite Hadapt Hive Hortonworks Hive IBM Cognos IBM DB2 IBM DataStage IBM Discovery IBM Federation Server IBM Lotus Notes IBM Netezza IBM Rational Rose IBM Rational Architect Informatica Metadata Mgr. Informatica PowerCenter

CoSORT ISO SQL Standard (DDL) MapR Hadoop Hive MicroFocus Microsoft Access Microsoft Office Excel Microsoft Visio Microsoft SQL Server Microsoft SSIS Microsoft Visual Studio Microstrategy Magic Draw OMG CWM Standard OMG UML Standard Oracle BI Answers Oracle BI Enterprise Edition Oracle BI Server Oracle DAC Oracle Data Integrator Oracle Data Modeler Oracle Database Oracle Designer Oracle Hyperion Applications Oracle Hyperion Essbase Oracle Warehouse Builder Pivotal Greenplum PostgreSQL

QlikView SAP BO Crystal Reports SAP BO Designer SAP BO Desktop Intelligence SAP BO Repository SAP BO Data Integrator SAP BO Data Steward SAP Master Data Management SAP Sybase PowerDesigner SAP Sybase ASE Database SAS Data Integration Studio SAS BI Server SAS Information Map SAS Metadata Management SAS OLAP Server Select Sparx Architect Syncsort Tableau Talend Teradata Tigris Visible W3C DTD & XSD Schema

Operational Integration (Movement / Transformation) Metadata Harvesting (Glossary, Lineage & Impact Analysis) Oracle Database Oracle Exadata Oracle Big Data Appliance Oracle TimesTen Oracle OLAP Oracle Business Intelligence Oracle BI Applications Oracle E-Business Suite Oracle JD Edwards Enterprise One Oracle JD Edwards World Oracle Fusion Applications Oracle Governance Risk and Compliance Oracle Fusion AIA Oracle Retail Applications Oracle Agile BI / DW Oracle Agile PLM for Process Oracle iFlex FlexCUBE Oracle iFlex Mantas Oracle Hyperion Applications Oracle PeopleSoft Oracle Siebel CRM / OnDemand Oracle Communications Oracle WebLogic Server Oracle Coherence Data Grid Oracle SOA Suite Oracle Enterprise Service Bus

+ open APIs and standards based meta-model

Comprehensive, Open & Heterogeneous Data Integration


Exploring Metadata

Text Search and Metadata Browser,

Model/Diagram Visualizers, Birds-eye View, Metadata Reporting

Data Flow and Lineage Analyzer

Sarbanes-Oxley Act (SOX) Compliance

Versioning and Model Compare

URI Based Call back from BI reports or Business applications

Stewardship Collaboration

Feedback Comments and Review Board

Metadata Tagging with Labels

Multimedia attachments (documents, videos, presentation, code)

URL link references for e-mail, blogging, social networking

Detailed Feature List Lifecycle Change Management

Metadata Harvesting from any DM, DW, ETL/DI, BI Tooling and Big Data/Hadoop

Enterprise Architecture Management (Metadata Configuration Management)

Impact of Change

Data Standardization & Data Governance

Supports Variety of Metadata Standards

Annotations and Tagging

Business Glossary with Semantic Lineage

Oracle Integrated

Oracle Database

Oracle Data Integrator, Oracle GoldenGate

Oracle Business Intelligence Enterprise Edition

Oracle Business Intelligence Applications (ODI and older Informatica based OBIA)


Metadata Harvesting from all Popular Platforms


Data Governance Lifecycle Tools

20

Operational Data Flows Business Sources

Quality KPIs Case Management

Governance Cockpit for Data Stewards & Stakeholders

Exception Review

Metadata Management

Business Glossary

Design Time


Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 21

Oracle Metadata Management Physical Architecture

Semantic Mapping

Enterprise Conceptual Model Metadata Repository

Repository Manager

Data Stewardship Governance

Metadata Browser Search

Lineage Impact Analyzer

Any Web Browser

Data Source

ETL

BI - Reporting

Business Glossary

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 23

DEMO


Friendly Views for Business & IT User Collaboration

Oracle Big Data Governance 24

• Havesting Metadaten • Oracle Datenbank • Oracle Data Integrator • Informatica Powercenter • Oracle Business Intelligence • Excel-Datei

• Data Lineage • Impact Analysis • Manual Data Mapping • Business Glossar • Metadata Explorer


Friendly Views for Business & IT User Collaboration

Oracle Big Data Governance 25

Comprehensive Data Lineage

Business Friendly Navigation

Business & IT Collaboration

Easy to Use, Search Driven


End-to-End Data Flow Architecture Views

See how systems are connected

while visualizing

annotations and system boundaries



Keyword Based Search on all Metadata

Full index text search

provides simple

navigational access to very

large metadata models



Graphical Browser for Data Model Diagrams

Birds-eye-view metadata

viewer enables zoom-in / zoom-out

around complex

metadata models



Versioning with Model Comparisons

Easily group and compare data models from similar databases to

determine crucial

differences



Social Collaboration for Stewardship Teams

Team-based approach is designed in

from the start



Find Data Provenance from the Report Layers

Visualize column and calculation

level dependencies

across systems, databases and

reporting views



Business Glossary is Built In (not a separate add-on)

Going beyond database

dictionaries, link business terminology

across metadata

models and systems



Perform Semantic Analysis

For a Business Term, find all the

physical implementations where the term is

used



Business Glossary workflow

Workflow for the entire

lifecycle from drafting to the final approval

of business terms



Connect with Oracle Data Integration

@OracleDI

Blogs.oracle.com/DataIntegration/

Oracle Data Integration

Oracle Data Integration


Questions & Answers

Oracle Data Integration Management (OEMM) Overview

Documents

Transcript of Oracle Data Integration Management (OEMM) Overview