Partner Webcast – Oracle Data Integration for Big Data
-
Upload
oracleimc-innovation-modernization-center -
Category
Technology
-
view
936 -
download
0
Transcript of Partner Webcast – Oracle Data Integration for Big Data
![Page 1: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/1.jpg)
![Page 2: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/2.jpg)
Stay Connected
BLOGS.ORACLE.COM/IMC
TWITTER.COM/ORACLEIMC
YOUTUBE.COM/ORACLEIMCTEAM
FACEBOOK.COM/ORACLEIMC
![Page 3: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/3.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integration For Big Data
Milomir Vojvodic Director Business Development Oracle Europe, Middle East and Africa central Data Integration team October 15th 2015.
Presented by
![Page 4: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/4.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Appliance
Oracle Exadata
Acquire Organize Analyze & Visualize Stream
Oracle Exalytics
Load from big data processing into your data warehouse for further analysis Access your customer information while you process through your big data in order to look for patterns
Big Data Value Is From Correlation
![Page 5: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/5.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 6: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/6.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Staging
6
Sqoop
HDFS
Hive
Flume
Cap
ture
Trai
l
Ro
ute
De
live
r
Pu
mp
Transformation
Data Streaming Kafka (MPP Pub/Sub)
Storm and Trident
Spark Streaming
HBase
Discovery Sandbox/s
R Oracle GoldenGate
Oracle Data Integrator
Oracle Data Governance
Oracle Data Preparation
Model First Analytics
• Reporting-oriented • Often enterprise wide
in scope, cross LoB • “you know the
questions to ask”
Reports & Dashboards
Data First Analytics
• Data Exploration • Highly visual and/or
interactive • “you don’t know the
questions to ask”
Discovery
• Telematics • Industry Services • Internet of Things • Sentiment
Data Services
Oracle Big Data Integration and Governance Ecosystem
![Page 7: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/7.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. | Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
BIG DATA MOVEMENT &
TRANSFORMATION
BIG DATA GOVERNANCE
BIG DATA PREPARATION
REAL-TIME BIG DATA
ORACLE BIG DATA
INTEGRATION
Load Data into Hadoop in Real-Time
Move and Transform Data in Bulk
Manage & Monitor Metadata and Data Quality
Reduce Time Spent on Data Preparation
Oracle Big Data Integration
![Page 8: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/8.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Target DB
OGG
Source DB
First OGG Differentiator Accessing directly transaction
logs
Second OGG Differentiator Moving only committed transactions
Why Is OGG Different?
![Page 9: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/9.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
TIME REQUIRED FOR THE END OF DAY
PROCEDURE
Hours
NO OF CPUs REQUIRED FOR SAME
PERFORMANCE*
No Of Required CPUs
ESTIMATED COSTS FOR SERVER AND
LICENSE**
Estimated Cost of Purchase in USD
0
20
40
60
80
100
120
140
Year1 Year2 Year3 Year4 Year5
Currently during the End Of Day utilizes the Server CPU by 40-50% and the IO by 90%. Probably the IO is the bottleneck.
0
20
40
60
80
100
120
Year1 Year2 Year3 Year4 Year5
Disaster Recovery Test and Development
Primary Site
$-
$,50
$1,0
$1,50
$2,0
$2,50
Millio
ns
Oracle License Costs
Hardware Costs
Daily load time can reach 5 days with the current HW
OR
Alternative To Batch Window
First OGG Differentiator Accessing directly transaction logs
After OGG Before OGG
Reduce source system overhead (and costs for stronger HW) by
70%
![Page 10: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/10.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
OR
Second OGG Differentiator Moving only committed
transactions
Alternative To Storage Replica
Begin, TX 1
Insert, TX 1
Begin, TX 2
Update, TX 1
Insert, TX 2
Commit, TX 2
Begin, TX 3
Insert, TX 3
Begin, TX 4
Commit, TX 3
Delete, TX 4
Begin, TX 2
Insert, TX 2
Commit, TX 2
Begin, TX 3
Insert, TX 3
Commit, TX 3
Begin, TX 2
Insert, TX 2
Commit, TX 2
Capture Checkpoint
Pump Checkpoint
Delivery Checkpoint
After OGG Before OGG
Reduce costs and efforts of data loss by 70%
![Page 11: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/11.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Cap
ture
Trai
l
Ro
ute
Del
iver
Pu
mp
New DB/ HW/OS/APP
Zero Downtime Upgrades & Data Migration
Fully Active Distributed DB
High Availability & Disaster Recovery
Application Offloading
Query & Report Offloading
Big Data, DW & Marts
Real-time BI, Hadoop Data Staging, Data Ingestion
Event Driven Architecture, SOA/JMS, Coherence
Message Bus & Data Grid
Data Synchronization Across the Enterprise
Global Data Centers
Real-time Analytics & Massive Parallelization
Data Streaming
GoldenGate
Real-time Data Delivery
11
Oracle GoldenGate For Big Data
![Page 12: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/12.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• Accessing Directly Transaction Log
• Delivery Of Committed Transactions
• Stability Of Replica Line
Target DB Source DB
What Is So Special About OGG for Big Data?
![Page 13: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/13.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
1. Parameter setting only
2. No need to learn current and emerging big data technologies
13
OGG for Big Data OGGAA for JMS and Flat File
Flat File Java JMS
Hive HDFS HBase Flume
OGG for Big Data – Additional Value
![Page 14: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/14.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• Name: Oracle GoldenGate for Big Data
• Separate Installer, available on eDelivery
• Includes base Java Adapter functionality, but no File adapter
14
Oracle
GoldenGate
Capture Database Transactions and Deliver to Big Data in Real-Time
JMS
Capture Trail Pump Route Deliver
HDFS (Files)
HBase (NoSQL)
Hive(SQL)
Flume (Streaming)
OGG for Big Data – Additional Value
![Page 15: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/15.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. | 15 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
BIG DATA MOVEMENT &
TRANSFORMATION
BIG DATA GOVERNANCE
BIG DATA PREPARATION
REAL-TIME BIG DATA
ORACLE BIG DATA
INTEGRATION
Load Data into Hadoop in Real-Time
Move and Transform Data in Bulk
Manage & Monitor Metadata and Data Quality
Reduce Time Spent on Data Preparation
Oracle Big Data Integration
![Page 16: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/16.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
OLTP & ODS Systems Data
Warehouse, Data Mart
Oracle PeopleSoft, Siebel, SAP
Custom Apps
Files Excel XML
Enterprise Performance
Custom Reporting Packaged Applications
Business Intelligence
Analytics
Data Federation
Data Warehousing
Custom
Data Marts Data Access
Data Silos
SQL Java
Batch Scripts
Data Hubs
Data Migration
Data Replication
OLAP
Replacing Manual Coding
![Page 17: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/17.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
OLTP & ODS Systems Data
Warehouse, Data Mart
Oracle PeopleSoft, Siebel, SAP
Custom Apps
Files Excel XML
Enterprise Performance
Custom Reporting Packaged Applications
Business Intelligence
Analytics
OLAP
Oracle Data Integrator
Replacing Manual Coding
After ODI Before ODI
Reduce data transformation maintenance costs by 80% (hard to change, every script contains
special rules, code stored in many machines)
![Page 18: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/18.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Journalize Read from CDC
Source
Load From Sources to
Staging
Check Constraints before
Load
Integrate Transform and
Move to Targets
Service Expose Data and Transformation
Services
Reverse Engineer Metadata
Reverse
Journalize
Load
Check
Integrate Services
CDC
Sources
Staging Tables
Error Tables
Target Tables
WS
WS
WS
SAP/R3
Siebel
Log Miner
DB2 Journals
SQL Server Triggers
Oracle DBLink
DB2 Exp/Imp
JMS Queues
Check MS Excel
Check Sybase
Oracle SQL*Loader
TPump/ Multiload
Type II SCD
Oracle Merge
Siebel EIM Schema
Oracle Web Services
DB2 Web Services
Sample out-of-the-box Knowledge Modules
Benefits
ODI Knowledge Modules
ODI Declarative Design
ODI Declarative Design
Define How : Built - in Templates
Define What You Want
Automatically Generate Dataflow
1 1 2 2
Define How : Built - in Templates
Define What You Want
Automatically Generate Dataflow
1 1 2 2
Define How : Built - in Templates
Define What You Want
Automatically Generate Dataflow
1 1 2 2
Define What You Want
Automatically Generate Dataflow
1 1 2 2 1 1 2 2
ODI E-LT
Staging Server
Data Warehouse
Second ODI Differentiator ODI Declarative Design and
ODI Knowledge Modules for reusing already written
down level SQL code
First ODI Differentiator Transformations using
the power of the Target Database – no
staging server
Why Is ODI Different?
After ODI Before ODI
Reduce ETL development
costs by 30% (no prebuilt code, need
to learn various languages, need to write and tune SQL)
After ODI Before ODI
Decrease the cost o of ETL HW by 100%
![Page 19: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/19.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Business rules implemented in SQL
Source (MySQL)
ORDERS
LINES
CORRECTIONS
File
Target (Oracle)
SALES
ERRORS
Join
ORDERS.ORDER_ID =
LINES.ORDER_ID
…
Mapping
SALES =
SUM(LINES.AMOUNT) +
CORRECTION.VALUE
• SALES_REP =
ORDERS.SALES_REP_I
D
Constraints
ID is flagged not null
in the model. Unique
index UK_ID is declared
on the SALES table.
Filter
ORDERS.STATUS=CLOSED
…
Implementing The Rules
![Page 20: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/20.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Target (Oracle)
SALES
ERRORS
Transform and
integrate
TEMP_
SALES
Check constraints/
Isolate errors
Source (MySQL)
ORDERS
LINES
CORRECTIONS
File
TEMP_1
Extract/Join/
Transform
TEMP_2
Extract/Transform
Join/Transform
1
2
3
4
5
Process Implementation Without ODI
![Page 21: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/21.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
1
2
3
4 5
![Page 22: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/22.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Target (Oracle)
SALES
ERRORS
Source (MySQL)
ORDERS
LINES
CORRECTIONS
File
TEMP_1
Extract/Join/
Transform
TEMP_2
Extract/Transform
Join/Transform
Transform and
integrate
TEMP_
SALES
Check constraints/
Isolate errors
LKM
LKM
LKM
CKM
IKM
Proprietary Engine
- Specific Language
Process Implementation With ODI
![Page 23: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/23.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Target (Oracle)
SALES
ERRORS
Transform and
integrate
TEMP_
SALES
Check constraints/
Isolate errors
Source (MySQL)
ORDERS
LINES
CORRECTIONS
File
TEMP_1
Extract/Join/
Transform
TEMP_2
Extract/Transform
Join/Transform
1
2
3
4
5
Process Implementation With ODI
![Page 24: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/24.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Logical Design
Physical Design
Oracle
MySQL
Hive
Sqoop
Sqoop
IKM
LKM
LKM
Oracle
Hive
MySQL
Hive
Logical and Physical Design with ODI
![Page 25: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/25.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. | 25
Flume/Kafka Hive on MR, Tez, Spark
Logs
OLTP DB
SQOOP
OGG
Pig on MR, Tez, Spark
ODI
SQOOP Any DW
OGG
Spark
Oozie
OEDQ OEMM
Data Validation & Cleansing
Metadata Mgmt & Lineage
API/File/HDFS
Hive/HCat, HDFS,HBase
Hive/HCat, HDFS,HBase
NoSQL
Flume/Kafka
Load to Oracle
OLH/OSCH
Oracle DB Big Data SQL
Files
HDFS/File KM
Oracle Data Integrator (ODI) Capabilities for Big Data
![Page 26: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/26.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• ODI Is Writing Code
• SW Project Lifecycle + Data Semantic Related Functionalities
• Reusability Principle
• ODI Is Easy To Learn
What Is So Special About ODI for Big Data?
![Page 27: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/27.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
1. Code Generation for Spark
2. Code Generation for Pig
3. Execution using Oozie
4. No Installation on Hadoop
27
ODI Advanced BD Option – Additional Value
![Page 28: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/28.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Hadoop Cluster
Spark
Sqoop Hive
Pig
ODI
Oozie
Sqoop
28
Hadoop Cluster
Spark Sqoop Sqoop
Hive
Pig
Manual Code Hadoop Cluster
ETL ETL HDFS
Hadoop Cluster
ETL ETL ETL
HDFS
1. Traditional ETL Tools (execute entirely outside of Hadoop)
2. ETL Tools with Native “on” Hadoop (require proprietary code on Data Nodes)
3. Manual Coding (ultimate flexibility, but very high cost)
4. ODI Native in Hadoop (no Engine & no Data Node footprint)
ETL
GG
generate manage
BEST
ODI Advanced BD Option – Additional Value
![Page 29: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/29.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
29
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
BIG DATA MOVEMENT &
TRANSFORMATION
BIG DATA GOVERNANCE
BIG DATA PREPARATION
REAL-TIME BIG DATA
ORACLE BIG DATA
INTEGRATION
Load Data into Hadoop in Real-Time
Move and Transform Data in Bulk
Manage & Monitor Metadata and Data Quality
Reduce Time Spent on Data Preparation
Oracle Big Data Integration
![Page 30: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/30.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Information
Security
Master Data
Management
Policy Management
Enterprise Metadata
Reporting
Data
Integration
Data Quality
DATA GOVERNANCE
Data Governance Consolidates Strategies
![Page 31: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/31.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Technology
Process
People
People • Governance Committee • Data Stewards
Process • Defined rules
& practices Technology
• Implements process and supports people
• Makes operation more consistent & cost-effective
• Basis for continuous process improvement
Data Governance – more than just technology
![Page 32: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/32.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Data Originator
Data Originator
Data Originator
Data Originator
Data Originator
Data Consumer Data Consumer Data Consumer Data Consumer
Data Governance, Integration & Management
Transformation
Cleansing
Replication
Matching Merging
Aggregation Denormalisation
Separation Between Originators and Consumers
![Page 33: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/33.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Customer ID Customer Name Address 1 Address 2 City State Zip Country Birth Date Gender
AD23298 Mr Peter Mayhew 9407 Main St Fairfax VA 22031-4001 USA 02/23/61 M
VS38611 Dr Ellen Van Der Heijde 144 E Grove St Kingston PA 18704 US 07/12/57
DC18223 Jalila Abdul-Alim (Do Not Call) 4548 Pennsylvania Ave Apt 205 Kansas City MO 64111-3349 USA 02/23/63 F
CO9387A Tayside Computers Inc. 4912 E 41st N Idaho Falls ID 83401 USA 31/03/2007 N/A
TZ35019 Mr Zachary P Jahn 98-1731 Ipuala Loop Aiea Hawaii 96701 1710 United States 06/12/86 Male
CB27843 Mrs Edith Y Baba Junior Baba Real Est. Corp. 209 Stony Point Trl Webster NY USA 11/17/1971 M
OX80306 Andrew & Mary Baxter 14 Oxbridge Way Milfrod NH 03055-4614 US 05/28/67 F
JP70210 Mr RJ & Mrs FB MacDonald 57 Hadleigh Close Westlea Swindon SN5 9BZ MA - USA - Y
RD48107 Mr Andy Baxter 14 Oxbridge Wy Milford NH 3056 USA 01/01/01 M
Inconsistent formats Abbreviations
(often ambiguous)
Attributes non-standard, missing
or invalid
Widespread
duplication
(often hard
to spot)
Compound Names
Embedded Additional Information
Mixed Business & Personal Names
Multiple Names
Mis-Fielded Data
Erroneous Data
International Date Formats
Default or Dummy Data
Why EDQ?
After EDQ Before EDQ
Avoid error costs (incorrect orders, inventory etc.) by 20%
![Page 34: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/34.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 35: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/35.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 36: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/36.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 37: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/37.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 38: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/38.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 39: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/39.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 40: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/40.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 41: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/41.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 42: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/42.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 43: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/43.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 44: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/44.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 45: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/45.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 46: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/46.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• A Match Rule is simply the combination of comparison results
• Rules are evaluated in order and if one hits, we stop
• Rules can be ‘negative’ to eliminate pairs that are too different with a ‘No Match’ rule
• Rules can easily be turned on & off during the tuning process
![Page 47: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/47.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 48: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/48.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 49: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/49.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
ETL
BI Dashboards
App
ETL
ETL
How was sales figure calculated?
What will happen if I change this table?
What reports use the mainframe
data?
Sys Admin
Executive
BI Developer
Where did this data come
from?
Application User
Which reports use this customer
data?
CDC
Hadoop Data Lake
Data Steward
Can I trust the sources of this
customer data?
ETL
Developer
I want to design an experiment to measure the success of a
signup page. What data do I have?
Data Scientist
GG
Data Warehouse
OGG BI&DW Synchronization and Loading
ODI OEDQ
OEMM
After OEMM
Before OEMM
OPEX - Reduce data maintenance costs by 80% (not anymore.. hard to
change, long to find)
Value Of Metadata Management
After OEMM
Before OEMM
CAPEX - Reduce analytical project costs by 30% (not anymore ..post
remediation costs, unnecessary mistakes )
![Page 50: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/50.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Metadata Repository
Any Web Browser
OEMM Architecture
![Page 51: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/51.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Metadata Requirements Stack
![Page 52: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/52.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
1. Title - the title of the data object
2. Creator - the person or entity responsible for creating the data object
3. Subject - subject terms or keywords that describe the data object
4. Description - a brief description, or abstract, of the data object
5. Publisher - the entity responsible for making the data object available
6. Contributor - a person or entity who contributed to the creation of the data object
7. Date - data of creation, publication, or revision of the data object
8. Type - the type of object. For data this would typically be "dataset"
9. Format - a description of the format or file type(s) of the data object
10. Identifier - a permanent identifier used to locate and identify the data object
11. Language - the language(s) used within the data object (if applicable)
12. Source - a relational element describing the lineage of the data object
13. Relation - a relational element describing the relationship of this data object to other objects, collections, or entities
14. Coverage - describes the spatial and temporal context of the data object
15. Rights - describes any rights, restrictions, or terms of use
Metadata
![Page 53: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/53.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 54: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/54.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 55: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/55.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 56: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/56.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 57: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/57.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 58: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/58.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 59: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/59.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 60: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/60.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 61: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/61.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 62: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/62.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 63: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/63.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 64: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/64.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 65: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/65.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 66: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/66.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Business Glossary Stewardship Workflow
![Page 67: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/67.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Business Glossary Manages Definitions
![Page 68: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/68.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Mapping of Business Term to Rules
![Page 69: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/69.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Capture Business Rule Definitions
![Page 70: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/70.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Semantic Linking of Business Rules
![Page 71: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/71.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Allowable Values of Business Terms
![Page 72: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/72.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Data Flow Architecture Views: End-to-End / Top-to-Bottom
![Page 73: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/73.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Graphical Browser for Data Model Diagrams
![Page 74: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/74.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. | Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
BIG DATA MOVEMENT &
TRANSFORMATION
BIG DATA GOVERNANCE
BIG DATA PREPARATION
REAL-TIME BIG DATA
ORACLE BIG DATA
INTEGRATION
Load Data into Hadoop in Real-Time
Move and Transform Data in Bulk
Manage & Monitor Metadata and Data Quality
Reduce Time Spent on Data Preparation
Oracle Big Data Integration
![Page 75: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/75.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Parse Click Stream Logs
Repair App Data
Classify Social Data
Structured
Unreliable
Unstructured High Velocity
Unstructured High Volume
SSN Credit Card Info
Supported Formats
Oracle Big Data Preparation Cloud Service
![Page 76: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/76.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 77: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/77.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 78: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/78.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 79: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/79.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 80: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/80.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 81: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/81.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 82: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/82.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 83: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/83.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 84: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/84.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 85: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/85.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 86: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/86.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 87: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/87.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 88: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/88.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 89: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/89.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 90: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/90.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 91: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/91.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 92: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/92.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
h
![Page 93: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/93.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 94: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/94.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 95: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/95.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 96: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/96.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 97: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/97.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 98: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/98.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 99: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/99.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 100: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/100.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 101: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/101.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 102: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/102.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
![Page 103: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/103.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Q&A
10
ISV Migration Center blog: http://blogs.oracle.com/imc ISV Migration Center email: [email protected]
![Page 104: Partner Webcast – Oracle Data Integration for Big Data](https://reader031.fdocuments.net/reader031/viewer/2022030309/58f2d6ea1a28abd4058b458f/html5/thumbnails/104.jpg)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• Oracle.com Partner Hub oracle.com/partners/goto/hub-ecemea
• Migration Center Team Blog blogs.oracle.com/imc
feeds.feedburner.com/oracleIMC
• Partner Webcast Recordings youtube.com/OracleIMCteam
• Partner Webcast Presentations slideshare.net/Oracle_IMC_team
Oracle Partner Hub ISV Migration Center • twitter.com/OracleIMC
• plus.google.com/+OracleIMC
• facebook.com/OracleIMC
• linkedin.com/groups/Oracle-Partner-Hub-Migration-Center-4535240