Oracle Data Integration€¦ · •Inbound customer flows were creating duplicate records...
Transcript of Oracle Data Integration€¦ · •Inbound customer flows were creating duplicate records...
Oracle Data Integration
Cloud Data Quality: Lessons Learned from Oracle’s Journey to the Sales CON9595
Martin Boyd Senior Director Product Strategy, Oracle Ravi Mikkilineni CRM Data Management, Oracle
Oracle Confidential – Internal/Restricted/Highly Restricted Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Confidential – Internal/Restricted/Highly Restricted 2
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integration Core Business Use Cases
Oracle OpenWorld 2014 4
Analytics & Business Intelligence • Big Data Integration & Governance • Data Warehouse / Mart Integration
IT Data Integration & Governance • Enterprise Shared/Common Data Services • Move, Migrate, Transform, Profile, Cleanse,
Glossary and Metadata Management Business Continuity • Active-Active for Maximum Availability • Zero Downtime Migrations
24 x 7 x 365
Cloud
On Premise
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 5
NoETL Engine 100% Native Data
Transformation
Data Integrator
Big Data Preparation
GoldenGate
Data Quality
Data Service Integrator
Metadata Management
Non-invasive CDC, Realtime streaming
data delivery
Profile, Cleanse, Match, and
Remediate Data
Prepare, Secure, Enrich and Publish Unstructured Data
Catalog, Trace and View Models across
the Enterprise
Federate Data Across DBs, Services
and Applications
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 6
NoETL Engine 100% Native Data
Transformation
Data Integrator
Big Data Preparation
GoldenGate
Data Quality
Data Service Integrator
Metadata Management
Non-invasive CDC, Realtime streaming
data delivery
Profile, Cleanse, Match, and
Remediate Data
Prepare, Secure, Enrich and Publish Unstructured Data
Catalog, Trace and View Models across
the Enterprise
Federate Data Across DBs, Services
and Applications
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Data Quality Management is more critical than ever
Oracle Confidential – Internal/Restricted/Highly Restricted 7
Data as a Capital Asset
Data Stewards
Business Control
Chief Data Officer
DATA GOVERNANCE
Operational Data Stores
Link to Big Data
Link to Social Data
Cloud Apps
MASTER DATA MANAGEMENT
Anti Money-Laundering
REGULATORY COMPLIANCE
Self Service
Data Lakes Cloud BI
BUSINESS INTELLIGENCE Basel &
Solvency III
Know Your
Customer
ENTERPRISE DATA QUALITY MANAGEMENT
Case Management KPIs Configuration
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Enterprise Data Quality
Profile
Standardize
Match
Govern
Quickly understand data content
Drive conformance to standards
Identify & merge duplicates
Monitor effectiveness & resolve problems
Com
mon A
ccess/U
I
Enterprise DQ Platform
Market-leading usability for all types of data
Unparalleled time-to-value
High performance engine
Out-of-the-box global knowledge-base
Foundation for governance program
8 Oracle OpenWorld 2014
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Introduction
Challenges
Approach
Solution
Results
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 9
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 10
Oracle Sales Cloud
• 450k Opportunities/year
• 1.5 Million Leads/year (Marketing + Partner)
• 72k Territories
• Customer Data Management team: 40 FTE
Oracle on Oracle Sales Cloud
1. One of the largest cloud CRM implementations in the world
2. Mission-critical for Oracle
• 30k Users
• ~2 Million Customer accounts (Sales Accounts + Prospects)
• ~2.4 Million Addresses
• ~3 Million Contacts
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 11
Garbage -in
Garbage -out
But even with the BEST Cloud CRM…
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 12
Data Quality Challenges
• Good customer data is central to driving up revenue, improving customer satisfaction and understanding your business
• We had several challenges with our data undermining CRM effectiveness:
• Inbound customer flows were creating duplicate records
• Confusion for users in selecting the correct record to use for a transaction
• End to end process flows failed due to inconsistent or incorrect attribution delaying processes such as lead flow, quoting, order booking, shipping and invoicing
• Incomplete view of our customer interactions
• Less targeted sales and marketing
• Legal and legislative compliance issues
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 13
Benefits of a Data Quality program for CRM
• Improved visibility opening up cross sell and up sell opportunities
• Better targeted marketing
• A more complete view of the issues our customers are facing
• Improving customer satisfaction by ensuring their transactions move quickly and accurately
Revenue Cost
• Increased automation based on richer and more accurate records
• Less duplication of work such as sales rep assignment, credit and risk management, cleansing activities, agreements
Bottom-line: High quality data (complete, standardized, verified, de-duplicated) is a core requirement for CRM
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Confidential 14
Understanding the Problem
• Initial analysis was based on 2 strategic customers
• Duplication was far greater than reported, > 50% for large customers
• Automatic duplicate identification was an issue because of:
• Incomplete or inaccurate customer names, for example Oracle, Oracle ECE, Oracle Corp, Oracle East Central Europe Limited
• Different entities of the same company at the same address:
• Difficulty comparing addresses with differing text i.e. CA and California
• Synchronizing customers between systems had a high error rate
• Needed manual intervention to correct issues – for example due to badly formed address data
Oracle East Central Europe Limited Company Registration number 335354. Registered office is at East Point Business Park, Fairview Dublin 3.
Oracle EMEA Limited Company Registration number 158973. Registered office is at East Point Business Park, Fairview Dublin 3. VAT Registration Number IE 6556973 V
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 15
Master Data Management Program and Approach
• The Customer MDM program was developed to solve the fragmented and inconsistent data in Sales Cloud and its associated systems.
• No additional resources were required. Existing resources were focused on delivering a solution for the root of the issues rather than the symptoms
• Project team:
• Oracle Applications Labs – Solution design and implementation
• Businesss LOB stakeholders – including Sales, Marketing, Finance and Support – Implement business process changes, testing and signoff
• Executive Steering committee – Solution and business process approval
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 16
Solution
• Needed a consistent and highly automated data quality process which could be integrated into each system flow.
• Needed to be fast enough for real time integration
• Have enough capacity for bulk cleaning existing records, and for batch creation
• Flexible enough to determine where a manual review was required
Validate Cleanse Enrich Match Report Maintain
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 17
Data Quality Infrastructure
Enterprise Data Quality Layer
Address Verification
DnB Enrichment
Party Matching
ERP
(Billing)
Sales Cloud (CRM)
Data Validation
Store Systems
B2B Partners
Marketing Systems
M&A Data Migrations
End User
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 18
Validation
• Validate attributes are correct at the point of entry
• Ensure consistent standards are applied across all inbound sources
• Sensitive to transaction type and required velocity
• Notification of errors and warnings Data Object Requirement Error or Review?
Customer Name • Always Required
• In specific countries, such as Japan, ensure both English and local language names are
provided
Error
Address • Check for minimum required fields – address1, city, country, zip etc. Error
Contact • Always Required, needs either phone or email Error
Tax Data • Always required for specific countries Error / Review
Configurable
Validate Cleanse Enrich Match Report Maintain
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 19
Address Cleaning
• Address validation & standardization
• Critical component of the solution
• Standardization improves matching rates by ~30%
• Clean Addresses Improve automation of geography based processes
• Sales Rep assignment, tax calculation, export control compliance and shipping
• Accurate Geo values enable customer integration across systems with different address standards
• Local Language Addresses
• Many countries require addresses both in local language and English.
• Automation to provide translated address significantly reduces the need for manual intervention
Validate Cleanse Enrich Match Report Maintain
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 20
Address Cleaning
• GEO co-ordinates
• Required to support Field Service, Warehousing and Logistics processes.
• Foundation for mobile applications
Validate Cleanse Enrich Match Report Maintain
Input Data: "300 Berry #1210 SF California"
PremiseNumber
ThoroughfareName
SubPremise
Locality
AdministrativeArea
PostalCode
300
Berry St
Unit 1210
San Francisco
CA
94158-1670
Format
300 BERRY ST UNIT 1210
SAN FRANCISCO CA 94158-1670
USA Geocode
Latitude 37.775837
Longitude -122.39557
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 21
Enrichment
• Match inbound records against sourced data, such as Oracle DaaS service
• Provide Industry, Company size, Hierarchy Information
• Additional information used to improve duplicate detection
Validate Cleanse Enrich Match Report Maintain
Single Match
CREATED_MODULE CUSTOMER_NAME ADDRESS1 ADDRESS2 CITY STATE COUNTY POSTAL_CODE COUNTRY ORG TAX_PAYER_IDTAX_REG_NUMBER
Webreg Amcom Softw are 10400 Yellow Circle Dr. Suite 100 Eden Prairie MN Hennepin 55343 US 1001
PARTNER_STORE Life Quotes, Inc. 8205 S Cass Ave Ste 102Suite 102 Darien IL Dupage 60561 US 1001
XML_EDI CORNING INC. 21 Lynn Morse Drive PAINTED POST NY 14870 US 1001
DIRECT_STORE Jefferson Contracting, Inc.9390 Middlew ay Pike Charles Tow n WV Jefferson25414 US 1001
MATCHSCORE DUNS_NUMBER BUSINESS_NAME ST_ADDRESS1 ST_ADDRESS_2CITY_NAME STATE POSTAL_CODE COUNTRY_NAMESTATE_PROVINCE_NAMECOUNTY_NAMEURL
88 435545535 Amcom Softw are, Inc. 10400 Yellow Circle Dr Eden Prairie MN 553439153 USA Minnesota Hennepinw w w .amcomsoft.com
95 676585856 Life Quotes, Inc. 8205 S Cass Ave Ste 102 Darien IL 605615319 USA Illinois DuPage w w w .insure.com
90 976545644 Corning Incorporated 21 Lynn Morse Dr Painted Post NY 148709198 USA New York Steuben
93 564353456 Jefferson Contracting, Inc9390 Middlew ay Pike Charles Tow nWV 254144996 USA West VirginiaJefferson
Input
Output
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 22
Matching
• Needed a fast flexible matching solution that could:
• Provide both real time matching and batch processing capabilities
• API’s that could be integrated into every flow
• Ability to configure match rules specific to different systems
• CRM: Allow significant variation in name and consider geography to support sales territory assignment
• Finance: Higher accuracy in naming, match only within the same Oracle business unit, consider tax identifiers
• Recognize standard synonyms and account for typos and other errors
• Provide confidence score to allow records to be routed for manual review
Validate Cleanse Enrich Match Report Maintain
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 23
Matching
Validate Cleanse Enrich Match Report Maintain
DUNS Number (Exact Match)
Party Name (Exact Match)
Address Line1 (Exact Match)
Postal Code (Exact Match)
Country (Exact Match)
DUNS Number (Exact Match)
Party Name (Exact Match)
Address Line1 (Fuzzy Match)
Postal Code (Exact Match)
Country (Exact Match)
DUNS Number (Exact Match)
Party Name (Exact Match)
Address Line1 (Exact Match)
Country (Exact Match)
100
99
98
Match Score
Party Name (Fuzzy Match)
Address Line1 (Fuzzy Match)
Country (Exact Match)
Party Name (Exact Match)
Postal Code (Exact Match)
Country (Exact Match)
Party Name (Fuzzy Match)
Postal Code (Exact Match)
Country (Exact Match)
76
75
74
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 24
Reporting
• Move form reactive to proactive maintenance
• Profile data frequently to detect attribution issues
• Allow integration with issue notification systems
• Provide key metrics
Validate Cleanse Enrich Match Report Maintain
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 25
Results – Account Reductions, While Business Growing
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
FY12 FY13 FY14
Vo
lum
e
Finance – Active Accounts
Accounts
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Jun-12 Oct-12 Feb-13 Jun-13 Oct-13 Feb-14 Jun-14 Oct-14 Feb-15
Vo
lum
e
Sales Accounts – Active vs. Total
Total Pre EDQ Post EDQ Predicted
34% Reduction 44% Overall
Reduction
+40% Annual Growth – Pre EDQ
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 26
EDQ for Oracle Sales Cloud – Results
• Targeted marketing with 70% leads assigned to Preexisting Sales Accounts
• 72%+ reduction in net new prospect creation from inbound sources
• Manual creation of accounts reduced by 80% - cost saving and time reduction
• Reduction in >45K per annum manual interventions required to correct address issues etc
• 85% of all Sales Accounts are integrated to Finance resulting in improved transaction velocity
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Questions and Answers
Oracle OpenWorld 2014 27
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Connect with Oracle Data Integration
@OracleDI
Blogs.oracle.com/DataIntegration/
Oracle Data Integration
Oracle Data Integration
Oracle Confidential 29