Dance with the elephant & Keeping all of your toesAn Architect’s Guide to Big Data
Robert Stackowiak, Oracle North America, VP Information Architecture & Big Data
Doug Cackett, Oracle EMEA, Chief Architect
September 30, 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 3
Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 4
Program Agenda
Framing the conversation
A methodology to deliver success
Defining future state architecture
How to avoid the oncoming elephant
Next Step
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Silos Standardize Optimize Information as a Platform
• One off tools/solutions• Bottom-up• Local business unit
driven• Many versions of truth• Independent data
marts
• Enterprise standard tools/solutions
• Data warehouse and dependent data marts
• IT & LOB partnership• Secure, consistent
access to all data
• Agile, flexible architecture
• Analyze “just in time” structured & unstructured data together
• Advanced analytics & real-time recommendations
• All stage 3 capabilities delivered as a platform (service/cloud)
• Access of tools and data among broad subscriber group
Maturity & Capability
Unified Information Architecture Maturity Phases
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Big Data Footprint & Scope of Architecture
zBY INDUSTRY & LINE OF BUSINESS
BIG
DAT
A AP
PLIC
ATIO
NS
DISCOVERY
BUSI
NES
SAN
ALYT
ICS
BUSINESS ANALYTICS
DATA RESERVOIR
BIG
DAT
AM
ANAG
EMEN
T
DATA WAREHOUSE
SOU
RCES
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Architecture Vision Common Emerging Platform Pattern
Data Warehouse / Data Marts
Business Intelligence Tools
ERP, CRM & Other Transactional Apps
Historic Source of Truth Reporting, Query and Analysis Tools
Information Discovery Engine
Advanced Analytics
Website Logs & Data NoSQL DBSensors
Hadoop High Volume Distributed File System
Structured Data
Semi-structured Data
Real-Time Analyticsand Recommendations
Recommend Location & User Profile
R, SAS
Discoveries
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Potential Oracle Products in the Footprint
Endeca Information Discovery on Exalytics
Cloudera HDFS on Big Data Appliance
Reliable, Available, Secure Source of Truth
Fast, Intuitive Data Discovery
Website Logs & Data Oracle NoSQL DB
Real-time Recommendations
Analyst Friendly Reporting Query & Analysis Tools
Unstructured Data Analysis
Sensors
Oracle Database DW on Exadata
Oracle BI Foundation Suite, Hyperion on
Exalytics
Oracle ERP & CRM Solutions on Exadata
OracleReal-Time Decisions
Structured Data Analysis
Big Data Connectors
ODI
OEP
Advanced Analytics, In-Memory, Big
Data SQL
R, SAS
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Extending to the Internet of Things
Smart Sensor
IoT Comm. Agent
Application Framework
Sensor
IoT Comm. Agent
Application Framework
Gateway
Application Framework
IoT Comm. Agent &
Mgmt Proxy
IoT Comm. Gateway
Protocol Mediation
Web Services Security
Web Services Mgmt
IoT Identity & Access Mgmt
Device Access Management
Device Identity Management
Identity Directory/Database
IoT Management
Device Registration & Mgmt
Message Cache
Management Console
Applications Provisioning
Data Routing & Analysis
Distributed Data Grid
Complex Event Processing
Big Data
Big Data Store
Data Aggregator
NoSQL DB
Key Value Data Store
Database
RDBMS
Business Intelligence
Performance Management
Alerts, Dashboards & Reports
Advanced Analysis & Data Science
Tools
Data & Knowledge Discovery Tools
Enterprise Integration
Complex applications integration & SOA
DatacenterManagement
Server Management & Monitoring
Gather Enrich Stream Manage Acquire Organize & Analyze
Local Event Analysis
Ca
pil
lary
Ne
two
rks
Ne
two
rk F
ire
wa
ll
Intr
an
et
Fir
ew
all
Da
tab
as
e F
ire
wa
ll
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
IT Focus
Implementation Rodmap
(IT & LOBs)
Business Information Mapping (LOBs)
Visioning & Innovation (IT & LOBs)
Technical Information Architecture (IT)
Use Case Hypothesis (LOBs)
Proven Methodology for Success
Implementation (IT, SI & LOBs)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Sample Use Case including Critical Success Factors & Measures
Perspectives
Financial
Optimize Channels and measure effectiveness of new product launches
Customer
Increase self-qualification, understand customer needs then targeting
Internal
Better simplify & optimize retail channels for better
close rates
Innovation & Growth
Measure success of campaigns &
advertising, better relationship
management
Critical Success Factors
Regional financial optimization- products / sales
Accurate cost analysis
Measure new launch success
Critical Success Factors
Self-serve web experience tied to right channel
Understand customers lost
Critical Success Factors
Right level of complexity in sales & order process
Reduce order cycle time
Critical Success Factors
Right ads and promotions at right markets / right time
Ability to measure impact
Improve Sales through All Channels (by 22 percent)
MeasuresSales, marketing expense,
markups /discounts and win rate, cost, price sold at, margin, warranty & service
MeasuresEnd-to-end sales process &
interactions, sentiment, reasons for losses, close ratio, customer retention
Measures Leads converted, time to close,
sales cost, requests for info, touch point effectiveness
MeasuresAd expense, sales, customer
satisfaction, market share potential vs. sales attained
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Defining a Business Information Map / Data Flow
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Defining a Roadmap with Achievable Goals & Payback
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 14
Rethinking Information Architecture
• Design for Analytics 3.0• Connect information to execution• A platform for all data• Establish an architectural framework• Enable discovery to keep it relevant• Automate to manage and govern
Architect to avoid getting trampled by the oncoming elephant
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Data as the Foundation for Business… but no longer the complete answer – agility is the key
Analytics 2.0
Analytics 3.0
Analytics 1.0
• Reporting with limited use of descriptive analytics
• Limited range of tabular data • Batch oriented analysis • Analysis bolted onto limited set
of business processes
• Firms “Competing on Analytics”• Extended analytics to larger and
less structured datasets• Emergence of Big Data into the
commercial world• Recognition of Data Science
• Platform for monetisation• Deeper analysis & more data• Faster test-do-learn iterations• Different types of data & wider
business process coverage• Analysts focus on discovery and
driving business value• Agile with operational elements
incorporated in design patterns
Adapted from Tom Davenport
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 17
Manage all the data in a Big Data Management SystemBut find the right place to put it
Tooling maturity
Availability & Business Continuity
ACID transactional requirement
Security
Variety of data formats
Data sparsity
Ingestion (coding) simplicity
Cost effectively store low value data
Ingestion rate
Straight Through Processing (STP)
0
5
HadoopRelationalMy Ap-pllication
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 18
Information Interpretation
Establish an architected frameworkBalancing agility with ingestion and interpretation costs
Virt
ua
lisa
tion
&
Qu
ery
Fe
de
ratio
n
Enterprise Performance Management
Pre-built & Ad-hoc BI Assets
InformationServices
Data Science
Data Ingestion Access & Performance Layer
Foundation Data Layer
Raw Data Reservoir
Data Sources
Data Engines & Poly-structured sources
Content
Docs Web & Social Media
SMS
StructuredDataSources
•Operational Data•COTS Data•Streaming & BAM
Master & ReferenceData Sources
• Incr
easin
g qua
lity &
enric
hmen
t
• Redu
cing c
osts
of qu
ery c
oncu
rrenc
y
• Incr
easin
g for
mali
satio
n of d
efini
tion
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 19
Recognise the value of discovery
• Discovery & exploitation require differing people, process & technologies
• Design to minimise time taken and size of governance step
• Agile BI must be part of the design thinking for ongoing commercial exploitation
Discovery is about data understanding NOT IT development
Commercial Exploitation
Discovery phase
Understanding of the data
Governance
Business Value
Time / Effort
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 20
Establish a data discovery capabilityAccelerate with automated provisioning / de-provisioning steps
Analysis Processing & Delivery
Discovery Lab & Data Science Tooling
Data Reservoir & Enterprise Data
Data Science(Primary Toolset)
Statistics Tools
Data & Text Mining Tools
Faceted Query Tools
Programming & Scripting
Data Modeling Tools
Query & Search Tools
Pre-BuiltIntelligence
Assets
IntelligenceAnalysis
Tools
Ad Hoc Query& Analysis Tools
OLAP Tools
Forecasting &Simulation Tools
Reporting Tools
Data Scientist
Vir
tua
lisa
tion
&
Info
rma
tion
Se
rvic
es
Data Factoryflow
Data Quality & Profiling
Graphical rendering tools
Dashboards & Reports
Scorecards
Charts & Graphs
Sandbox – Project 3
Sandbox – Project 2
Sandbox – Project 1
Data store Analytical
Processing
General BI flow
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 21
Next Step: Contact Oracle for Help
• North America– Information Architecture & Big Data: [email protected]– Oracle Enterprise Architects / Information Architecture: [email protected]
• EMEA– Information Architecture & Big Data: [email protected]
• Download the Big Data & Information Management white paper here
Top Related