Diving insights from data with Dell Cloudera big data Hadoop solutions
-
Upload
dell-world -
Category
Business
-
view
256 -
download
2
description
Transcript of Diving insights from data with Dell Cloudera big data Hadoop solutions
1 Dell World 2014
Driving insights from data with Dell │ Cloudera big data Hadoop solutions
Joey Jablonski, Enterprise Technologist, Office of the CTO, Dell Rami Lokas, Director of Research, Omneo
Dell World 2014
2 Dell World 2014
Get past the hype to turn data into insights
Saving lives Reducing emissions
Delighting customers
A hospital makes real-time decisions that prevent life-threatening infections after surgery.
A steel producer optimizes monitoring and reporting and manages costs.
A financial institution tailors offers to deliver a superior customer experience.
Make data the lifeblood of the business for competitive advantage.
3 Dell World 2014
You can make data the lifeblood of your business
Implement effective data management • Focus on all data • Leverage existing
investments • Optimize performance • Break down silos
Enable the business • Connect intelligence • Build analytics capabilities • Deliver secure self-service
access
Commit to IT/business collaboration • Define objectives • Assess environment • Measure
performance
Make data the
lifeblood of your
business
4 Dell World 2014
Build analytics capabilities to extract insights from all data types
Business reporting
and analysis
Data integration and
consolidation
Data collection and
basic analysis
Predictive analytics
Cognitive
analytics
“Who are my top customers?”
“Which are my top performing sales regions?”
“How are we performing against the organizational goals?”
“What is the optimal inventory based on historical trend?
“Would they recommend us to a friend?”
Inc
reasi
ng
matu
rity
5 Dell World 2014
IT economics Business outcomes
• Acquisition
• Support
• Skills
• Overhead and royalties
• Speed business execution
• Improve operations
• Increase revenue
• Improve products
Defining big data initiative ROI
6 Dell World 2014
Assuring big data analytics capability across the user base
Infrastructure and management
Data integration
Analytics LoB, BU, data scientist
DBA, data scientist
IT architect, systems engineer
7 Dell World 2014
Services: IT/business alignment, infrastructure readiness, analytics maturity, metrics
Infrastructure
Put the right data in the right
place at the right time
Management
Improve data platform
performance
Integration
Get real-time data movement
Turn data into insights for
better, faster decisions
Advanced analytics & BI
Platforms: Hadoop, Oracle, SQL Server, Sybase, DB2, MySQL and more.
Comprehensive solutions enable your success
Partners: Cloudera, Intel, Microsoft, Oracle, SAP
8 Dell World 2014
Dell data management solutions
• Reference architecture
• Flexible
• Validated
• Open source
• Built on PowerEdge
• Prescriptive
• Pre-deployed
• Faster deployment
• Turn real-time data into insights
• Bundled
• Open, secure
• Simplifies procurement and deployment
• Aggressive entry price point
• Customer guidance
• Use-case-based solutions
• End-to-end
• Hadoop design guide
• Cassandra, MongoDB
Meets different customers’
needs
Dell │ Cloudera for Apache Hadoop
Dell In-Memory
Appliance for Cloudera Enterprise
Dell QuickStart for Cloudera Hadoop
Custom big data solutions
Faster time to insights
Quick response to business
needs
Quickly overcome
barriers
Dell World 2014
Rami Lokas Director of Research, Omneo
10
Business goal To enable brand owners to
manage product performance
and customer experience
Dell World 2014
Solution Dell │ Cloudera solution on top of Cloudera Distribution Hadoop® platform on a cluster of Dell PowerEdge C8220 servers
Benefits • 360 degree view of supply chain data
• Search billions of records in less than three seconds
• Scales to support 300M records every month
• Detect emerging issues
• Save millions of dollars and boost productivity
• Improve product quality, performance, customer experience, and compliance
11 Dell World 2014
What exactly is “supply chain”?
Raw Materials Supplier Manufacturing
Distribution Customer Consumer
12 Dell World 2014
Plant
Supply chains are complex and fast-paced
13 Dell World 2014
How do you manage quality for
1 million customers 80,000 parts
1600 products 200 suppliers
2 billion annual events
14 Dell World 2014
Imagine answering questions in 3 seconds or less.
15 Dell World 2014 0
10
10
10
1
01
01
01
00
10
11
Design Manufacturing Quality
Procurement Engineering Field
Your supply/value
chain Supplier
Mfg Sites
ODMs CMs Service
Call Center
Service Repair Depot
01
01
01
01
0
01
01
01
01
0
01
01
01
01
11
01
00
01
01
0
01
01
01
01
11
01
00
0
Supplier test and ship data
As built data Dispatch and RMS data
Repair and refund data
Call home data
Design Spec Data
Failure analysis data
Assembly test data
Field device data
Service data
Creates siloed, fragmented
data
Only some of it can be
consumed
Complex supply chains create data chaos
16 Dell World 2014
Data challenges result
• Massive data volumes of various formats • Disconnected data sources • Missing fields and inconsistent attributes • Not available when needed • Not in context of supply chain product
performance
and the existing RDBMS solution wasn’t cutting it
17 Dell World 2014
Solution evaluation criteria
• Functional fit
• Technical fit
Performance
Scalability
Fault tolerant
Highly available
• Time to market
• Time to value
• TCO
18 Dell World 2014
Technical assessment
Solution Functional
fit Technical fit
Time to market
Time to value
TCO Overall score
Private Cloud with Oracle EE 3 2 5 2 1 2.6
Private Cloud with Hadoop/Dell/Intel, Pentaho ++
5 4 3 4 5 4.2
Private Cloud with Netezza Appliance, Pentaho ++
5 5 4 4 2 4.0
Amazon EC2 with Hadoop, InfoBright, Pentaho ++
5 3 3 4 3 3.6
Azure with Hadoop and SQL Azure, SQL BI ++
3 2 1 3 3* 2.4
(*): Assumed to be comparable to Amazon EC2
Optimal solution
5 Excellent
4 Good
3 Average
2 Poor
1 Unacceptable
19 Dell World 2014
Cost assessment
20 Dell World 2014
Why Dell│Cloudera big data Hadoop solution
Industry-leading performance, cost-effectiveness, scalability, and open source promise of flexibility
Complete-ness of Hadoop cluster
offering
Cloudera relationship
and top 5 supply chain
Widespread adoption of Hadoop and
rapid innovation
Superb enterprise-
grade support and easy to get
started
Rich set of modules to
address real-time nature
21 Dell World 2014
Omneo solution
• SaaS, multi-tenant enterprise data and analytics hub built on Cloudera
• Ingesting data from design systems, factories, suppliers, customer call centers, field services, after-market repairs and re-manufacturing
• MapReduce, HBase, Cloudera Search, Impala and Parquet in use to support different use cases
22 Dell World 2014
Unified together
For every stakeholder
Any data type
Design Manufacturing Quality Procurement Engineering Field
Supplier test and ship data
As built data
Dispatch & RMS
data
Repair and refund data
Call home data
Design spec data
Failure analysis data
Assembly test data
Field device
data
Service data
All data sources Supplier
Mfg Sites
ODMs CMs Service
call center
Service repair depot
Omneo clarifies and unifies product data
23 Dell World 2014
How Omneo works
• A new end state for your product big data
– “Garbage-in/Gold-out” and contextualize all data no matter where it exists
• Centralized, cloud-based supply chain data
– Hours vs weeks: transform, store and contextualize terabytes
– Fault Tolerant Storage
• Validate and learn
– Data validation during ETL and rapidly analyze and diagnose based on errors generated from aggregation and contextualization
• Adapt
– NOT another static Enterprise Data Warehouse schema
– Designed for continuous adaptation and domain-specific usage
• Improve
– Flexibility + eternal source data storage → Continuous improvement
– Conduct frequent design of experiments
– Constantly refine assumptions and improve context based on changing needs
Apps
Omneo Application Builder
Omneo Unified Model
Omneo Metadata
RAW Data
Models
24 Dell World 2014
Data Interchange
Gateway
Remote data
collectors
Loader
Existing
Summarize Contextualize
Dispatch
Big Data Store
Flexible Schema De-normalized Pre-aggregated Multi-Tenancy
Operational reporting
4-Multitenant Data Structures
Mining Patterns Store
Extract Transform
MES CIO
External system
Collaboration Test EQP
CSS Probe
Contextual search
Multi-tenant deployment and multi-tier role-based security
Unified end-to-end supply chain model increases visibility
Very large scale, fully searchable data store
Query response times with low latency- under 3 secs.
Accuracy enhancing data quality cleansing
Release 1 Release 2
Camstar Cloud System
Management
Interactive analysis
Predictive analysis
Discover
Knowledge Discovery
Q4, 2014
Release 3-X
Omneo conceptual architecture
25 Dell World 2014
Vision: business process improvement
Diagnostic
Why? How can I control it?
Prescriptive Descriptive
What happened?
Predictive
When will it happen again?
Fast contextual search
Performance analytics
Root cause analytics
Machine learning
26 Dell World 2014
Search
Result counts
Search results
Omneo in action: search
27 Dell World 2014
Ask the questions you don’t know to ask
Rapidly identify outliers
Omneo in action: discovery
28 Dell World 2014
Omneo in action: monitoring
KPT tiles
Performance monitoring
FCS Charts
29 Dell World 2014
Omneo in action: exploration
Issue discovery
Analysis drill-down
30 Dell World 2014
Omneo search query performance
0.0
0.5
1.0
1.5
2.0
2.5
3.0
1,494 1,014 281 106 73 46 24 9 9 8 8
Re
spo
nse
Tim
e i
n S
ec
on
ds
Matching Records in Millions
Query time(s) searching 1.5B records
Query time (s)
Trendline (s)
Powered By
31 Dell World 2014
The “secret sauce”
32
Outcomes
Dell World 2014
• 360 degree view of supply chain data
• Investigations that would take weeks or
months can be answered in minutes or hours
• Search over three billion records in less than
three seconds
• Clients report savings of $15-25M each due to
insights
• Infinitely scalable to support 300M new
events/month
• Analyzes 1.2M product dimensions in under 1
minute
33
Lessons
Dell World 2014
• Understand the limitations and constraints of
traditional RDBMS and data warehouses
• Domain, domain, domain! Use cases are
critical to creating value and a reusable,
scalable, rapid solution
• Identify, assess, and qualify data and its
quality
• Identify high value, low data quality barrier use
cases for early wins
• Establish Master Data Management and data
governance with high-value data sources
34 Dell World 2014
Visit Dell Big Data in the Dell World Expo
Dell World 2014
Thank you.
Want to learn more about Dell’s enterprise solutions?
Learn via email. Start here.
Sign me up!