Denodo DataFest 2017: Get More Business Value with Certified Data Marketplace
Denodo DataFest 2017: Conquering the Edge with Data Virtualization
-
Upload
denodo -
Category
Data & Analytics
-
view
51 -
download
4
Transcript of Denodo DataFest 2017: Conquering the Edge with Data Virtualization
Conquering the Edge with Data Virtualization
Lakshmi Randall
Director of Product Marketing, Denodo
Twitter: @LakshmiLJ
2
Source: Gartner Blog
(http://blogs.gartner.com/thomas_bittman/2017/03/06/the-edge-will-eat-the-
cloud/)
The edge will eat the cloud. And this is
perhaps as important as the cloud
computing trend ever was.”
Which workloads will need an edge presence?The need for an edge presence will be defined & driven
by data requirements …
SpeedAvailability& reliabilityof access
Rate of generation or security
…or any combination
Source: 451 Research
3
Which deployment locations do you plan to use to store and analyze IoT data in 2017Source: 451 Research, Voice of the Enterprise: IoT Budgets and Outlook, 2017. n = 288
4
5
IoT Network Architecture
• IoT networks will contain multiple ‘layers’ of compute systems
• Each layer will persist data
• Some storage (e.g. at the sensor) might be temporary
• e.g. last hour of readings
• Not all data can be transmitted to next layer
• Regulatory, privacy, security reasons leaving data where it is
• Data moving to higher levels is typically aggregated for analytics
and monitoring
6
Layer 1 Layer 2 Layer 3 Layer 4
Layered Architecture
7
Data Virtualization: Five Essential Capabilities
4. Self-service data services
5. Centralized metadata, security & governance
1. Data abstraction
2. Zero replication, zero
relocation
3. Real-time information
8
• Data Virtualization provides data access at the edge layer and data center/cloud layer
• Connected DV platforms optimize data access by pushing processing to the data
• Data Virtualization can be deployed at any intermediary layers between ‘edge’ and data center/cloud layer
• e.g. Organizational, Regional, or Country based layers
Data Virtualization at the Edge
9
Connect rather than Collect ParadigmCase Study Example from Gartner
Single Logical View
10
DV Layer @ EdgeDV Layer @ Data
Center/Cloud
Layer 1 Layer 2 Layer 3 Layer 4
Layered Architecture
Business Need Solution Benefits
11
Leading Heavy Equipment Manufacturer Improves Service Delivery and Revenue
• Competitive pressure from low-cost Chinese manufacturers
• Needed a proactive approach to customer service to differentiate
• Sought to improve equipment and services delivery through predictive maintenance
• Phased rollout systematically improved asset performance and proactive maintenance
• Increased revenue from sale of services and parts
• Reduced warranty costs of parts failure
• Future – optimize pricing for services and parts among global service providers
• Telemetry (IoT) data from sensors embedded in the equipment is stored in Hadoop to perform predictive analytics
• Denodo integrates analytics data with parts, maintenance, and dealer information stored in traditional systems
• It then feeds the predictive maintenance information to a customer dashboard
In business for over 90 years and is the world’s leading manufacturer of construction and mining equipment, diesel and natural gas engines, industrial gas turbines and diesel-electric locomotives with 2014 net revenue of $54.6 billion.
Denodo @ Layer 4 (Data Center/Cloud)
12
Denodo @ Layer 4 (Data Center/Cloud)Telemetry Analytics
Business Need Solution Benefits
13
World’s Largest Independent Biotechnology Firm Successfully Increases Production Yield and Plant Efficiency
• Manufacturing data had to be aligned in real time to identify “weak signals” that could hinder production and cost millions
• Streaming data from manufacturing, laboratory facilities, etc. had to be analyzed for optimal plant efficiency
• “Golden Batch” had to be defined for increased production yield after combining current/historical data
• Improving yield by 1–3% has saved $10 –30 million per plant
• Concerns with operating a new plant at low capacity are eliminated
• ROI of Denodo easily justifies all work –will be used for all future applications
• Data Virtualization through Denodo is cheaper and more efficient than ETL
• Denodo Platform combines current manufacturing process data with historical data and feeds it to analytical systems
• The systems analyze virtualized data, determine “golden batch,” and take corrective steps to ensure ideal yield
• Denodo combines data from 46 source systems, of which 12 are key systems
Headquartered in Thousand Oaks, California, the company is the world’s largest independent biotechnology firm, and sells a number of products treating autoimmune diseases, rheumatoid arthritis, etc. In 2015, the company reported a revenue of $20 billion with its 18,000 employees.
Denodo @ Layer 3 (at the Edge)
14
Denodo Beyond the Edge
• Data Virtualization can also be leveraged beyond the edge – at
local data collection level
• e.g. factory or production line, retail store, ICU in hospital, etc.
• Collection point for real-time data streams from sensors, devices
• Stored in local databases, files, etc.
• For sensor management and control
• DV provides data access for analytics and reporting
• Lightweight deployment for lower capability systems
• e.g. Denodo in Docker Container
15
DV Layer @ EdgeDV Layer @ Data
Center/CloudLightweight DV Layer
Beyond Edge
Layer 1 Layer 2 Layer 3 Layer 4
Layered Architecture
Business Need Solution Benefits
16
Large US Telecom Company Improves Inventory Alignment with Virtualized Master Data Views
• Business users needed single view of networking device, customer, and site information from systems
• The single views were needed for ticket and event enrichment, compliance and audit, and inventory alignment
• The information was dispersed across multiple systems with varying levels of completeness and data quality
• Customer satisfaction: High quality MDM data helped end customer receive better quality information
• Inventory alignment: Improved information availability to internal and external applications
• Improved compliance: Better inventory management and analytics helps with better compliance and audit standards
• Denodo is used as a virtualized master data management (MDM) system
• Denodo connects to multiple systems and Hadoop
• Denodo aggregates and exposes the master data views via an integrated API layer
• Data is consumed by ticket and event enrichment, dashboard, and compliance / audit applications
The company is the second largest cable company in the U.S. by revenue, operating in 29 states, with corporate headquarters located in Midtown Manhattan, New York City. The company recorded revenue of US$23.69 billion in 2015. It’s Network Operations group is a thriving business providing the entire range of telecommunications services to both individual and commercial consumers.
Denodo Beyond the Edge
17
Denodo and Streaming Data
• Integrate with streaming data
systems e.g. Spark, Kafka, IBM
Streams, etc.
• Read data from queues
or streams (e.g. Spark
data frames) during
ingestion process
• Also enrich streaming
data through Denodo
Platform
• Access temporal data in specific
time window
• e.g. last 2 hours of
readings/data
Read temp window buffers
SQL basedenrichment
Secure +
Combine +
Enrich
18
Complete enterprise information, combining Web, cloud, streaming, and structured data
ROI realization within 6 months, with the flexibility to adjust to unforeseen changes
An 80% reduction in integration costs, in terms of resources and technology
Real-time integration and data access, enabling faster business decisions
“Get it Real-time and Get it Fast!”
The Benefits of Data Virtualization
Thank you!
© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.
#DenodoDataFest