Denodo DataFest 2017: Conquering the Edge with Data Virtualization
Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration
-
Upload
denodo -
Category
Data & Analytics
-
view
115 -
download
0
Transcript of Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration
O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A
#DenodoDataFest
RAPID, AGILE DATA STRATEGIESFor Accelerating Analytics, Cloud, and Big Data Initiatives.
The Role of Data Virtualization in IoT Integration
Lakshmi Randall
Head of Product Marketing, Denodo
Twitter: @LakshmiLJ
Agenda1.What’s so important about IoT Integration?
2.How does Denodo support IoT Data Integration?
3.Customer Case Study
3
IoT is invading our kitchen!
4
What’s so important about IoT Integration?
5
The Importance of IoT Integration
Investment in IoT devices is soaring
IoT is proliferating across all business and
consumer sectors
Data generated in the IoT offers a Data
Monetization Model
6
IOT Investment
2016: IoT hardware purchases surpass
$2.5 million per minute.
2021: one million IoT devices procured
and installed per hour.
IoT Proliferation
2018: 6 billion connected devices
require support.
2020: >21 billion connected devices
in operation.
2020: industry-specific devices number
2.9 billion (nearly 200% growth since
2015).
IoT Investment & Proliferation Milestones
7
Source: Gartner 2016
IoT Monetization
Enhance traditional products with sensors
and connectivity
Offer bundled services for connected things
(e.g., connected cars)
Collect, Aggregate, Anonymize and
Monetize.
8
IoT Use Cases
9
Preventative & Proactive
Maintenance
Data Monetization (information-
oriented products & services)
Customer Satisfaction &
Retention
Operational Efficiency (asset &
equipment optimization)
Safety Security
Fraud DetectionReal-time Analytics
Patient care
IoT Endpoints for 2020 by Sector
10
Source: Gartner, April 2016
How does Denodo support IoT Data Integration?
11
Data-in-transit and Data-at-rest
Big Data Connectivity
BigData and Cloud Databases Connectivity
■ Hadoop Ecosystem:
■ SQL on Hadoop: Hive, Impala, Presto,…
■ HDFS, Parquet, Avro, CSV…
■ Execution of map/reduce Jobs
■ Certified with major Hadoop distributions
■ In-memory platforms: Apache Spark, Presto DB, HANA,…
■ Parallel DWs and Appliances: Vertica, Impala, Teradata, Greenplum,…
■ Cloud RDBMS: Redshift, Snowflake, DynamoDB,…
■ NoSQL (MongoDB, CouchDB, Neo4J, Redis, Oracle NoSQL, Cassandra, etc.)
■ Streaming data (Spark streams, Splunk, IBM Streams, Kafka,…)
12
Enhanced Adapters for Big Data ecosystem
13
Request-Response:Named adapters for stream services: Kafka IBM Streams
Streaming:Extend current JMS support with: Enhanced support for
temporary windows Support for MQTT
Enhanced Integration with IoT - StreamingEnhanced Adapters for the Internet of Things Ecosystem
JMSMQTT
JMSMQTT
Data Ingestion
■ Batch, On-demand and Streaming Data
Ingestion
■ Simultaneously supports Batch and
Streaming data integration
■ Learns to extract structured data from
semi-structured content using Machine
Learning
■ Ingest the data in a schema-agnostic way
including schema-on-read and multiple
schemas
14
Batch, On-demand and Streaming Data Ingestions
Enrich Machine Data and Combine with Other DataIngest, Integrate & Deliver
Persisted(In-memory, Hadoop)
Streams(specific time window)
Message Queue
Machine-generated/Event data Alerts
Workflows
Operational Processes
Analytical Processes
Consumers
Visualization
Data VirtualizationEnrich and Combine IoT Data with Other Data
Historians
Streams
ERP/SCM
DW
AnalyticalDB
MDM
Apps
Data Marts
Hadoop NoSQL
16
Security
Data in Motion – secure channels
• Using SSL/TLS
• Client-to-Denodo and Denodo-to-source
• Available for all protocols (JDBC, ODBC, ADO.NET and WS)
Data at Rest – secure storage
• Cache: third party database. Can leverage its own encryption mechanism
• Swapping to disk: serialized temporarily stored in a configurable folder that can be encrypted by the OS
Encryption/Decryption and Data Masking
• Support for custom decryption for files and web services
• Transparent integration with RDBMs encryption
Authentication and Authorization
• LDAP/AD, Kerberos support, Granular data security,
Securing data
Customer Case Study
17
Leading Construction Manufacturer - Telematics & Predictive Maintenance
Dealer
Maintenance
Parts Inventory
OSI PI Hadoop Cluster
Tableau: Dealer / Customer Dashboard
Business Benefits
Improved asset performance and proactive maintenance.
Reduced warranty costs due to proactive maintenance of
parts preventing parts failure.
Optimized pricing for services and parts among global service
providers.
New Business Model opportunities based on real-time
analysis of detailed sensor data.
Data Virtualization BenefitsImplement a Single Logical Data Lake Using Data Virtualization
Improves the enterprise func-tionality of data
lakes by combining one or
more physical data lakes with other enterprise
data
Provides a way to access data from separate systems
through an abstraction layer
that makes it appear as if the data were in a single data lake
Improves an organization’s
ability to govern and extract more
value from its data lakes by
extending them as logical data
lakes
20
Key Takeaways
Identify if and how IoT data will benefit your organization
Identify your potential IoT Data sources
Employ Data Virtualization to combine IoT data with other data to
enhance the use and value of data assets
Employ a Logical Data Lake/Logical Data Warehouse architecture to
eliminate the cost of storing information in multiple places, to
govern IoT data access, and to prevent IoT data from becoming
siloed.
21
Q&A
Thank you!
© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.
O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A
#DenodoDataFest