Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

23
OCTOBER 18,2016 SAN FRANCISCO BAY AREA, CA #DenodoDataFest RAPID, AGILE DATA STRATEGIES For Accelerating Analytics, Cloud, and Big Data Initiatives.

Transcript of Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Page 1: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A

#DenodoDataFest

RAPID, AGILE DATA STRATEGIESFor Accelerating Analytics, Cloud, and Big Data Initiatives.

Page 2: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

The Role of Data Virtualization in IoT Integration

Lakshmi Randall

Head of Product Marketing, Denodo

Twitter: @LakshmiLJ

Page 3: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Agenda1.What’s so important about IoT Integration?

2.How does Denodo support IoT Data Integration?

3.Customer Case Study

3

Page 4: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

IoT is invading our kitchen!

4

Page 5: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

What’s so important about IoT Integration?

5

Page 6: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

The Importance of IoT Integration

Investment in IoT devices is soaring

IoT is proliferating across all business and

consumer sectors

Data generated in the IoT offers a Data

Monetization Model

6

Page 7: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

IOT Investment

2016: IoT hardware purchases surpass

$2.5 million per minute.

2021: one million IoT devices procured

and installed per hour.

IoT Proliferation

2018: 6 billion connected devices

require support.

2020: >21 billion connected devices

in operation.

2020: industry-specific devices number

2.9 billion (nearly 200% growth since

2015).

IoT Investment & Proliferation Milestones

7

Source: Gartner 2016

Page 8: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

IoT Monetization

Enhance traditional products with sensors

and connectivity

Offer bundled services for connected things

(e.g., connected cars)

Collect, Aggregate, Anonymize and

Monetize.

8

Page 9: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

IoT Use Cases

9

Preventative & Proactive

Maintenance

Data Monetization (information-

oriented products & services)

Customer Satisfaction &

Retention

Operational Efficiency (asset &

equipment optimization)

Safety Security

Fraud DetectionReal-time Analytics

Patient care

Page 10: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

IoT Endpoints for 2020 by Sector

10

Source: Gartner, April 2016

Page 11: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

How does Denodo support IoT Data Integration?

11

Data-in-transit and Data-at-rest

Page 12: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Big Data Connectivity

BigData and Cloud Databases Connectivity

■ Hadoop Ecosystem:

■ SQL on Hadoop: Hive, Impala, Presto,…

■ HDFS, Parquet, Avro, CSV…

■ Execution of map/reduce Jobs

■ Certified with major Hadoop distributions

■ In-memory platforms: Apache Spark, Presto DB, HANA,…

■ Parallel DWs and Appliances: Vertica, Impala, Teradata, Greenplum,…

■ Cloud RDBMS: Redshift, Snowflake, DynamoDB,…

■ NoSQL (MongoDB, CouchDB, Neo4J, Redis, Oracle NoSQL, Cassandra, etc.)

■ Streaming data (Spark streams, Splunk, IBM Streams, Kafka,…)

12

Enhanced Adapters for Big Data ecosystem

Page 13: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

13

Request-Response:Named adapters for stream services: Kafka IBM Streams

Streaming:Extend current JMS support with: Enhanced support for

temporary windows Support for MQTT

Enhanced Integration with IoT - StreamingEnhanced Adapters for the Internet of Things Ecosystem

JMSMQTT

JMSMQTT

Page 14: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Data Ingestion

■ Batch, On-demand and Streaming Data

Ingestion

■ Simultaneously supports Batch and

Streaming data integration

■ Learns to extract structured data from

semi-structured content using Machine

Learning

■ Ingest the data in a schema-agnostic way

including schema-on-read and multiple

schemas

14

Batch, On-demand and Streaming Data Ingestions

Page 15: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Enrich Machine Data and Combine with Other DataIngest, Integrate & Deliver

Persisted(In-memory, Hadoop)

Streams(specific time window)

Message Queue

Machine-generated/Event data Alerts

Workflows

Operational Processes

Analytical Processes

Consumers

Visualization

Data VirtualizationEnrich and Combine IoT Data with Other Data

Historians

Streams

ERP/SCM

DW

AnalyticalDB

MDM

Apps

Data Marts

Hadoop NoSQL

Page 16: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

16

Security

Data in Motion – secure channels

• Using SSL/TLS

• Client-to-Denodo and Denodo-to-source

• Available for all protocols (JDBC, ODBC, ADO.NET and WS)

Data at Rest – secure storage

• Cache: third party database. Can leverage its own encryption mechanism

• Swapping to disk: serialized temporarily stored in a configurable folder that can be encrypted by the OS

Encryption/Decryption and Data Masking

• Support for custom decryption for files and web services

• Transparent integration with RDBMs encryption

Authentication and Authorization

• LDAP/AD, Kerberos support, Granular data security,

Securing data

Page 17: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Customer Case Study

17

Page 18: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Leading Construction Manufacturer - Telematics & Predictive Maintenance

Dealer

Maintenance

Parts Inventory

OSI PI Hadoop Cluster

Tableau: Dealer / Customer Dashboard

Page 19: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Business Benefits

Improved asset performance and proactive maintenance.

Reduced warranty costs due to proactive maintenance of

parts preventing parts failure.

Optimized pricing for services and parts among global service

providers.

New Business Model opportunities based on real-time

analysis of detailed sensor data.

Page 20: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Data Virtualization BenefitsImplement a Single Logical Data Lake Using Data Virtualization

Improves the enterprise func-tionality of data

lakes by combining one or

more physical data lakes with other enterprise

data

Provides a way to access data from separate systems

through an abstraction layer

that makes it appear as if the data were in a single data lake

Improves an organization’s

ability to govern and extract more

value from its data lakes by

extending them as logical data

lakes

20

Page 21: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Key Takeaways

Identify if and how IoT data will benefit your organization

Identify your potential IoT Data sources

Employ Data Virtualization to combine IoT data with other data to

enhance the use and value of data assets

Employ a Logical Data Lake/Logical Data Warehouse architecture to

eliminate the cost of storing information in multiple places, to

govern IoT data access, and to prevent IoT data from becoming

siloed.

21

Page 22: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Q&A

Page 23: Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

Thank you!

© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.

O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A

#DenodoDataFest