kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Real-time IoT data analytics and visualization with Kaa, Apache Cassandra,

and Apache Zeppelin

Agenda

Why Kaa? Why Cassandra? Why Zeppelin? Reference architecture overview Hands-on

Sandbox setup Intel Edison application code walkthrough Cassandra appenders configuration Zeppelin configuration Live demo

Q&A

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

About Kaa

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Fully-featured IoT middleware platform 100% open-source (Apache license 2.0) Deploy anywhere: on-prem, in the cloud, in hybrid mode Horizontally, elastically scalable, and fault tolerant Guaranteed data delivery and reliable local storage Built-in transport security Efficient data serialization Rapid application development using C / C++ / Java / Objective C SDKs 10 Kb RAM footprint (with C SDK) Integration with popular device platforms

Why Kaa?

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Fault tolerant Performant Horizontally scalable Easy deployment Integration with popular analytics platforms

Why Cassandra?

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Multiple language backend Apache Spark integration Data visualization Dynamic forms Collaboration & Publish Open-source project

Why Zeppelin?

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Problem description

Zone 1 Zone 2

Zone 3 Zone 4

Zone 5 Zone 6

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Reference architecture

Solar panels

StructuredData

Solar panels

Raw data

Intel Edison

Kaa SDKClient application

Intel Edison

Kaa SDKClient application Kaa cluster/sandbox

Kaa node

Cassandra appender

Cassandra

Kaa node

Cassandra appender

Zeppelin instance/sandbox

Zeppelin dashboard

Zeppelin dashboard

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Development environment setup

Sample project repository: https://github.com/kaaproject/kaa-zeppelin-sample

Kaa Sandbox: http://www.kaaproject.org/download-kaa

Intel Edison:https://docs.kaaproject.org/display/KAA/Intel+Edison

Zeppelin:http://zeppelin.incubator.apache.org

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

https://github.com/kaaproject/kaa-zeppelin-samplehttps://github.com/kaaproject/kaa-zeppelin-samplehttps://github.com/kaaproject/kaa-zeppelin-samplehttps://github.com/kaaproject/kaa-zeppelin-samplehttps://github.com/kaaproject/kaa-zeppelin-samplehttp://www.kaaproject.org/download-kaa/http://www.kaaproject.org/download-kaa/https://docs.kaaproject.org/display/KAA/Intel+Edisonhttps://docs.kaaproject.org/display/KAA/Intel+Edisonhttp://zeppelin.incubator.apache.org/http://zeppelin.incubator.apache.org/

Data modeling

Single zone per row Single panel per row with zone partitions

Kaa data collection schema:{ "type":"record", "name":"PowerReport", "namespace":"org.kaaproject.kaa.sample", "fields":[ {"name":"timestamp", "type":"long"}, {"name":"zoneId", "type":{"type":"string", "avro.java.string":"String"}}, {"name":"panelId", "type":{"type":"string", "avro.java.string":"String"}}, {"name":"power", "type":"double"}]}

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Single zone per row

Partition Key: Zone ID Clustering Key: Timestamp, Panel ID

Zone ID

Timestamp, Panel ID

Power

Timestamp 2,Panel ID

Power

Timestamp N,Panel ID

Power...

Zeppelin query:%cassandra select zone_id, panel_id, power as "power, MW", timeFormat(timestamp) as time from logs.zone_per_row where zone_id={{zone_id='Sierra'}} and timestamp > logs.timeDelta(toUnixTimestamp(now()), 60000) ORDER BY timestamp ASC;

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Single panel per row with zone partitions

Partition Key: Zone ID, Panel ID Clustering Key: Timestamp

Zone ID,Panel ID

Timestamp N

Power

Timestamp N-1

Power

Timestamp 1

Power...

kaaproject.

org 2016 CyberVision, Inc. All rights

reserved.

Zeppelin query:%cassandra select zone_id, panel_id, power as "power, MW", timeFormat(timestamp) as time from logs.panel_per_row where zone_id={{zone_id='Sierra'}} and panel_id={{panel_id='Panel_0'}} and timestamp > logs.tmBefore(toUnixTimestamp(now()), 60000) ORDER BY timestamp ASC;

kaaproject.orgcybervisiontech.com

THANK YOU FOR YOUR ATTENTIONQUESTIONS?

Fault-tolerance and horizontal scalability

kaaproject.org

Operations servers

2016 CyberVision, Inc. All rights reserved.

Bootstrap servers

Zookeeper quorum

Endpoints (Kaa SDK embedded)

Control servers

standby

active