WEBINAR

How To Get Monitoring Right For Streaming And Fast Data Systems Built With Spark,

Mesos, Akka, Cassandra and KafkaPaul Jasek, Senior Director of Global Solution Architects

Agenda1. Fast Data & Streaming Applications 2. The Challenges of Monitoring Fast Data Applications3. What To Look For In a Fast Data Application 4. Intelligent End-To-End Monitoring from Lightbend5. Live Demo 6. Questions

reactivemanifesto.org

Reactive Underpinnings: Fast Data and streaming applications often incorporate, or are based on, Reactive principles

• Real-timepersonalization

• Real-timedecision-making

• IoTdataprocessing

• Legacybatchprocessing

modernization

Growing Number Of Use Cases Across Industries

• Serveexistingcustomersbetterandreducechurn

• Attractnewonesanddrivegrowth

• Launchnewproductsmoreeasily

• Enternewmarketsmorequickly.

• Rapidly Evolving Ecosystem • Understanding the Data Pipeline • Dynamic Architectures • Intricately Interconnected • Distributed And Clustered

The Challenges of Monitoring Fast Data Applications

Apache Spark, As An Illustrative ExampleThe Challenges of Monitoring Fast Data Applications

Concern QuestionsToAsk

DataHealth(foragivenapplication)

• Throughput:isdataprocessingoccurringattheexpectedrate?• Latency:isdataprocessingoccurringwithintheexpectedtimeframe?• Error/quality:arethereproblemswiththedatabeingproduced?• Inputdata:areinputdatastreamsflowingintoSparkbehavingnormally?Forinstance,whatarethe

throughputratesforKafkatopicsfeedingintotheSparkjob?

DependencyHealth • Arethesystemsfeedinginputintothestormjob(suchasKafka)healthy?• Arethesystemsthattheapplicationisdependenton,suchasMemcacheorotherAPIendpoints,

healthy?

ServiceHealth • IstheSparkmasteroperatingnormally?Ifnot,engineeringwillbeunabletore-balanceworkloadsorrestartjobs.

ApplicationHealth • AretheapplicationKPIswithinnormaloperatingparameters?

TopologyHealth • ArethereresourcesassignedtothegivenSparktopology?• •AretheSparktasksandexecutorswell-distributedamongsttheSparkcluster?• •Aretheperformancecounters(emitted,failed,latency,etc.)forthegivenSparktopologynormal?

NodeSystemHealth • Arethekeysystemmetrics(load,CPU,memory,net-i/o,disk-i/o,diskfree)operatingnormally?

Can traditional monitoring tools help?

Why traditional monitoring tools won’t help you

• Builttomonitormonolithicapplications

• Canonlybeusedtoextractmetricsandtraceinformationbasedonasynchronousflow

• Notbuiltforasynchronousflows(i.e.inFastDataandstreamingapplications)

• Cannoteasilyhandlestreamingsystemsrunningondistributedclusters

• Deep Telemetry

• Domain Expertise

• Automated Discovery

• Real-Time Topology Visualization

• Intelligent, Rapid Troubleshooting

What users need to effectively monitor Fast Data and streaming applications

• Lightbend Monitoring takes a modern approach to instrumenting and

visualizing distributed streaming systems

• Helps users not just in production but also in development (so they can

build their applications right from Day 1)

• Shows the end-to-end status of applications, data frameworks, and the

associated infrastructure in a single view.

Intelligent, End-To-End Monitoring

• Deep Telemetry

• Domain Expertise

• Intelligent Anomaly

Detection

• Fine-Grained

Visibility, with Drill-

Down Capabilities

Data-Science Driven Anomaly Detection

• Automated Topology

Discovery

• Automatic Metric

Collection

• Real-Time Topology

Visualization

Automated Discovery, Configuration & Topology Visualization

• Single Pane of Glass

Visibility

• Rapid Root Cause

Analysis

• Reduced Mean-Time-

To-Repair (MTTR)

Intelligent, Rapid Troubleshooting

• Dramatically reduce the time and cost to identify and remediate issues across application life-cycle.

• Create happier, more satisfied customers – and lower churn

• Lower HW/infrastructure costs and reduce concerns about chargebacks & SLA penalties

• Deliver rapid time to value because everything you need for monitoring is packaged into an easy-to-use solution

Benefits for your business

On to the demo…

Upgrade your grey matter!Get the free O’Reilly book by Dr. Dean Wampler,

VP of Fast Data Engineering at Lightbend

bit.ly/lightbend-fast-data

End-To-End Monitoring For Your Fast Data And Streaming Applications From Lightbend

SET UP A 20-MIN DEMO