Integrating Elasticsearch for Real-Time Analytics with Kibana
14
description
Speaker: David Ostrovsky, Couchbase The Cross Datacenter Replication (XDCR) feature in Couchbase Server lets us synchronize data between different database clusters. One of the most interesting uses of XDCR is replicating data from Couchbase to ElasticSearch in near real-time. On its own, Couchbase is awesome; combining it with the search and analytics capabilities of ElasticSearch turns the awesomeness up to eleven! In this session, we’ll learn how to tie Couchbase and ElasticSearch together into a seamless data storage and analysis platform. We’ll examine the benefits that each brings to the table and how we can use the former for efficient storage and retrieval, and the latter for fast indexing, full-text searches, and geo-spatial querying. But mainly, we'll explore the most interesting use case of combining ElasticSearch with Couchbase: using Kibana to create dynamic, real-time data analytics dashboards on top of the data stored in Couchbase Server.
Transcript of Integrating Elasticsearch for Real-Time Analytics with Kibana
@DavidOstrovsky
Senior Architect @ SelaCo-author of:
“Pro Couchbase Server” – Apress Media 2014“Essentials of Developing Windows Store Apps Using C#”“Advanced Windows Store App Development Using C#”
Big data and NoSQL nerd
CodeHardBlog.azurewebsites.net
What’s up?
Why Couchbase and ElasticSearch?How do I connect the two?What can I do with this?Does it scale?
Why use Couchbase Server?
If you don’t know this, you might be in the wrong room.
Why use ElasticSearch?
Fast full-text searchesSupports over a hundred languages
Geo-querying, scoring, facetingEasy to deploy, use and scaleHuge number of plug-insHigh buzzword count
Why these two in particular?
Both use JSON documentsComplimenting strengths:
ElasticSearch for flexible search, geolocation, analytics tools, and moreCouchbase for efficient data storage and retrieval, extremely fast response times
Couchbase uses XDCR to push data to ElasticSearchLower latency than polling, e.g. with River
Demo
Demo time!
Common use cases
Ad-hoc querying an existing Couchbase databaseGeospatial searches on dataReal-time analytics of Couchbase data
Demo
More demo time!
Putting it all together
A typical Couchbase-ElasticSearch deployment uses a 2:1 or 3:1 ratio of nodes
ElasticSearch only stores indexed dataSource data stored in CouchbaseAll CRUD operations go to Couchbase
Kibana is a dashboard for real-time visualization of streaming dataTalks directly to ElasticSearchDrag-and-drop dashboard construction
No coding required
Live analytics with Kibana
CouchbaseElasticSearch KibanaServer ClientCouchbase
Couchbase
Demo
So many demos!
Summary
Connecting Couchbase to ElasticSearch and Kibana is awesome!Lets developers and analysts focus on what each does bestProvides a resilient, scalable data storage and BI solution
Questions
