VISUALISING REAL TIME TRAFFIC DATA USING ELASTICSEARCH AND ... · VISUALISING REAL TIME TRAFFIC...
Transcript of VISUALISING REAL TIME TRAFFIC DATA USING ELASTICSEARCH AND ... · VISUALISING REAL TIME TRAFFIC...
VISUALISING REAL TIME TRAFFIC DATA USING ELASTICSEARCH AND C3JS
@jettroCoenradie Trifork Amsterdam
Case Study ANWB (Royal Dutch Automobile Association)
FACT SHEET
Jettro Coenradie Software engineer @ Triforkspecialised in search
Twitter @jettroCoenradie@gridshore
Gihub https://github.com/jettro
Linkedin https://www.linkedin.com/in/jettro
Blogs http://www.gridshore.nlhttp://blog.trifork.com/author/jettro/
WHAT IS ANWB?• Dutch Automobile Driver Assistance
• Sister from:
FDM (Danmark)
ADAC (Germany)
AA (England)
WHAT IS ELASTICSEARCH
• Distributed / Scalable search
• Structured and full-text
• Data analytics
• Log analysis
FLOW OF THE PROJECT
• Get to know the data: Logstash / Kibana
• Start improving data quality
• Present data using our own charts
TECHNICAL OVERVIEW
Data view
Data integration
Data Store
Tomcat - Spring mvc - c3js
Spring Integrationxml / csv
elasticsearch
SCHEMA-LESS
• There is always a schema
• The schema can be dynamic
• Often you want to be specific
Dates / Numbers / Geo locations
Dynamic schema
DocDocDoc
Set of documents
Condition
Bucket Bucket Bucket Bucket
Term: red, blue, green, yellowRange: 0-10, 10-20, 20-30, 30-40
GEO LOCATIONS
Two types of locations
• Using latitude and longitude
• Using geohash (creating a grid)
PERCOLATOR
Geo polygon filter
Zuid-WestNoord-WestNoord-OostZuid
{ location: [
3.5123, 46.3412 ]}
Zuid-West