Model monitoring & alerting
-
Upload
robertzaremba -
Category
Data & Analytics
-
view
294 -
download
10
Transcript of Model monitoring & alerting
![Page 1: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/1.jpg)
Model monitoring & alerting
Robert ZarembadotGo
2015-11-09, Paris
![Page 2: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/2.jpg)
● Spotting the biggest barriers to growth is the most important company challenge. This challenge goes around all company teams:
– sales (What are the conversion rates),
– dev operations (Does the infrastructure perform well enough?),
– product (Which features bring the most value?)
– developers (Do we have errors?).
● You can't answer this questions without monitoring.
![Page 3: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/3.jpg)
Model monitoring
● Monitoring goes through different layers: from user behaviour, to system operations. Here I'm presenting monitoring of the system model (business) changes based on InfluxDB and alerta.io
● Model monitoring reflects the business aspects of the application.
![Page 4: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/4.jpg)
What is business monitoring?
● Measurement used to track and assess some specific business process.
● Each report / metrics related task you receive.
![Page 5: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/5.jpg)
What is business monitoring?
● Measurement used to track and assess some specific business process.
● Each report / metrics related task you receive.
● Everything what beats you just after bugs.● Tracking where and how $$$ are coming.
![Page 6: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/6.jpg)
What is business monitoring?
● Reports● Analysis● Charts● Alerts
![Page 7: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/7.jpg)
Business monitoring
● Big business: hundreds of companies raised around it
● http://alternativeto.net/software/google-analytics/
![Page 8: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/8.jpg)
Kissmetrics
![Page 9: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/9.jpg)
Amplitude
![Page 10: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/10.jpg)
Google Analytics
![Page 11: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/11.jpg)
Do we need anything else?
![Page 12: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/12.jpg)
Do we need anything else?
YES
![Page 13: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/13.jpg)
Why we need anything else?
● Because storing metrics is cool.● Because you want to have your own
metrics.● Because you like monitoring.● Because we have almost all blocks to build
our custom business monitoring service.● Other services are expensive.
![Page 14: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/14.jpg)
What you need?
● Metrics DB● Event collector service● Alerting service
![Page 15: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/15.jpg)
Metrics DB - InfluxDB
● SQL like query language.● Can store anything.● The upcoming 0.9.5 release will improve
storage performance.● Is trendy and we like trendy things!● Written in GO!
![Page 16: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/16.jpg)
Alerting: alerta.io
● A tool used to consolidate and de-duplicate alerts from multiple sources for quick ‘at-a-glance’ visualisation.
● Basically you fire events to alerta and it makes alerting management for you (alerting, deduplication, grouping, visualization, …).
● Can be integrated with other Open Source monitoring systems: Nagios, Zabbix, Sensu ...
![Page 17: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/17.jpg)
Event service: alerta-collector
● A service which reads InfluxDB queries and fires events to alerta.io
● Plugin based: extensible to any other source: RDMS, NoSQL, SaaS, Graphite, Nagios, ...InfluxDB is just the first plugin.
![Page 18: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/18.jpg)
Architecture
InfluxDBUI
(Grafana)
Search for interesting events
alerta-collector
alerta.io
Nagios-nrpeNagios-nrpe
Postgresql
RabbitMQ
Define events
Alert (email, sms, ...)
![Page 19: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/19.jpg)
Defining events
● Now: yaml file● In the future: yaml file + API + UI
![Page 20: Model monitoring & alerting](https://reader034.fdocuments.net/reader034/viewer/2022042515/58a91d991a28ab6f508b4d8d/html5/thumbnails/20.jpg)
Roadmap
● Defining a UI to list event definitions● More plugins
● https://github.com/robert-zaremba/alerta-collector