Metrics - You are what you measure (DevOps Perth)
-
Upload
rob-crowley -
Category
Technology
-
view
140 -
download
4
Transcript of Metrics - You are what you measure (DevOps Perth)
![Page 1: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/1.jpg)
Metrics
You are what you measure
@robdcrowley
robdcrowley
![Page 2: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/2.jpg)
Empower you with new ideas to bring your team together!
Metrics. What are they? What to measure? How to visualise?
Tooling. Some secret sauce to make our lives easier
Metrics as a catalyst for cultural change
Goals for Session
![Page 3: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/3.jpg)
Metrics 101What metrics are and why we measure
![Page 4: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/4.jpg)
To informTo compareTo motivate To understandTo improveTo protectNot: Just because successful companies do!
Why do we measure
![Page 5: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/5.jpg)
A quantifiable measure of any component or process used to gauge the performance of your business
– Application– Business– People– Process
– System
What is a metric
![Page 6: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/6.jpg)
Basis of measurementAssumptionsLevel and usageDesired trendWhen to useWhen to stopHow to gameWarnings
Facets of a metric
![Page 7: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/7.jpg)
Red is good, Green is worthless
![Page 8: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/8.jpg)
Metrics 201What should you measure
![Page 9: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/9.jpg)
“Lack of direction, not lack of time, is the problem. We all have twenty-four hour days.”
- Zig Zigler
![Page 10: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/10.jpg)
Pivot: Base conversations on metrics instead of non actionable opinions
Use stakeholder input as the basis for your initial set of metrics
– “What do you want out of this?”– “How quickly do you want this?”
Metrics can help guide us
![Page 11: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/11.jpg)
Awesome book, poor title
Core Principles
– Minimum Viable Product (MVP)– Continuous Deployment– Split Testing (aka A/B Testing)– Actionable Metrics– Pivot– Innovation Accounting– Build-Measure-Learn Loop
![Page 12: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/12.jpg)
![Page 13: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/13.jpg)
Acquisition – finding new usersActivation – getting users to give your product a tryRetention – making sure those users stick aroundReferral – have your loyal users invite othersRevenue – making some money from all this
Note: These steps are not strictly sequential
Pirate Funnel (AARRR!)
![Page 14: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/14.jpg)
Do not focus solely on metrics that focus on reducing errors
Focus on achieving excellence and positive outcomes will ensue
![Page 15: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/15.jpg)
DevOps State of the Union focuses on these four core metrics
– Deployment Frequency– Lead Time for Changes
– Mean Time to Recover (MTTR)– Failed Change Rate
Example Metrics
![Page 16: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/16.jpg)
Fact: We only realise business value from the work we do when it’s released into production
The bottleneck or constraint is the limiting factor on our ability to deliver more quickly
Strengthening any link other than the weakest is a waste of time and effort
Use metrics to identify current constraint
Theory of Constraints
![Page 17: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/17.jpg)
Metrics 301How to visualise and share metrics
![Page 18: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/18.jpg)
Display all metrics on a dashboard
– Business (i.e. Application dropouts)– Dev (i.e. Performance metrics)– Ops (i.e. Web Server CPU Usage)
Single URL for all data
Make it easy for colleagues to access dashboard
Visualisation
![Page 19: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/19.jpg)
Use a tool that can handle different kinds of metrics
A few standouts are:
– StatsD/Graphite/Grafana – Heka
– Logstash/Elastic Seach/Kibana
Tools
![Page 20: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/20.jpg)
StatsD WhisperCarbon
GraphiteDashboardApps
UDP
TCP
HTTP
Graphite and StatsD
![Page 21: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/21.jpg)
Metrics 401Potential pitfalls when using metrics
![Page 22: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/22.jpg)
Case Study: US Health Service
In the 1990s the US Health Service decided to make hospital mortality rates public. What better metric for hospital performance?
Observed Behaviour: Best way to improve rating was not to admit critically ill patients in the first place
You are what you measure
![Page 23: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/23.jpg)
“The actual company values, as opposed to the nice-sounding values, are shown by who
gets rewarded, promoted or let go”
- Patty McCord
![Page 24: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/24.jpg)
Case Study: Enron
Enron which went bankrupt from fraud and whose leaders went to jail, had the following company values:
Integrity, Communication, Respect, Excellence
Where these really the core values at Enron?
You are what you measure
![Page 25: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/25.jpg)
“In the absence of clarity around an objective, any measure will do”
- Bob Paladino
Too Many Metrics
![Page 26: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/26.jpg)
Depict rosiest picture possible but do not accurately reflect the key drivers of your business
– Inaccessible– Not auditable– Not actionable
Vanity Metrics
![Page 27: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/27.jpg)
Metrics based on output rather than outcomes
Aim to maximise outcomes and minimise output
– Total lines of code– Number of bugs fixed– Total hours worked– Lines of code per developer
Useless Metrics
![Page 28: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/28.jpg)
“We’re not doing this data analysis thingjust because. We’re gathering data points
because we’ve seen it work to greateffect in other organisations and industries.”
- Anonymous
Cargo Cult Metrics
![Page 29: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/29.jpg)
Metrics being used for evil rather than for good
Confronting this organisational anti-pattern can be a good first step
Information sharing is the best defense
Break down knowledge silos in the organisation
Weaponized Metrics
![Page 30: Metrics - You are what you measure (DevOps Perth)](https://reader034.fdocuments.net/reader034/viewer/2022051520/58ed88261a28ab723c8b459f/html5/thumbnails/30.jpg)
Questions