Performance Metrics for your Delivery Pipeline - JAX London - October 14, 2014
Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title:...
Transcript of Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title:...
![Page 1: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/1.jpg)
Production Microservices
Chaos from Order
![Page 2: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/2.jpg)
![Page 3: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/3.jpg)
A subtitle…
![Page 4: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/4.jpg)
Production …
![Page 5: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/5.jpg)
Hates YOU
![Page 6: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/6.jpg)
Essential Instabilitythrough Change
![Page 7: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/7.jpg)
What do we aim for?
![Page 8: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/8.jpg)
Fragility?
![Page 9: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/9.jpg)
Robust/Resilient?
![Page 10: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/10.jpg)
Antifragile
![Page 11: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/11.jpg)
Antifragile Systems
![Page 12: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/12.jpg)
What is the core challenge?
![Page 13: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/13.jpg)
Availability
![Page 14: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/14.jpg)
Aggregate Availability
![Page 15: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/15.jpg)
Time-Based Availability
![Page 16: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/16.jpg)
What’s your risk tolerance?
![Page 17: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/17.jpg)
Consider…
![Page 18: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/18.jpg)
Target Availability Failure Types
Cost
![Page 19: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/19.jpg)
Stressors-First
![Page 20: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/20.jpg)
From Stressors to Objectives
![Page 21: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/21.jpg)
From Objectives to Indicators
![Page 22: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/22.jpg)
From Indicators to Agreements
![Page 23: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/23.jpg)
Types of Failure to Consider
![Page 24: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/24.jpg)
Infra
![Page 25: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/25.jpg)
Hardware
Infra
![Page 26: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/26.jpg)
Latency
Infra
![Page 27: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/27.jpg)
Network Loss
Infra
![Page 28: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/28.jpg)
Resource Exhaustion
Infra
![Page 29: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/29.jpg)
Inter-Service Comms
Infra
![Page 30: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/30.jpg)
Platform
![Page 31: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/31.jpg)
State Transmission
Platform
![Page 32: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/32.jpg)
Race Conditions
Platform
![Page 33: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/33.jpg)
Application
![Page 34: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/34.jpg)
Functional
Application
![Page 35: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/35.jpg)
Dependencies Malfunction
Application
![Page 36: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/36.jpg)
People!
![Page 37: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/37.jpg)
Configuration
Application
![Page 38: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/38.jpg)
PEBKAC
People
![Page 39: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/39.jpg)
Black Swans
![Page 40: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/40.jpg)
Observability
![Page 41: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/41.jpg)
Indicators
![Page 42: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/42.jpg)
Actionable Information
![Page 43: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/43.jpg)
How to get this??
![Page 44: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/44.jpg)
How to get people to care about this?
![Page 45: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/45.jpg)
Game Days
![Page 46: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/46.jpg)
From Game Days to Chaos
![Page 47: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/47.jpg)
Why Chaos?
![Page 48: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/48.jpg)
Cynefin
![Page 49: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/49.jpg)
We don’t know… … what we don’t know
![Page 50: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/50.jpg)
First Rule of Chaos…
![Page 51: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/51.jpg)
Not just about pain
![Page 52: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/52.jpg)
No surprises
![Page 53: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/53.jpg)
Starts with a Hypothesis
![Page 54: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/54.jpg)
What do you want to Learn?
![Page 55: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/55.jpg)
To Probes
![Page 56: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/56.jpg)
To Actions
![Page 57: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/57.jpg)
Engage with all parties
![Page 58: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/58.jpg)
Build Experiments…
![Page 59: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/59.jpg)
Or thing to try…
![Page 60: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/60.jpg)
Manual and Automated
![Page 61: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/61.jpg)
Automated!
![Page 62: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/62.jpg)
Repeat is Crucial
![Page 63: Production Microservices - JAX London · 2018-06-20 · @russmiles @chaostoolkit. Title: ChaosToolkitTemplate Created Date: 20171009155508Z](https://reader036.fdocuments.net/reader036/viewer/2022081607/5edc8d37ad6a402d6667430e/html5/thumbnails/63.jpg)
@russmiles
http://chaostoolkit.org https://join.chaostoolkit.org
@chaostoolkit