Expect the Unexpected - USENIX...Expect the Unexpected. Help > About John Arthorne...
21
Expect the Unexpected
Transcript of Expect the Unexpected - USENIX...Expect the Unexpected. Help > About John Arthorne...
Expect the Unexpected
Help > About
John Arthorne
● Developer/Manager/SRE at Shopify
Shopify
● Software for Commerce● 30 → 120 SREs● Average $1700 GMV / second
●●●
What We’ll Cover
high failure novelty rate
●
●
●
Transparent Response
●
●
●
Incident Simulation
●
●
●
Game Days
●
●
●
Turn Rusty Knobs
●
●
●
Automated Failure Tests
●
●
●
Software Change Rate
People Change Rate
High
High
Low
Low
Software Change Rate
People Change Rate
High
High
Low
Low
Incident TransparencyAutomated Tests
Game Days
Incident SimulationsRusty Knobs
novel failures?
Magic Recipe for Novel Failures
Training Exercise Formula
●●●●●
Summing Up
Thank You!github.com/jarthorn/lego-incident-response