1 COREQUISITE REMEDIATION Too many students start college in remediation. 50% 2 Remediation.
Remediation patterns
-
Upload
jez-humble -
Category
Technology
-
view
3.449 -
download
1
description
Transcript of Remediation patterns
![Page 1: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/1.jpg)
© 2012 ThoughtWorks, Inc.
Remediation PatternsOctober 12, 2011, GOTO Amsterdam
Jez Humble, ThoughtWorks Studios@jezhumble #continuousdeliveryhttp://continuousdelivery.com/
![Page 2: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/2.jpg)
ITIL: “Recovery to a known state a!er a failed Change or Release.”
Recovery: “Returning a Con"guration Item or an IT Service to a working state.”
Jez: “Fixing shit when it breaks”
remediation
![Page 3: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/3.jpg)
prevention
patterns for low-risk release
patterns for incremental delivery
strategies for remediation
![Page 4: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/4.jpg)
1oz of prevention
Diagram invented by Brian Marick
![Page 5: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/5.jpg)
deployment pipeline
Delivery team Version control Build & unit tests
Automated acceptance tests
User acceptance tests
Release
Check in
Feedback
Trigger
Check in
Feedback
Trigger
Trigger
Check inTrigger
Trigger
ApprovalApproval
Feedback
Feedback
FeedbackFeedback
![Page 6: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/6.jpg)
testing on production environments
creating maintainable acceptance tests
testing cross-functional requirements
the hard bits
![Page 7: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/7.jpg)
optimize for resilience
automate provisioning and deployment
devs, testers and ops collaborate throughout
reducing release risk
![Page 8: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/8.jpg)
optimize for TTR
Thanks to John Allspaw: http://slideshare.net/jallspaw/
MTBF
http://www.flickr.com/photos/chuecy/2629242132/
MTTR
![Page 9: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/9.jpg)
infrastructure as code
Destroy works of art
![Page 10: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/10.jpg)
low risk releases are incremental
STATIC CONTENT
/static/1.1
/static/1.0
DEPENDENT SERVICE
1.0 1.1
Abstraction layer Abstraction layer
APPLICATION
Database
Router /Load balancer
Interwebs
![Page 11: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/11.jpg)
canary releasing
Diagram by Martin Fowler
![Page 12: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/12.jpg)
canary releasing
Diagram by Martin Fowler
![Page 13: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/13.jpg)
reduce risk of release
A/B testing
performance testing
canary releasing
![Page 14: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/14.jpg)
immune system
what if someone replaced your “buy” button with spacer.gif?
T cells http://www.flickr.com/photos/gehealthcare/3326186490/
![Page 15: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/15.jpg)
Business metrics - revenue, # orders, # users
Ops metrics - changes, incidents, TTD, TTR, TBF
Technical metrics - TPS, response time, hits
monitoring
http://www.flickr.com/photos/wwarby/3296379139/
![Page 16: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/16.jpg)
analyzing root causes
collaboration
data
the hard bits
![Page 17: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/17.jpg)
incremental delivery
John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr
![Page 18: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/18.jpg)
develop on mainline
feature toggles and branch by abstraction
dark launching
incremental delivery
![Page 19: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/19.jpg)
feature toggles
[featureToggles]wobblyFoobars: trueflightyForkHandles: false
Config File
<toggle name=wobblyFoobars> ... various UI elements</toggle>
some.jsp
forkHandle = (featureConfig.isOn(‘flightlyForkHandles)) ? new FlightyForkHander(aCandle) : new ForkHandler(aCandle)
other.java
Stolen from Martin Fowler http://martinfowler.com/bliki/FeatureToggle.html
![Page 20: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/20.jpg)
branch by abstraction
Component A
Component B
Seam
Component A
![Page 21: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/21.jpg)
branch by abstraction
Component AComponent A
Abstraction layer
Component B Component B’
![Page 22: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/22.jpg)
dark launching
Diagram by Martin Fowler
![Page 23: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/23.jpg)
dark launching
Diagram by Martin Fowler
![Page 24: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/24.jpg)
How long would it take you to release a change to a single line of code?
Ops metrics - changes, incidents, TTD, TTR, TBF
If your data center blew up, how long would you take to restore service?
measuring e#ectiveness
![Page 25: Remediation patterns](https://reader036.fdocuments.net/reader036/viewer/2022062307/554bb186b4c905ae618b5930/html5/thumbnails/25.jpg)
http://thoughtworks-studios.com/
[email protected]://continuousdelivery.com/
@jezhumble #continuousdelivery
© 2012 ThoughtWorks, Inc.