Dependable Cloud Architecture - SWOCC Edition
-
Upload
michael-wood -
Category
Technology
-
view
696 -
download
1
description
Transcript of Dependable Cloud Architecture - SWOCC Edition
![Page 2: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/2.jpg)
“Failure is alwaysan option.”
Image: Discovery Channel, Fair Use
![Page 3: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/3.jpg)
Protection From:
What are we looking for?
Check out: http://bit.ly/wazbizcontImages: Office ClipArt & Godzilla Releasing Corp (Fair Use)
Hardware Failure Data Corruption Network Failure Loss of Facilities
![Page 4: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/4.jpg)
Image: FOX, Fair Use
Human Error
![Page 5: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/5.jpg)
What we’re trying to achieve
1. Monitoring2. Resilient Solutions
Image: Cohdra
![Page 6: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/6.jpg)
Image: Office ClipArt
Cost vs Risk
99.999% $1, … ,000.00
To get more 9’s here add more 0’s here.
![Page 7: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/7.jpg)
Image: NASA
Monitoring
![Page 8: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/8.jpg)
Functional Transparency
Image: Office ClipArt
Logging Messages
Hardware Health
Dependent Services Health
![Page 9: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/9.jpg)
Telemetry
![Page 10: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/10.jpg)
Image: NASA
Analyze your Data
![Page 11: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/11.jpg)
ResilienceImage: Office ClipArt
![Page 12: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/12.jpg)
Remember: Failure is always an option.
Common Points of Failure• Machine\application crashes• Throttling (exceeding capacity)• Connectivity\Network• External service dependencies
Focus less on the uptime of hardware and more about how the solution handles it WHEN
something fails!
![Page 13: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/13.jpg)
Try/catch != Resilient
private void createFile() {
string fileName = @"c:\workingDirectory\someFileName.txt";
try {
File.Create(fileName);}catch (DirectoryNotFoundException ex)
{Trace.WriteLine(String.Format("Unable to create {0}. {1}",
fileName, ex));
throw; } } }
![Page 14: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/14.jpg)
Image: Michael Wood
Decompose your system…
![Page 15: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/15.jpg)
Capacity BufferingContent Delivery Networks (CDN’s)
Distributed Application Cache
Local Content Cache
Enables recovery during outages or
spikes in load
Image: jepler
![Page 16: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/16.jpg)
Always carry a spare75% Capacity, half of our load 75% Capacity, half of our load
50% more capacity then needed• Can absorb of temporary spikes• Time to react if need to add capacity
100% of load, 150% Capacity0% Capacity, redirect all load
Over allocated, but still functioning• Degrade, but don’t fail
SYSTEM FAILURE!!!
Image: Kevin Rosseel
![Page 17: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/17.jpg)
Request Buffering
Image: Joe Shlabotnik
QueuesRetry PoliciesAsync Workloads
![Page 18: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/18.jpg)
Dept. of Redundancy Dept.
Have a backup, somewhere elseMore than one? Cost to benefit Ratio?
Ready StateHot = full capacityWarm = scaled down, but ready to growCold = mothballed, starts from zero
Image: Mr. White
![Page 19: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/19.jpg)
Redundancy - Its about probability
95% uptime 95% uptime 95% uptime 95% uptime
1 box : 5% downtime or 438hrs per year
2 boxes : 5/100 * 5/100 = 25/10,000 = 0.25% downtime or 22hrs per year
4 boxes : 5/100 * 5/100 * 5/100 * 5/100 = 625/100,000,0000.000625% downtime or 3.285 MINUTES per year
(that’s 18 ½ days!)
![Page 20: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/20.jpg)
Total Outage duration =
Time to Detect+ Time to Diagnose+ Time to Decide+ Time to ActImage: Office ClipArt
![Page 21: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/21.jpg)
Dynamic Addressing & Configuration
![Page 22: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/22.jpg)
What about your data?
Image: barrymieny
![Page 23: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/23.jpg)
Availability via Degradation
Image: Michael Wood
![Page 24: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/24.jpg)
Images: Gizmodo
Virtualization and Automation
![Page 25: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/25.jpg)
Images: Orion Pictures owns Terminator Franchise
![Page 26: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/26.jpg)
The “HI” Point
Check out: http://bit.ly/wazinternalsImages: Office Clip Art
![Page 27: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/27.jpg)
Image: NASA
![Page 28: Dependable Cloud Architecture - SWOCC Edition](https://reader034.fdocuments.net/reader034/viewer/2022051816/5464ab94af795950608b5804/html5/thumbnails/28.jpg)
“Don't be too proud of this technological terror you've constructed…”
ADMIT:• Your Solution WILL fail at some point• You can learn from others just as
well as yourself
DO:• Root cause analysis• Read other root cause analysis• Plan for failure
DON’T:• Get cocky• Stick your head in the sand
Images: LucasFilm, Fair Use