Removing the Human SPOF

Justin Huff

jhuff@google.com

Tim Wilfong

twilfong@google.com

What is Picnik?

Early Picnik Operations:A One-Man Show

(mostly)

Don't Try This at Home, Kids

Humans are part of the system!

No single person should ever control exclusive knowledge of, or access to, a critical system.

Complete failure is not the only kind of component fault

When do you expand the team?

Sooner is Better

Acceptable Risk

Avoid adding a new SPOF with a new hire

Hiring

At ur interviewz testing my skillz

It's time to grow up!

Oh yeah:

Automate, Automate,Automate!

Good Documentation is Key

Things Change

Be Flexible

Next step: Google scale!

Questions?

Thank you!

Images used under Creative Commons Attribution 2.0 Generic license:

6/15/10

Image URLhttp://www.flickr.com/photos/helico/404640681/http://www.flickr.com/photos/zappowbang/1548058686/http://www.flickr.com/photos/fncll/135465558/http://www.flickr.com/photos/cursedthing/2221808168/http://www.flickr.com/photos/moria/3076393/http://www.flickr.com/photos/deerwooduk/579761138/

Slide #479

101314