Post on 19-Mar-2017
ED BUKOSKI • SENIOR SOFTWARE ENGINEER • NETFLIX • @EBUKOSKI
Culture, Speed, Innovation
DevTools At Netflix
Just text by itself, for impact.
Production BitBucket (i-a1044e67)
October 15, 2015
WorkCoffee & Netflix WTF
October 15, 2015
Production BitBucket (i-a1044e67)
Technical
Culture
Scale
DevTools
S C A L E & C U LT U R E
D E V TO O L S AT N E T F L I X
S U P P O R T
Agenda
Q & A
T E A M S
I N N O VAT I O N & C H A N G E
Scale & Culture
0
3.5
7
10.5
14
2008 2010 2012 2014 2016
Streaming Hours
Netflix scale
13 billion streaming hours/month
83 million subscribers
35% peak downstream US internet traffic
0
125
250
375
500
2008 2010 2012 2014 2016
Build slaves
Netflix scale
473 build slaves
1,450 developers
7,700 builds per day
Netflix Culture
Freedom and responsibility
Context, not control
Highly aligned, loosely coupled
Operating Context for DevTools
High Availability
High Resiliency
High Demand
Netflix Scale Netflix Culture
Freedom & Responsibility
Context, Not Control
Highly Aligned, Loosely Coupled
DevTools at Netflix
Change & innovationSupportTeams
Change & innovationsSupportTeams
Teams
DEVELOPERS
Context over control
DEVTOOLS
HALC
Marketing
Microservices
F&R Choose own tech stack
Choose their own tools
#NEXTBIGTHING
The obvious choice
Friction free Supported Common
Code Build CI Scale
Resiliency
Instrumentation
more…
Deploy
Change & innovationsSupportTeams
Team support
Walk-insChat Email
Team support
What we do
Z z
What we don’t doFirst responder
1:1 sessionsHands-on
Feedback into training
TicketsAutomation
Formal SLAs24/7 support
Operational support
Automated metrics & monitoring
Automated backups
Automated alerts
Automated upgrades
Operational support
Bitbucket metrics • System • Load • Database • JMX
~100 metrics total
Realtime
Alert enabled
All automated
Change & innovationSupportTeams
Change
Replace obsolete technology
Cost reduction Regulatory change(SOX)
Change
OWNER _
DEVTOOLS
ON TRACK
IMPACTED
Change campaign tools
Identifies impacted assets
Locates asset owners
Notifies owners
Track progress
Just text by itself, for impact.
Innovation
Reward
Risk
DevTools
Innovation
Reward
Risk
DevTools
Pioneers
Settlers
Legacy
Innovation
Pioneers
DevToolsDevTools
Critical mass
Partner
Freedom and responsibility enables teams to pioneer
Freedom to pick tech stack
Responsibility to tool and maintain it
Pioneers
DevToolsDevTools
Critical mass
Partner
Reality
Velocity mismatch
Focus disconnect
Tech incompatibilities
Team constraints
DevTools at Netflix - Takeaways
Blend DevTools with culture
Build paved roadsPromote best practices, tools, and culture
Create tools to facilitate change
Embrace your customers/Empower innovators
Automate operational support
Alignment and communication are critical
Thank you!
ED BUKOSKI • SENIOR SOFTWARE ENGINEER • NETFLIX • @EBUKOSKI