Post on 13-Jul-2015
Zero to Cloud with @NetflixOSS
Justin Ryan (@quidryan)
Who We Are• Justin Ryan • Michael Tripoli • Various other Netflix Employees in the Audience
#netflixoss
#netflixoss
Netflix is the world's leading Internet television network with over 48 million members in more than 40 countries enjoying more than one billion hours of TV shows and movies per month, including original series. For a low monthly price, Netflix members can watch as much as they want, anytime, anywhere, on nearly any Internet-connected screen. Members can play, pause and resume watching, all without commercials or commitments. (Credit: ir.netflix.com)
Tutorial Format• 30 Minutes of Lecture • 30 Minutes on Console • 30 Minutes on Jumphost • 30 Minutes deploying Asgard • 30 Minutes Edda and Eureka • 30 Minutes Karyon
#netflixoss
Tutorial Purpose• Help adoption of @NetflixOSS components • Answer questions along the way • Less trail blazing for us
#netflixoss
Assumptions• Stay Local (using us-west-2) • Non-destructive operations • Follow instructions closely • Security through obscurity • Not a “developing for the cloud” tutorial • This will cost money
#netflixoss
#netflixoss
Step 1 - Sign Up for AWS
Network
#netflixoss
Jumphost
Laptop
Asgard InstanceInstanceInstances
InstanceInstanceInstancesElastic Load
Balancer
Elastic Load
Balancer
SSH
HTTP
#netflixoss
Step 2 - AWS Console
#netflixoss
Step 3 - Key Pair
#netflixoss
Step 4 - Create Jumphost
#netflixoss
Step 5 - Create Role
#netflixoss
Step 6 - Create User
Cloud Deployment Model
#netflixoss
Auto Scaling Group
Elastic Load
Balancer
Launch Configuration
Security Group
Amazon Machine Image
Configuration
Runtime
InstanceInstanceInstances
#netflixoss
Step 7 - Security Group
Immutable Infrastructure• Simplifies Operations • Baking with Aminator
#netflixoss
Foundation
Base AMI
Application
#netflixoss
Step 8 - Foundation AMI
#netflixoss
Step 9 - Setup Jumphost
#netflixoss
Step 10 - Credentials
Building• Bake a system package (RPM, DEB) • Use ospackage plugin to generate
#netflixoss
Foundation
Base AMI
Application
WAR DEBJava
Python
#netflixoss
Step 11 - Base AMI
netflix.github.io
#netflixoss
@netflixoss
#netflixoss
Who• Platform Infrastructure • Cloud Database Engineering • Cloud Solutions Engineering Tools • Cloud and Platform Engineering • (Anyone with code that’s not a competitive advantage)
#netflixoss
What• Infrastructure • Caching • Database • Cloud • Building • Deployment
!
#netflixoss
• Testing • Monitoring • Networking • Robustness • Security • Configuration
Not• Streaming • Encoding • Merchandising • Movie metadata • Recommendations • 10’ UI
#netflixoss
Open source infrastructure• Github • Cloudbees for Builds • http://techblog.netflix.com/ • http://twitter.com/netflixoss • Apache 2.0 license (when possible) • Meetups - Season 2 going on right now!
#netflixoss
#netflixoss
Step 12 - Bake Asgard
Islands of Applications
#netflixoss
Search Auto-complete Ratings
Signup API Streaming Starts
Islands of Applications
#netflixoss
Search Auto-complete Ratings
Signup API Streaming Starts
Application Application Application
Application Application Application
Inventing the Application• Problem
- Application is not an Amazon concept • Solution
- Create an Application domain in SimpleDB - Enforce naming conventions on Amazon objects
#netflixoss
Red/Black Deploy
#netflixoss
Elastic Load
Balancer
InstanceInstanceInstances
api-usprod-v007
InstanceInstanceInstances
api-usprod-v008
Fast Rollback
#netflixoss
Elastic Load
Balancer
InstanceInstanceInstances
api-usprod-v007
InstanceInstanceInstances
api-usprod-v008
Inventing the Cluster• Problem
- Two ASGs on purpose - Different names • Solution
- Maintain same prefix - Append version - Derive Cluster from ASG names
#netflixoss
Asgard vs Console• Asgard
- Provide Application and Cluster - Simplifies object construction - Introduce workflow - Tightly couple with internal systems • Console
- Access to all objects
#netflixoss
#netflixoss
Step 13 - Asgard/Asgard
#netflixoss
Step 14 - Deploy Edda
Mid-tier Calling
#netflixoss
EurekaData Plane
Control Plane
InstanceInstanceApp A InstanceInstanceApp B
#netflixoss
Step 14 - Deploy Eureka
#netflixoss
Extra - Karyon
Gift Cards
#netflixoss
Thanks, Amazon!