Managing your cloud assets
Ryan Shuttleworth – Technical Evangelist @ryanAWS
Agenda
Setting the scene – a spectrum of system definition and control
A shrink wrapped approach – Elastic Beanstalk
Fine grained control – CloudFormation
Complementary uses – CloudFormation & Beanstalk working together
Your feedback is important
Tell us:
What’s good, what’s not
What you want to see at these events
What you want AWS to deliver for
you
Cloud assets are growing quickly
Defined architectural patterns More applications
More data More services
aws.amazon.com/architecture
Greater agility = faster moving environments
How do we maintain control over what is deployed in the cloud?
Two ends of the spectrum…
Controlled chaos
Meticulous management
Controlled chaos
Meticulous management
Prototype an app Deploy to an instance
Test and develop cycles Go to production
Deploy more instances Deploy RDS
Add load balancers Add autoscaling
Audit what you have?
Quickly becomes DevOps nightmare
Controlled chaos
Meticulous management
Prototype an app Deploy to an instance
Test and develop cycles Go to production
Deploy more instances Deploy RDS
Add load balancers Add autoscaling
Audit what you have?
Quickly becomes DevOps nightmare!
Prototype an app Plan infrastructure
Build standards Deploy environment
Manage through change control Formal processes
Formal tools Embed configuration in
chef/puppet
Tightly controlled, but lots of effort to achieve
Controlled chaos
Meticulous management
People Processes Controls
Tools
Prototype an app Deploy to an instance
Test and develop cycles Go to production
Deploy more instances Deploy RDS
Add load balancers Add autoscaling
Audit what you have?
Quickly becomes DevOps nightmare!
Prototype an app Plan infrastructure
Build standards Deploy environment
Manage through change control Formal processes
Formal tools Embed configuration in
chef/puppet
Tightly controlled, but lots of effort to achieve
High level, shrink
wrapped
Fine grained, declarative
control
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
Quickly deploy and manage apps in AWS…
Elastic Beanstalk
CloudFormation
Tools
…into a range of containers
Elastic Beanstalk
CloudFormation
Tools
.Net PHP
Java Python
…into a range of containers
Elastic Beanstalk
CloudFormation
Tools
http://aws.typepad.com
Keep updated on new containers:
Application package is deployed into Beanstalk
Elastic Beanstalk
CloudFormation
Tools
User Application
Application Service
HTTP Service
Language Interpreter
Operating System
Host
Which creates the container in EC2
Elastic Beanstalk
CloudFormation
Tools
Beanstalk takes care of the environment…
Elastic Beanstalk
CloudFormation
Tools
…adding Elastic Load Balancer…
Elastic Beanstalk
CloudFormation
Tools
…Auto Scaling groups
Elastic Beanstalk
CloudFormation
Tools
…and launching instances
Elastic Beanstalk
CloudFormation
Tools
…all wired into the Beanstalk framework
Elastic Beanstalk
CloudFormation
Tools
…and published under a CNAME
Elastic Beanstalk
CloudFormation
Tools
…with logs and app versions held in S3
Elastic Beanstalk
CloudFormation
Tools
How do you create an app for AWS ElasticBeanstalk?
Like any other Java / .Net / PHP / Python
app
No dependencies on AWS inherent for Beanstalk deployment
Elastic Beanstalk
CloudFormation
Tools
Source uploaded and executed in AWS
Source downloaded
and executed locally
Package up as normal…
Elastic Beanstalk
CloudFormation
Tools
Java .war file
Microsoft Web
Deploy package
PHP .zip file
Python .zip file
Git integration IDE plugins
Package up as normal…
Elastic Beanstalk
CloudFormation
Tools
Java .war file
Microsoft Web
Deploy package
PHP .zip file
Python .zip file
Git integration IDE plugins
Console deployments and version updates
Elastic Beanstalk
CloudFormation
Tools
Command line tools and wizards
Elastic Beanstalk
CloudFormation
Tools
CheckDNSAvailability elastic-beanstalk-check-dns-availability
CreateApplication elastic-beanstalk-create-application
CreateApplictaionVersion
elastic-beanstalk-create-application-version
CreateEnvironment
elastic-beanstalk-create-environment
eb init wizard to initialize an application
eb start/stop start/stop an application
eb update
update application version
eb status
get status of a running application
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Git integration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Full admin access on underlying resources EC2 instances, AMIs, AutoScaling groups…
Application prototype
One-click deployment
Production ready AWS infrastructure
Idea App
Idea App Heavy lifting
70%
30%
Idea App Infrastructure
services
Idea App Elastic Beanstalk
Elastic Beanstalk
Idea App
Company launching product into new geography
Online application & marketing site deployed to support launch
Site crashed under exceptional demand - DoS existing datacenter
Needed fast deployment mechanism with no interest in underlying infrastructure
Company launching product into new geography
Online application & marketing site deployed to support launch
Site crashed under exceptional demand - DoS existing datacenter
Needed fast deployment mechanism with no interest in underlying infrastructure
Turned to Beanstalk - deployed & scaled seamlessly, restoring website
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
What happens under the covers of Beanstalk?
Elastic Beanstalk
CloudFormation
Tools
Elastic Beanstalk
AWS
Create security group
Create auto-scaling policy
Create elastic load balancer
Add instance
Install software
Consolidate logs
What happens under the covers of Beanstalk?
Elastic Beanstalk
CloudFormation
Tools
AWS
Create security group
Create auto-scaling policy
Create elastic load balancer
Add instance
Install software
Consolidate logs
A series of calls into AWS to marshal resource
creation/configuration
CloudFormation – underlying automation framework
Elastic Beanstalk
CloudFormation
Tools
Security Scaling
Database
Networking Monitoring
Messaging
Workflow
DNS
Load Balancing
Backup CDN
Compute
Storage
Elastic Beanstalk
CloudFormation
Tools
CloudFormation
A consistent scripting interface for creating/configuring AWS
resources
A framework for lifecycle management of resources
created using scripts
Security Scaling
Database
Networking Monitoring
Messaging
Workflow
DNS
Load Balancing
Backup CDN
Compute
Storage
CloudFormation – underlying automation framework
CloudFormation components & terminology
Elastic Beanstalk
CloudFormation
Tools
Template CloudFormation
Stack
JSON formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS services
Comprehensive service support
Service event aware
Customisable
Framework
Stack creation
Stack updates
Error detection and rollback
Template examples
Elastic Beanstalk
CloudFormation
Tools
Template
JSON formatted file
Parameter definition
Resource creation
Configuration actions
{
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}
Snippet: Taking user defined parameters
Template examples
Elastic Beanstalk
CloudFormation
Tools
Template
JSON formatted file
Parameter definition
Resource creation
Configuration actions
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Create RDS with fixed username and password",
"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
}
}
}
}
Snippet: Creating RDS instance
Template examples
Elastic Beanstalk
CloudFormation
Tools
Template
JSON formatted file
Parameter definition
Resource creation
Configuration actions
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql" : [],
"mysql-server" : [],
"httpd" : [],
"php" : [],
"php-mysql" : []
}
},
"sources" : {
"/var/www/html" : "https://s3.amazonaws.com/
cloudformation-examples/
CloudFormationPHPSample.zip"
},
Snippet: Installing software
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
Parameter driven Dynamic and user-driven
templates
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
Parameter driven Dynamic and user-driven
templates
Collaboration Share templates with ease as just files
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
Known configuration Store stack configuration in source control
Parameter driven Dynamic and user-driven
templates
Collaboration Share templates with ease as just files
NASA TV & Curiosity Route 53 DNS Multi-region & AZ
Elastic Loadbalancing Amazon Live Streaming
Cloud Front Cloud Formation
NASA TV & Curiosity Route 53 DNS Multi-region & AZ
Elastic Loadbalancing Amazon Live Streaming
Cloud Front Cloud Formation
Template ELBs to front secondary cache
~100 Nginx secondary cache servers
2-3 Nginx mid-tier cache servers
Stack
CloudFormation template defined video caching
infrastructure
NASA TV & Curiosity Route 53 DNS Multi-region & AZ
Elastic Loadbalancing Amazon Live Streaming
Cloud Front Cloud Formation
Availability Zone
As demand increased new stacks added to an availability
zone using template
One-click complex
rollout
Where to get more information?
Elastic Beanstalk
CloudFormation
Tools
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
Server templates High availability configurations
RDS configurations Open source stacks
Snippets
+ Cloud Former (prototype)
Where to get more information?
Elastic Beanstalk
CloudFormation
Tools
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
Server templates High availability configurations
RDS configurations Open source stacks
Snippets
+ Cloud Former (prototype)
Comprehensive service support
Elastic Beanstalk
CloudFormation
Tools
Amazon EC2 Instances (On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Elastic Block Store (EBS) Volumes Elastic Load Balancers Elastic IP Addresses Amazon EC2 Security Groups Auto Scaling Groups Amazon Relational Database Service (RDS) Instances Amazon RDS Security Groups Amazon ElastiCache Clusters Amazon ElastiCache Security Groups Amazon ElastiCache Parameter Groups
Amazon CloudWatch Alarms Amazon Simple Storage Service (S3) Buckets
Amazon Simple DB Domains Amazon Simple Queue Service (SQS) Queues
Amazon Simple Notification Service (SNS) Topics Amazon SNS Subscriptions
Amazon Route 53 DNS Records Amazon CloudFront Distributions and Multi-Origin support
AWS Identity and Access Management users and groups AWS Identify and Access Management policies
Amazon Virtual Private Cloud VPCs, Subnets, Gateways Amazon DynamoDB Tables
Comprehensive service support
Elastic Beanstalk
CloudFormation
Tools
Amazon EC2 Instances (On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Elastic Block Store (EBS) Volumes Elastic Load Balancers Elastic IP Addresses Amazon EC2 Security Groups Auto Scaling Groups Amazon Relational Database Service (RDS) Instances Amazon RDS Security Groups Amazon ElastiCache Clusters Amazon ElastiCache Security Groups Amazon ElastiCache Parameter Groups
Amazon CloudWatch Alarms Amazon Simple Storage Service (S3) Buckets
Amazon Simple DB Domains Amazon Simple Queue Service (SQS) Queues
Amazon Simple Notification Service (SNS) Topics Amazon SNS Subscriptions
Amazon Route 53 DNS Records Amazon CloudFront Distributions and Multi-Origin support
AWS Identity and Access Management users and groups AWS Identify and Access Management policies
Amazon Virtual Private Cloud VPCs, Subnets, Gateways Amazon DynamoDB Tables
+ AWS Elastic Beanstalk
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
+
+
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Developer sandboxes with production ready
infrastructure
+
+
Comprehensive control of all aspects of infrastructure
Get started fast with Beanstalk (control the chaos)
Take fine grained control with CloudFormation
(auditable environment specs)
http://aws.amazon.com/elasticbeanstalk/
http://aws.amazon.com/cloudformation/
Find out more…
aws.amazon.com
get started with the free tier
Top Related