Accelerating Innovation with DevOps on AWS

Accelerating Innovation with DevOps on AWS Adrian White Solutions Architect, AWS

v

Innovation, Quality, Governance

v

An example CI/CD workflow

Stacks / Environments(s)

CI/CD Tool Issue Tracker

SCM

Infrastructure automation / deployment

Test tools / harnesses

v

An example CI/CD workflow

Tasks for AWS

PHPUnit jQuery …

CloudFormation Stack(s)

v

A CI/CD pipeline

Teardown

Release

Deploy

Test

Code

v

Getting your source under control

Prod Feature UAT

Code Test

Deploy Release Teardown

Code Test


Code Test


Code Test


Code Test


v

Automated merging

UAT Feature 2 Feature 1

Code Test


v

What does CI give us?

•  Test driven promotion (of development change)

•  Increasing velocity of feedback cycle through iterative change

•  Contain change to reduce risk

•  Bugs are detected quickly

•  Automated testing reduces size of testing effort

Code Test


v

Demo: Reviewing a CI pipeline

v

What does CD give us?

•  Changes are pushed quickly to production

•  Immediate feedback from users

•  Supports A/B testing or “We test customer reactions to features in production”

•  Hardens, de-risks the deployment process

•  Gives us a breadth of data points across our applications

Code Test


v

AWS Application Management Services

Elastic Beanstalk OpsWorks CloudFormation

Application Container Application & Resource Management

Automated Code Deployment

Provision and manage AWS Resources

You want to quickly get code running scalably and reliably without worrying about the underlying resources. “Application-centric approach”

You want to run an application with the ability to customize and control software or resources.

“DevOps automation”

You want to rapidly release new features and avoid

downtime during complex deployments.

“Automated deployments”

You want a simple, yet finely controllable way to model and version collections of AWS resources.

“Fine grained control”

ALMA Service Console Icons

AWS CodeDeploy

ALMA Service Console icons and names are subject to review.

CodeDeploy

CodeDeploy

AWS Pipelines

AWS tPipelines

Pipelines

AWS Revision Control


Revision Control

CodeDeploy

Code Test


v

Deployment approaches

•  Deploy in-place •  Manage interruption •  Fast deployment

•  Discrete environment •  Bake AMI •  Multiple environments from branches •  Support A/B testing •  “Rolling DNS”

Code Test


v

AWS CodeDeploy Automated Deployments Minimize Downtime Centralized Control Easy Integration

15

ALMA Service Console Icons

AWS CodeDeploy

ALMA Service Console icons and names are subject to review.

CodeDeploy

CodeDeploy

AWS Pipelines

AWS tPipelines

Pipelines



Revision Control

Code Test


Automated Deployments

Easy deployment to EC2 Auto Scaling support Consistent and repeatable

Dev Test Prod

Code Test


v2 v1 v1 v1 v1 v1 v1 v1

v2 v2 v1 v1 v1 v1 v1 v1

v2 v2 v2 v2 v1 v1 v1 v1

v2 v2 v2 v2 v2 v2 v2 v2

One-at-a-time Min. healthy hosts = 99%

[Custom] Min. healthy hosts = 75%

Half-at-a-time Min. healthy hosts = 50%

All-at-once Min. healthy hosts = 0

Configurable Deploy Speed Code Test


Product Integrations

Any application Reuse setup scripts Integrate with toolchain

Shell scripts

PowerShell scripts

Custom programs

Configuration recipes

Code Test


v

Baking an AMI

•  Start a builder instance •  Bootstrap with cfn-init, cfn-signal

•  Bake your AMI •  EC2 CreateImage

•  Tag it

•  Cost remediation •  Reporting

•  Destroy / clean up the builder instance

OS

Platform

Framework

Container

Web

Your Code

OS

Platform

Framework

Container

Web

Your Code

OS

Platform

Framework

Container

Web

Your Code

Code Test


v

AWS::CloudFormation::Init "AWS::CloudFormation::Init" : {

“cfn” : {

"packages" :{

"yum" : { "httpd" : [] }

},

"files":{

“/home/ec2-user/myfile.html:{

"source" : { "Fn::Join" : [ "", ["https://s3-ap-southeast-2.amazonaws.com/",{ "Ref" : "S3Bucket" },”/myfile.html”]] },

"mode":"000644",

"owner":"root",

"group":"root",

"authentication":"S3AccessCreds”

},

v

AWS::CloudFormation::Init "services": {

"sysvinit" : {

"httpd" : {

"enabled" : "true",

"ensureRunning" : "true"

}

}

}

}

v

Demo: Reviewing a CD pipeline (data sources + web application)

v

Release

Release

Test the new stack

Match the traffic between the two stacks

Update the “floa>ng” DNS record

Send No>fica>on(s)

ROLLBACK

v

Blue green deployment

cicd-‐bootcamp-‐2.com cicd-‐bootcamp-‐2.com

cicd-‐bootcamp.com

Code Test


v

Teardown

Teardown

Ensure that no traffic is moving though ELB

Teardown the CloudForma>on Stack

Deregister the AMI

v

Teardown

cicd-‐bootcamp-‐2.com cicd-‐bootcamp-‐2.com


Code Test


v

Teardown

cicd-‐bootcamp-‐2.com


Code Test


v

Advanced Techniques

•  Managing CD sprawl / Segregation of duties •  CloudFormation componentization and re-use

•  Extending CloudFormation with custom resources •  Injecting dynamic data into your templates •  Manage custom resources e.g. relational database schemas

•  Extending your CICD tools •  Making your CI and CD tools AWS-aware

v

Stack chaining

v

CloudFormation Merging

Git Git

CloudFormation

Operations Repo Application Repo

VPC Subnets

Security Groups CloudFormation Frameworks

Best Practice

Application Code Application CloudFormation Load Balancing Setup

v

CloudFormation Custom Resources

Region

SQS Queue

AWS CloudFormation

Custom Resource Topic

Auto scaling Group

Custom Resource Implementation

•  Add New Resources •  Including AWS resources not currently

supported by CFN •  Interact with the CloudFormation Workflow •  Inject dynamic data into a stack •  Extend the capabilities of existing resources •  Data management via CloudFormation •  It’s really simple if you use aws-cfn-

resource-bridge •  Install or fork from

https://github.com/aws/aws-cfn-resource-bridge

Create app:WebApp version:1.2 os:linux

DynamoDB

1 2

3

4

6

Query table, get AMI ID where: app=WebApp version=1.2 os is linux

5

Output ami-id:ami-xxxxx

Returns AMI id in ami-id key 7

Deploy EC2 Instance with AMI ami-xxxxx

v

Custom Resources - DatabaseSchema "MyDBSchema" : {

"Type" : "Custom::DatabaseSchema”, "Version" : "1.0",

"Properties" : {

"ServiceToken": "arn:aws:sns:us-east-1:12345EXAMPLE:DBSchema",

"databaseChangeLog" : [ {

"changeSet" : {

"id" : "1",

"author" : "adamthom",

"changes" : [ {

"createTable" : { … }

} ]

}

}

}

}

v

Custom Resources - DatabaseSchema "createTable" : {

"tableName" : "example",

"columns" : [ {

"column" : {

"name" : "id",

"type" : "int",

"autoIncrement" : true,

"constraints" : {

"primaryKey" : true,

"nullable" : false

}

}

} ]

}

v

Extending your CI and CD tools

DynamoDB Tasks for AWS

v

Situation Awareness Burden of Responsibility

APIs

Tasks for AWS

v

Containerisation

•  Build environments for artifacts, don’t update environments with artifacts

•  All environments are transient

•  Standardisation, abstraction and portability

v

Docker & AWS Elastic Beanstalk

•  A framework for managing Linux containers

•  LXC containers are more lightweight than VMs

•  Amazon Linux (2014.03) bundles Docker 0.9 and LXC 0.9

•  Docker containers on Beanstalk are Go!

v

EC2 Container Service (Preview)

•  Highly scalable, high performance container management

•  Supports Docker containers within your VPC

•  Manages container lifecycle and definitions

•  Cluster and container scheduler

v

Demo: Release & Teardown 2

1

Code Test


Production Code Test


v

Innovation, Quality, Governance

Discrete environments for each branch

Automated testing on every commit on every branch

Leverage CD tools to provide separation of duties

Audit Logs

Git approvals process

Use custom resources to extend CloudFormation

Leverage DNS

Interface with the API

Environments for artifacts

Accelerating Innovation with DevOps on AWS

Technology

Transcript of Accelerating Innovation with DevOps on AWS