Gov 2.0: Scaling, Automation, & Management in the Cloud

Speaker:

‣ jesse@opscode.com‣ @jesserobbins‣ www.opscode.com

Jesse Robbins CEO

Scaling in the Cloud

Opscode makes a new kind of Infrastructure Automation, offered as a hosted Service.

http://www.flickr.com/photos/timyates/2854357446/sizes/l/

•Developers?

•Systems Administrators?

•Developers?

•Systems Administrators?

•Executives/Leaders?

For Developers...

• Do it yourself.

For Developers...

• Do it yourself.

• The infrastructure is the application (and vice versa).

For Developers...

• Do it yourself.

• You are not a Systems Administrator.

For Developers...

• Do it yourself.

• You are not a Systems Administrator.

• You need tools.

Sysadmins..

http://covers.oreilly.com/images/9780596007836/lrg.jpg

Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft

Sysadmins..• Say “Yes”.

• You never liked rack and stack that much anyway.

• You have never been more critical.

• Lean into it.

Executives...

• Not a magic unicorn

Executives...

• Benefits come from efficiency, not raw Capex

Executives...

• Has real cultural implications at every level

Executives...

• Has real cultural implications at every level

• You are the biggest asset to success

(http://radar.oreilly.com/archives/2007/10/operations-advantage.html)

“Traditional” Operations

101520

1 2 3 4 5 6 7 9 10 11 12

Week #

Operations - The “Secret Sauce”

UpkeepConfigOS InstallHardware

101520

1 2 3 4 5 6 7 9 10 11 12

Week #

ExistingNew

(http://radar.oreilly.com/archives/2007/10/operations-advantage.html)

“Traditional” Operations

101520

1 2 3 4 5 6 7 9 10 11 12

Week #

Operations - The “Secret Sauce”

UpkeepConfigOS InstallHardware

101520

1 2 3 4 5 6 7 9 10 11 12

Week #

ExistingNew

This is the secret of Cloud Computing.

Every other virtue stems from here.

You are 10% Unique

And itʼs probablythe things you did wrong

Infrastructure is Hard

1999Inventory, packaged file transers and desktops

2005Unattended bare metal servers “very very” hard7k Nodes took 5 days w/90 success

2007 Unattended bare metal in under 10 minutesFully configured in under 3 mins

2008 Unattended server in 2 minutes 5000 servers in a week

201010k Nodes in under 5 minutes

Infrastructure is changing

‣ Easier to get (good!)...but harder to manage (bad!)

‣ Demand is dynamic

‣ Developers are crucial to Operations

‣ Web / Cloud services are proliferating...and Enterprise is following along.

‣ Manual configuration no longer a crutch

‣ Few tools to solve a ubiquitous problem

Managing Infrastructure Is HardHas Always Been

•Solve very little of the problem...

•Reach just a handful of large, enterprise customers

•Require custom implementations with large professional services bills

•Deployed exclusively on-premise

•Acquired by companies with large consulting organizations (IBM, HP, CA)

Previous Attempts Typically...

Proprietary Solutions

Google, Amazon, Microsoftbuilt their own tools

but it’s “secret sauce”

everyone else is here

... inexperienced & poorly equipped for the world they must now operate in.

“Cloud”

Alistair’s mom’s definition

Cloud Web=

Cloud Web= Internet=

Cloud Web= Internet= Useless=

Slide courtesy Alistair Croll - alistair@rednod.com

Private Public

Managedhosting

Virtualization

Private Public

Managedhosting

Virtualization

Private Public

IaaS IaaS

Managedhosting

Virtualization

Private Public

PaaS PaaS

IaaS IaaS

Managedhosting

Virtualization

Private Public

PaaS PaaS

IaaS IaaS

Managedhosting

Virtualization

Private Public

PaaS PaaS

IaaS IaaS

If you want to

talk clouds,

pick one first.

Infrastructure as a Service(IaaS)

Amazon EC2, Rackspace Cloud, Terremark, Gogrid, Joyent (and nearly every private cloud built on Zenserver or VMWare.)

Dedicatedhardware

On-premiseprivate clouds

Virtualprivate clouds

Third-partypublic clouds

Always on premise

Private

Compliance-enforced

Need to track and audit

Legislative

Data near local computation

Always on premise

Private

Compliance-enforced

Legislative

Can be done anywhere

Testing

Training

Prototyping

Batch processing

Seasonal load

Always on premise

Private

Compliance-enforced

Legislative

Testing

Training

Prototyping

Batch processing

Seasonal load

Always in cloud

Partner access

Proximity to cloud services (storage,

CDN, etc.)

Massively grid/parallel (genomic,

modelling)

Always on premise

Private

Compliance-enforced

Legislative

Testing

Training

Prototyping

Batch processing

Seasonal load

Always in cloud

Partner access

CDN, etc.)

modelling)Lo

Always on premise

Private

Compliance-enforced

Legislative

Testing

Training

Prototyping

Batch processing

Seasonal load

Always in cloud

Partner access

CDN, etc.)

modelling)Lo

Always on premise

Private

Compliance-enforced

Legislative

Testing

Training

Prototyping

Batch processing

Seasonal load

Always in cloud

Partner access

CDN, etc.)

modelling)Lo

Virtual machine(infrastructure cloud)

Always on premise

Private

Compliance-enforced

Legislative

Testing

Training

Prototyping

Batch processing

Seasonal load

Always in cloud

Partner access

CDN, etc.)

modelling)Lo

Compute task(service cloud)

Automation

Bootstrapping

Bootstrapping ApproachesGood Bad Time

Corp Approvals

Agile Corp Approvals

Known Costs, No Variation.

Anything you want, as long as IT pre-approved it.

High Waste (Hoarding)Red Tape

Expensive ($/Time)Long lead time

Known Costs.Total Hardware Control.

Trivial Approvals.

Lower WasteLess Red Tape

Still slowExpensive ($/Time)Shorter lead time

Variable Costs.Highly Adaptable.Minimal lead time.Trivial approvals.

No humans needed.

Variable Costs.No control over hardware.

Must re-train.5-10m

curl -O http://brainspl.at/velocity.sh && sh velocity.sh

Configuration

Configuration ApproachesGood Bad

Manual

Ad-Hoc

Infrastructure as Code

You can do anything.Results in an intimate knowledge

of the details.

Slow.Error Prone (Bus Error!)

Non-repeatable.Difficult knowledge transfer.

More repeatable.Knowledge is dispersed.

Built your way, with your model.

Rarely idempotent.Hard to collaborate.

Brittle.No API.

Repeatable.Idempotent.

Agile.Sharable.

Self documenting.

Have to learn how to use it.Hard things remain hard.

Not magic. (Yet!)

Command and Control

Command and ControlGood Bad

Meatcloud*

Ad-Hoc

Framework

Super flexible.Can do almost anything.

Always easy to find someone to blame.

Free will.

Error Prone.Slow.

Expensive to Scale.Not repeatable.

Free will.

More repeatable.Easier to scale.

Less error prone (hopefully!)

One-off by neccessity.Tooling sprawl.

Hard to share solutions.Much higher learning curve.

One system to learn.Scales well.

Paint by numbers.Repeatable.

Two-Way.

Not everything maps cleanly.Trades depth of knowledge for

ease of use.

*Meatcloud appears in this presentation courtesy of Andrew Shafer - http://is.gd/Ega

Lightning Strikes!

Webservers

Database Servers

Lightning Strikes!

Webservers

Database Servers

Lightning Strikes!

Webservers

Database Servers

Configuration

BootstrappingCommand &Control

MonitoringSystem Updates

Signals Moar!

Provisions

Lightning Strikes!

Webservers

Database Servers

Configuration

Signals Moar!

Provisions

Monitoring Signals Nanite

/node/down Service

Lightning Strikes!

Webservers

Database Servers

Configuration

Signals Moar!

Provisions

DOOMNanite

boots new EC2 Instances, with

Chef Role + Attribute

Nanite removes nodes in Chef

Lightning Strikes!

Webservers

Database Servers

Configuration

Signals Moar!

Provisions

DOOMProvisions

Instances, EBS, Elastic IPs

Lightning Strikes!

Webservers

Database Servers

Configuration

Signals Moar!

Provisions

Chef configures nodes

according to assigned

Lightning Strikes!

Webservers

Database Servers

Configuration

Signals Moar!

Provisions

Chef updates the monitoring

system

A word about Scaling...

Typical Peak Load

Graphs in this portion of the presentation taken from Theo Schlossnaglehttp://omniti.com/seeds/dissecting-todays-internet-traffic-spikes

1.Bring on capacity as traffic ramps up2.Take down capacity as it ramps down3.10-15 Minutes on either side, fully unattended

Atypical Load

1.Hope you know it is coming.2.Increase capacity in advance.3.Take down capacity as it ramps down.

Graphs in this portion of the presentation taken from Theo Schlossnaglehttp://omniti.com/seeds/dissecting-todays-internet-traffic-spikes

No way around

Capacity Planning

However,you are

still better off!

Capacity Planning is king.

http://www.flickr.com/photos/allspaw/2095439645/sizes/l/

Have a queue?

Does it scale linearly with more resources?

Have a queue?

Does it scale linearly with more resources?

Congratulations - you can auto-scale!

http://www.flickr.com/photos/wingler/3429634150/sizes/l/

CAP Theorem

• Consistency

• Availability

• Partition Tolerance

PickTwo

Most SQL Databases

• Choose Consistency over all

• Availability comes distant second

Web Applications need...

• Availability

• Partition Tolerance

“Global temporal consistency is a fiction”

Christopher Brown

Choosing Consistency for your Web App...

Means failure is global

When you choose Partition Tolerance and

Availability...

You fail or succeed for a subset of users

Apologies

• Apologize after the fact for failures

• Better than nothing at all

• Many different tools

• They tweak CAP differently

• CouchDB

• Cassandra

• Redis

• MongoDB

Speaker:

‣ jesse@opscode.com‣ @jesserobbins‣ www.opscode.com

Jesse Robbins CEO

Scaling in the Cloud

Gov 2.0: Scaling, Automation, & Management in the Cloud

Technology

Transcript of Gov 2.0: Scaling, Automation, & Management in the Cloud

Gross Measurement Procedures 6 - British Columbia › assets › gov › farming-natural-resources-and... · 6-2 November 1, 2011 Scaling plays a critical role in fulfilling the financial

7 Pillars for Scaling Your Automation Program...into scaling RPA To realize the true value of RPA, you need to scale beyond initial task automations. Every company is different, but

Gov 1.0 vs. Gov 2.0

Scaling MATLAB - MathWorks · Scaling MATLAB for Your Organisation and Beyond Rory Adams. 2 MATLAB at Scale Front-end scaling ... automation control system • Variety of sensors

Growth in the Machine...scaling automation deployment 4. The critical building blocks in designing an intelligent automation strategy. Intelligent automation defined For this study,

Gov 2.0 in a gov 1.0 world

March 3 - 6, 2019 Mesa, Arizona Archive...–Density scaling –Speed scaling –Power scaling –Cost scaling •CPU architectures are not scaling –Processor frequency scaling ended

Scaling your Application for Growth using Automation (CPN209) | AWS re:Invent 2013

Parker Automation ControllerScaling for each axis is located on the SoftMotion Drive: Scaling/Mapping tab. Scaling is set by 3 steps: 1. Defining the motor’s feedback resolution/increments.

Continuous Testing Transformation...resulted in a significant test automation bottleneck. The director of testing, aware that this would prevent the team from scaling test automation

Scaling Up Robotics Process Automation | Accenture

Gov. 21 Gov. Gov. 1955. - Federal Reserve Bank of St. Louis

Intelligent Process Automation The challenge of scaling up ...€¦ · embracing intelligent automation powered by the “golden triangle” of robotic process automation (RPA), artificial

Gov state and local gov 50m

DevOps Best Practices: Managing and Scaling Release Automation Using Visual and Code-centric Methods

Test Design: Scaling UI Automation from the Ground Up

Scaling PCR Workflows from Benchtop to Automationhpst.cz/.../files/attachments/scaling...automation.pdfScaling PCR Workflows from Benchtop to Automation Broadcast Date: Tuesday, June

082510 gov role of gov 2 50m

Scaling Automation Mpato Mokobori. Pharmaceutical Clinical Support in Gauteng.

Scaling Threat Intelligence Practices with Automation