Infosec at Ludicrous Speeds - Rugged DevOps

67
@RealGeneKim, [email protected] Session ID: Security is Dead. Long Live Rugged DevOps: IT at Ludicrous Speed… Gene Kim IT Revolution Press

Transcript of Infosec at Ludicrous Speeds - Rugged DevOps

Page 1: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Session ID:

Security is Dead. Long Live Rugged DevOps:

IT at Ludicrous Speed…

Gene KimIT Revolution Press

Page 2: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Act I: IT Ops Fixing Fragile Artifacts

Page 3: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 4: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Act 2: The Product Managers

Page 5: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Act 3: The Developers

Page 6: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 7: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 8: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Act 4: IT Ops And Dev At War

8

Page 9: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Act 5: Nothing Left For Infosec

Page 10: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 11: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The DownwardSpiral…

Page 12: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The IT Core Chronic Conflict

Every IT organization is pressured to simultaneously: Respond more quickly to urgent business needs Provide stable, secure and predictable IT service

Source: The authors acknowledge Dr. Eliyahu Goldratt, creator of the Theory of Constraints and author of The Goal, has written extensively on the theory and practice of identifying and resolving core, chronic conflicts.

Page 13: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Every Company Is An IT Company…

95% of all capital projects have an IT component…

50% of all capital spending is technology-related

We are here…

Where we need to be…

IT is always in the way(again…)

Page 14: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

There Must Be A Better Way…

15

Page 15: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: John Allspaw

Page 16: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 17: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: John Allspaw

Page 18: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: John Allspaw

Page 19: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: John Allspaw

Page 20: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 21: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: Theo Schlossnagle

Page 22: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: Theo Schlossnagle

Page 23: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: Theo Schlossnagle

Page 24: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: James Wickett

Page 25: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]: John Jenkins, Amazon.com

Page 26: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Three WaysAnd Six Prescriptive StepsInfosec Can Take

27

Page 27: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

If I Could Wave A Magic Wand, Everyone Will…

Become conversant with DevOps and recognize the practices when you see them

Be energized about how information practitioners can contribute in this organizational journey

Leave with some concrete steps to get some great outcomes

Become a part of a team that starts putting DevOps practices into place

28

Page 28: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The First Way:Systems Thinking

Page 29: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The First Way:Systems Thinking

(Business) (Customer)

Page 30: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The First Way:Systems Thinking (Left To Right)

Understand the flow of work Always seek to increase flow Never unconsciously pass defects downstream Never allow local optimization to cause global

degradation Achieve profound understanding of the system

Page 31: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

“Annual business planning sessions can be madding. They think IT Operations is an ‘all you can eat buffet.’”

-Ben Rockwood, Director Systems Engineering, Joyent

Page 32: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Practice #1: Define The Work and Make It Visible

Business projects (e.g., new order entry system) Internal IT projects (e.g., create new

environments, infosec remediation) Changes (e.g., deploys, improve database

performance) Unplanned work (e.g., site down, site impaired,

security incident)

33

Page 33: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Day 2: PMO Meeting

Page 34: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Practice #2: Create One Step Environment Creation Process

Make environments available early in the Development process

Make sure Dev builds the code and environment at the same time

Create a common Dev, QA and Production environment creation process

Page 35: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Change the Agile sprint policy:

“At the end of each sprint, we must have working code and the environment it runs in!”

Page 36: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Infosec Insurgency

Find the automated infrastructure project team (e.g., puppet, chef) Release managers can provide hardening guidance Integrate and extend their production configuration

monitoring Put ASSERTs to find misconfigurations, enforce https,

etc. Define what changes/deploys cannot be made

without triggering full retest

37

Page 37: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The First Way:Outcomes

Creating single repository for code and environments

Determinism in the release process

Consistent Dev, QA, Int, and Staging environments, all properly built before deployment begins

Decreased cycle time

Reduce deployment times from 6 hours to 45 minutes Refactor deployment process that had 1300+ steps

spanning 4 weeks Faster release cadence

Page 38: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Second Way:Amplify Feedback Loops

Page 39: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Second Way:Amplify Feedback Loops (Right to Left)

Understand and respond to the needs of all customers, internal and external

Shorten and amplify all feedback loops: stop the line when necessary

Create quality at the source Create and embed knowledge where we need it

Page 40: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Toyota Andon Cord

41

Page 41: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

“We found that when we woke up developers at 2am, defects got fixed faster than ever.”

Patrick Lightbody CEO, BrowserMob

Page 42: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Pattern #3: Embed Dev Into IT Ops

Embed Dev into IT Ops incident escalation process

Invite Dev to post-mortems/root cause analysis meeting

Have Dev and Infosec cross-train IT Operations Ensure application monitoring/metrics to aid in

Ops and Infosec work (e.g., incident/problem management)

Page 43: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Second Way:Outcomes

Defects and security issues getting fixed faster than ever

Reusable Ops and Infosec user stories now part of the Agile process

All groups communicating and coordinating better

Everybody is getting more work done

Page 44: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Third Way:Culture Of Continual Experimentation And Learning

Page 45: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Third Way:Culture Of Continual Experimentation And Learning

Foster a culture that rewards: Experimentation (taking risks) and learning from

failure Repetition is the prerequisite to mastery

Why? You need a culture that keeps pushing into the danger

zone And have the habits that enable you to survive in the

danger zone

Page 46: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Break Things Early And Often

“Do painful things more frequently, so you can make it less painful… We don’t get pushback from Dev, because they know it makes rollouts smoother.”

-- Adrian Cockcroft, Architect, Netflix

Page 47: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 48: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Pattern #5: Inject Failures Often

Page 49: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

You Don’t Choose Chaos Monkey…Chaos Monkey Chooses You

Page 50: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Pattern #6: Break Things Before Production

Enforce consistency in code, environments and configurations across the environments

Add your ASSERTs to find misconfigurations, enforce https, etc.

Add static code analysis to automated continuous integration and testing process

Page 51: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Pattern #6: Allocate 20% Of Cycles To Technical Debt Reduction

Page 52: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Recognize Compounding Technical Debt…

Page 53: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

That Gets Worse…

Page 54: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

And Fixing It…

Source: Pingdom

Page 55: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 56: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

An Innovation Culture

“By installing a rampant innovation culture, they now do 165 experiments in the three months of tax season.

Our business result? Conversion rate of the website is up 50 percent. Employee result? Everyone loves it, because now their ideas can make it to market.”

--Scott Cook, Intuit Founder

57

Page 57: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Why Do I Think This IsImportant?

58

Page 58: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The DownwardSpiral…

Page 59: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 60: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

The Three Ways: Some Patterns

First Way Second Way Third WayDefine The Work And Make It Visible

Make Environments Available Early

Wake Up Developers

Embed Dev Into IT Operations

Break Things Early And Often

Reserve 20% Of Cycles For Technical Debt Reduction

Page 61: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 62: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Help The Business Win…

Page 63: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

With Support From Your Peers…

Page 64: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

And Do More With Less Effort…

Page 65: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Page 66: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

When IT Fails: A Business Novel and The DevOps Cookbook

Coming January 15, 2013 and Q1 2013

“The greatest IT management book of our generation.” Branden Williams, CTO Marketing, RSA

“The lessons in When IT Fails might just save your business if IT fails for you. Every IT executive should share this book with their business peers.” James Turnbull, VP Operations, Puppet Labs and author of “Pro Puppet”

“This book will have a profound effect on IT, just as The Goal did for manufacturing.’ Jez Humble, co-author of the Jolt award-winning book Continuous Delivery, and Principal at ThoughtWorks Studios.

Page 67: Infosec at Ludicrous Speeds - Rugged DevOps

@RealGeneKim, [email protected]

Our Mission: Positively Impact The Lives Of One Million IT Workers By 2017

For these slides, the “Top 10 Things You Need To Know About DevOps,” Rugged DevOps resources, and updates on the book:

Sign up at http://itrevolution.com Email [email protected]

Or text “[email] 74730” to +1 (858) 598-3980

Visit: http://www.instantcustomer.com/go/74730