Modelling Devops & Seeing the system

Modelling

Devops

http://farm6.staticflickr.com/5291/5540608712_a041894564_z.jpg

@patrickdeboishttp://jedi.be/blog




http://jedi.be/blog

http://jedi.be/blog

This presentation๏Devops ‘Models’ in the wild

๏Seeing the system๏‘Monitoring Stream Mapping Technique’๏Observations

๏Focus on Flow๏Identifying 4 areas of improvement๏Example practices

models to explain devops

DEV OPS

Paul Peisner

https://twitter.com/dysinger

MyopicDevops



http://dev2ops.org/2012/09/use-devops-to-turn-it-into-a-strategic-weapon/

Damon Edwards



https://cacoo.com/diagrams/uapwdcN6SDfwClDY-351A0.png

Mathias Marschall

An emerging set of practices



http://www.slideshare.net/dev2ops/you-cant-change-culture-but-you-can-change-behavior-and-behavior-becomes-culture

GeneKim





1. “Seeing”the System

Value Stream Mapping

“Monitoring”Stream Mapping

aka Reverse Value Stream Mapping

• Cross-Silo Group exercise

• (both dev, test, ops ...)

• Use a big wall

• preferred near their officespace

• preferred permanent

• Whiteboard/Long Paper for drawing

• Post-it with different colors

Exercise Preparation

ProductionDraw 4 quadrantsStart upper right

ProductionPut all production

components on the board

ProductionEach component =

1 post-it

Production

Components (architecture)

Put all production

components on the board

“I don’t know”can be a great conversation

starter

Production


External

Add components outside your control your team depends

outside your company (like cloud services)

Production


External

Add components outside your control

your team depends on inside your company(like firewall, dns, mail,

ldap)

Production


Supportive

Add the supportive components

(backup, monitoring...)

Production


Explain how these components interact

(architecture)

Production


Explainhow you know they are working correctly

“monitoring”

Production

EndUserComponents (architecture)

Add ‘core’ functionalities you’re providing to the

end-user

Production


Map the functionalities to your components

(is your monitoring covering all components?)

Production

People (process)


Now repeat the processfor people/groupsinvolved in support

Production

People (process)


Add all groups/people involved for support

Production

People (process)


External

Add all externalgroups/people involved

for support

Production

People (process)


Supportive

Add all supportivegroups/people involved

for support

Production

People (process)


Explain how these people interact

(process)

Production

People (process)


How do you the process is working well?

(KPI’s)

Production

People (process)


Relate the people/groups

to each of the components(who get alerts)

Production

Components (architecture)People (process)

EndUser

Add the enduser support

services you offer

Production


EndUser

Relate enduser supportto the process and

to the components to the functionalities

“To fix something we know we have to

go through development

process”

Production


EndUserBusiness

Enter Project Mode

Production


EndUserBusiness

Repeat mappingfor people/process

initiate improvements in production

Production


Dev, Test, QA

EndUserBusiness

Repeat mapping for

dev/test components

Production


Dev, Test, QA

EndUserBusiness

“Seeing the devops system”

a first step ofevolving

“commonground”

Some Observations

Production


Dev, Test, QA

EndUserBusiness

More maturepeople consider

dev/test/qa as part of production

Production


Dev, Test, QA

EndUserBusiness

More maturedev/test/qa/prod

components are the same

Production


Dev, Test, QA

EndUserBusiness

More mature= Less ????

about end-user functionalities

Production


Dev, Test, QA

EndUserBusiness

More mature= > monitoring/

support for external/cloud

services

2. Focus on Flow

Production


Dev, Test, QA

EndUserBusiness

Remove friction

http://philippe.kruchten.com/2013/11/24/friction/

“Friction: the resistance that one surface or object encounters when moving over another.” [Merriam-Webster dict.]



Production


Dev, Test, QA

EndUserBusiness

Find your bottleneck(s) = friction

Production


Dev, Test, QA

EndUserBusiness

Technical Debt

Social Debt

OPSDEV

http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/

4 areas of improvement



OPSDEV

Area 1: Extend delivery to production




OPSDEV

Area 2: Extend operations feedback to project





OPSDEV



Area 3: Embed Projectknowledge into Operations




OPSDEV

Area 4: Embed Operations knowledge into Project







Tools Can you ‘technically’ do it

Process Should you do it

People(culture)

Will you do it

“Layers per Area”

Think ‘tags’ of things you do in an area

Area X

“Area Maturity Level”a way to quantify

your progress

http://groups.google.com/group/devops/browse_frm/thread/f3de603a4cea493e?scoring=d&



Initial Unpredictable poorly controlled and reactive

Managed Focused on project, often and reactive

Defined Focused on organization and proactive

Quantitatively Managed Measured and controlled

Optimizing Focus on Improvement

CMMI - Maturity Levels (Process centric)

Intro Using Source Control ...

Novice Builds Triggered by Commit ...

Intermediate Automated Deployment to Testing ...

Advanced Automated Functional Testing ...

Insane Continuous Deployment to Prod ...

Alternative Maturity Levels(cfr. Continuous Integration Model)

http://blogs.urbancode.com/continuous-integration/continuous-integration-maturity-model/



Name AreaAreaAreaArea

Provision dev/test and prod from the same src

DEVDEV delivery to ProdEmbed Project

knowledgeProvision dev/test and prod from the same src Embed Operations

knowledgeEmbed Operations

knowledgefeedback from Prod OPS

Tools IntroLayer Level

Practice: Use a configuration mangement system like chef/puppet to provision dev,test and prod from the same source

Pattern: Automation, reuse of code

Principles: By reusing the code it gets tested more && often more frequent/earlier feedback

Production


Dev, Test, QA

EndUserBusiness

Different places where we can improve

http://devopsdays.org/blog/2010/05/16/the-panel-experiment-and-ignite-devops/

Andrew Schaefer



Some examplepractices

always understand the context before applying

http://www.jedi.be/blog/2013/05/24/Infrastructure%20as%20Code/

cfengine, puppetchef, ansible, salt, ...

OPS DEV





Infrastructure as code





OPS DEV






Using same technologyversioning/testing/...





OPS DEV







Executable ‘documentation’





OPS DEV







Executable ‘documentation’

Common language



OPS DEV





http://www.hashicorp.com/

Reproduceableenvironments



OPS DEV







Dev/Test machines are using the same setup as

Production



OPS DEV








Production

When your app requires changes on the system you can try them and change them ourself



OPS DEV








Production

When your app requires changes on the system you can try them and change them ourself

Ops can propagateproduction changes to

dev easily



OPS DEV





Monitoring & Metrics

#monitoring{love,sucks}

OPS DEV







Enable devs to see the production usage/

problems(graphite)

OPS DEV








problems(graphite)

Devs can add metrics in their code(Statsd)

OPS DEV








problems(graphite)

Devs can add metrics in their code(Statsd)

Make non-subjectivedecisions at the start of

the project

OPS DEV




Area 3: Embed Projectknowledge into OperationsDevelopers

wearing pagers

OPS DEV





wearing pagers

Devs you have run operations learn and

take it back to project

OPS DEV





wearing pagers



Devs know the app better and can teach/

train operations people

OPS DEV





wearing pagers



Devs know the app better and can teach/

train operations people

Better Tests to preventproduction problems/

pain

OPS DEV





https://github.com/Netflix/SimianArmy



OPS DEV






Experience problemsearlier in the pipeline

(less costly)



OPS DEV





OPS DEV





Visualize frequency of production errors and enable extraction of

‘context’

OPS DEV





https://speakerdeck.com/jnewland/chatops-at-github



OPS DEV






Shared view on the system events



OPS DEV





(Public)Post-Mortems

OPS DEV





Objective viewon problems

(think retrospective)

(Public)Post-Mortems

OPS DEV





Game Days

http://www.youtube.com/watch?v=LCZT_Q3z520&noredirect=1



OPS DEV





Find bottlenecks in procedures/technology

Game Days




OPS DEV





SadOps

http://github.com/railsmachine/sadops



Training through simulation

OPS DEV





SadOps

http://github.com/railsmachine/sadops



“You build ityou run it”

Amazon

DEVOPS

3. Find Feedback loops

Caveat: DifferentiateSymptoms vs real cause

Your bottleneck will move

is it even in Dev & Ops ?

MGMT FIN SALESHR

Business CustomerOPSDEV

Beware of the DEVOPS WIZARDSthat don’t know your context

Devops is not just about tools

or automation

Understanding the systemneeds to be done together

Takeways

Everbody should care about the whole system

4. Look forContinuous

Improvement

Places to look for ideas

Places to look for ideas (2)

Modelling Devops & Seeing the system

Technology

Transcript of Modelling Devops & Seeing the system