Agenda

• A brief history of the State of DevOps Report

• Theory, survey design, demographics and psychographics

• IT and security performance findings

• Continuous integration/delivery redux

About the authors

Jez Humble

@jezhumble

Nicole Forsgren, PhD@nicolefv

Gene Kim

@realgenekim

Nigel Kersten

@nigelkersten

Alanna Brown

@alannapb

DevOps Grows Up

2012: What is devops?

2013: DevOps adoption is accelerating.

2014: Holy cow! DevOps works!

2015: IT goes lean.

2016: Shifting left.

IT performance matters!

Firms with high-performing IT organizations were twice as likely to exceed their profitability, market share and productivity goals.

http://bit.ly/2015-devops-repor t/http://bit.ly/2014-devops-repor t/

IT performance

• Lead time for changes

• Release frequency

• Time to restore service

• Change fail rate

Not all data is created equal

• Who here thinks surveys are sh*t?

[Nicole should probably turn around]

• Who here LOVES the data from their log files?

And who has seen sh*t data in a log file?

What is a latent construct?

We use

PSYCHOMETRICS

to make our survey data good*

*or give us a reasonable assurance that it’s telling us what we think it’s telling us (& some of this can also apply to your log data)

Psychometrics includes:

Construct creation (manual)

• When possible: use previously validated constructs

• Based on definitions and theory, carefully and precisely worded, card sorting task, pilot tested

Construct evaluation (statistics)

• Establishing Validity: discriminant and convergent

• Establishing Reliability

Analysis methods

Statistics bingo!

• Measure assessment: measures exhibit good psychometric properties (composite reliability, AVE, EFA measures load well and do not cross-load, etc)

• Model assessment: PLS was used to assess the structured equation models

Analysis methods

A note about prediction: one of three conditions must be met:

1. Longitudinal (no, this is cross-sectional)

2. Randomized, experimental design (no, this is a non-experimental)

3. Theory-based design

When this condition was not met, only correlations were tested and reported.

Demographics

* We don’t’ see significant differences among enterprise organizations

Demographics

Demographics

High-performing organizations are decisively outperforming their lower-performing peers in terms of throughput.

“What is your lead time for changes?”

“How long does it take to go from code committed to code successfully running in production?”

IT Performance Over the Years

Deploy Frequency Change Lead Time Mean Time to Recover

Employees in high-performing organizations are 2.2 times more likely to recommend their organization as a great place to work.

More likely to recommend their organization to a friend.

Because they address security at every stage, high-performing teams spend less time fixing security issues.

Capital One: DevOpsSecInformation Security

Business Development Operations

• Application Security• Information Security

• Security Security• Infrastructure Security

• Requirements• Feature Request• Roadmap

• Architecture• Design• Code• Test

• Infrastructure• Platforms• Environment• Deployment• Incident Mgmt.• Change & Release Mgmt.

DevOpsSec

• Conduct security review for all major features

• Integrate Information Security into the entire software delivery lifecycle

• Test security requirements as part of automated testing process.

• Ensure that Information Security defines pre-approved, easy-to-consume libraries, packages, toolchains and processes

Security is a priority in my organization

These industries responded highest (75% Agree or Strongly Agree)

• Financial Services

• Government

Strongly agree

Strongly disagree

Neutral

These departments responded highest (71-73% Agree or Strongly Agree)

• Professional Services

• Release Engineering

High performers spend 29% more time on new work than low performers, and 22% less time on unplanned work and rework.

21%

Impact of continuous delivery

Effective test data management

Comprehensive, fast and reliable test and deployment automation

Trunk based development and continuous integration

Application code and app & system configuration all in version control

Incorporating security (and security teams) into the delivery

process

Together the factors on the left model continuous deliver y which leads to

Less rework

Lower levels of development pain

Generative performance-oriented culture (per Westrum’s Model)

Higher levels of IT performance (higher throughput and stability)

Identifying strongly with the organization you work for

Higher level of org performance (productivity, market share, profitability)

Lower change fail rates

“Identifying strongly with the organization”

• I am glad I chose to work for this organization rather than another company.• I talk of this organization to my friends as a great company to work for.• I am willing to put in a great deal of effort beyond what is normally expected to help my

organization to be successful.• I find that my values and my organization's values are very similar.• In general, the people employed by my organization are working toward the same goal.• I feel that my organization cares about me.These items are adapted from Kankanhalli, Atreyi, Kwok-Kee Wei, and Bernard C.Y. Tan (2005)

Lean product management

Gathering, broadcasting, and implementing customer feedback

Splitting work into small batches and making visible the flow of work through

the deliver y process

Together the factors on the left model continuous deliver y which leads to

Generative performance-oriented culture (per Westrum’s Model)

Higher levels of IT performance (higher throughput and stability)

Identifying strongly with the organization you work for

Higher level of org performance (productivity, market share, profitability)

Thank you!