Agenda

A brief history of the State of DevOps Report

Theory, survey design, demographics and psychographics

IT and security performance findings

Continuous integration/delivery redux

About the authors

Alanna Brown@alannapb

Nicole Forsgren, PhD@nicolefv

Jez Humble@jezhumble

Nigel Kersten@nigelkersten

Gene Kim@realgenekim

DevOps Grows Up

2012: What is devops?

2013: DevOps adoption is accelerating.

2014: Holy cow! DevOps works!

2015: IT goes lean.

2016: Shifting left.

IT performance matters!

“Firms with high-performing IT organizations were twice as likely to exceed their profitability, market share and productivity goals.”

http://bit.ly/2015-devops-report/http://bit.ly/2014-devops-report/

IT performance

lead time for changes

release frequency

time to restore service

change fail rate

Not all data is created equal

Who here thinks surveys are sh*t?

[Nicole should probably turn around]

Who here LOVES the data from their log files?

And who has seen sh*t data in a log file?

What is a latent construct?

We use

PSYCHOMETRICSto make our survey data good*

*or give us a reasonable assurance that it’s telling us what we think it’s telling us (& some of this can also apply to your log data)

Construct creation (manual)

When possible: use previously validated constructs

Based on definitions and theory, carefully and precisely worded, card sorting task, pilot tested

Construct evaluation (statistics)

Establishing Validity: discriminant and convergent

Establishing Reliability

Psychometrics includes:

Statistics bingo!

Measure assessment: measures exhibit good psychometric properties (composite reliability, AVE, EFA measures load well and do not cross-load, etc)

Model assessment: PLS was used to assess the structured equation models

Analysis methods

A note about prediction: one of three conditions must be met:1. Longitudinal (no, this is cross-sectional)2. Randomized, experimental design (no, this is a non-

experime3. Theory-based design

When this condition was not met, only correlations were tested and reported.

Analysis methods

High-performing organizations are decisively outperforming their lower-performing peers in terms of throughput.

“What is your lead time for changes?”

“How long does it take to go from code committed to code successfully running in

production?”

IT Performance Over the Years

Employees in high-performing organizations are 2.2 times more likely to recommend their organization as a great place to work.

Because they address security at every stage, high-performing teams spend less time fixing security issues.

23

High performers achieve better outcomes by: ● Conducting security review for all major

features while ensuring that the security review process does not slow down development.

● Integrating Information Security into the daily work of the entire software delivery lifecycle. This includes providing input during the design of the application, attending software demos and providing feedback during demos.

● Testing security requirements as a part of the automated testing process.

● Ensuring that Information Security has made pre-approved, easy-to-consume libraries, packages, toolchains and processes for developers and IT operations to use in their work.

Security is a priority in my organization

Strongly agree

Strongly disagree

Neutral

These industries responded highest (75% Agree or Strongly Agree)

● Financial Services ● Government

These departments responded highest (71-73% Agree or Strongly Agree)

● Professional Services● Release Engineering

High performers spend 29% more time on new work than low performers, and 22% less time on unplanned work and rework.

Impact of continuous delivery

“Identifying strongly with the organization”

● I am glad I chose to work for this organization rather than another company.● I talk of this organization to my friends as a great company to work for.● I am willing to put in a great deal of effort beyond what is normally expected

to help my organization to be successful.● I find that my values and my organization's values are very similar.● In general, the people employed by my organization are working toward the

same goal.● I feel that my organization cares about me.

These items are adapted from Kankanhalli, Atreyi, Kwok-Kee Wei, and Bernard C.Y. Tan (2005)

Lean product management

Join us to celebrate the launch of the 2016 State of DevOps Report!

Stop by the Puppet booth (#311) from 5:30 - 6:30 PM tonight for a black and tan on us. And bring your favorite dev or ops friend!

Get your copy of the 2016 State of DevOps Reportpuppet.com/devops-report-2016

Sign up for our ROI whitepaper & get peer-reviewed researchdevops-research.com

Email jezhumble@sendyourslides.com with the subject DevOps for more info!

Thank you!

32