Succeeding with Integration

Integrating the work of several Scrum teams

christophe.achouiantz@avega.se

DevelopmentTeam

DevelopmentTeam

Product / Serviceincrement in production

The Ideal Team

DevTeamDev

Team

DevTeamDev

Team

DevTeamDev

Team

IntegrationTeam

IntegrationTeam

Product / Serviceincrement in production

The Integration Team: integrating work from several teams

The Classical Integration/Release process

Environments

Dev Project IT Dept.Release Project

Handover(at the end)

Handover(at the end)

The Integration Team: to integrate continuously

Integration Team

Development Teams

Concurrent, iterative, incremental Activities

Development Phase Release Phase

10 Key success factors for the integration team @ PPM

1. Integrate early2. Team has the necessary resources3. Continuous integration4. Automated tests5. At least two test environments6. Early performance tests7. Stop the line8. Clear contract between Dev. and Integration teams9. Make the process visible10. Make the status visible (and understandable)

• Slice User Stories the ”right way” to start integrating from the start (”tracer bullet”)

• Multiple component teams must synchronize their backlogs to achieve the “tracer bullet” effect

1 - Integrate early

Front-end Back-end DBMiddle-wear Sprint 1

Sprint 2

Sprint 3

2- Team has the necessary resources: competence

Testers• Functional/Integration testers• Performance testers

Technical Support• Code build & deploy• Environment setup and config• Components Responsible

Business• Acceptance testers

Product Owner

Integrationbacklog

Team is responsible for setting-up and maintaining the required test environments

Environments @ PPM:

Virtualized HWTest data

(Deploy when needed)

Production-like HWProduction data

(Deploy when needed)

Production HWProduction data

(Controlled deploy)

Virtualized HWTest data

(Daily deploy)

2- Team has the necessary resources: environments

3- Continuous Integration

Continuous Integration goals @ PPM:• After each commit

– Build– Run all unit tests– Create RPM– Deploy RPM in dev environments

• Each night– Build, run unit tests, create RPM– Deploy RPM in integration environment

Continuous Integration @ PPM

CVSSVNCVSSVN

CruiseControl& Maven

(+nexus +clover)

CruiseControl& Maven

(+nexus +clover)

Checkout, build,run unit tests,build RPMs

PuppetPuppet RPMrepoRPMrepo

Dev.env.

Server

Dev.env.

ServerDev.env.

Server

Dev.env.

ServerServerServer

RPM

Developer

poll

Dev.env.

Server

Dev.env.

ServerDev.env.

Server

Dev.env.

ServerServerServer

Devlopment Environment(mocked)

Integration Environment(un-mocked)

After every successfull build

Nightly

4- Automated Tests

To regression test as often as needed

Test goals @ PPM: Test the result of the integration (no-mocks)– Unit tests– Functional regression tests (automated & manual)– Performance benchmarking (automated)

Automated Tests @ PPM(model based testing)

SVNSVNTest

AutomationExpert Test definitions .xml

Test models & scripts

ExecutionsResults

(times, log files, screen capture)

ExecutionsResults

(times, log files, screen capture)

MBTMBT

Test Runner (perl)

2. Collect results

JMeterJMeterLoad-

RunnerLoad-

Runner

ShellShell

PerlPerl

Integration Environment(un-mocked)

Virtual machine

QTPQTPSeleniumSelenium

WindowsLinuxMac OS

Virtual machine

1. Run

API

Perf.

GUI

FirefoxIESafariOpera

5 - At least two test environments

Integration env. (nightly builds)Acceptance env. (stable)

Virtualized HWTest data

(Deploy when needed)

Virtualized HWTest data

(Daily deploy)

6 - Early performance tests

Create performance tests early to use as benchmarks

(Delivered by Development teams)

7 - Stop the line

Integration Team stops new developmentif integration is broken

8- Clear Contract between Dev & Integration Teams

Code deliverable (maven, RPM)Configuration info (Puppet scripts)Tested! (x% unit tests coverage)

Requirements & Test cases in QaTraqAutomated tests integrated in test framework

”Stop the line!”Trouble Reports

Running Tested Features

9 - Make the process visible

Integration processIntegration process

ContractContractcheckcheck

IntegrationIntegrationenv.env.

AcceptanceAcceptanceenv.env.

ProductionProductionenv.env.

Work to do to pull the features

through the process

Work to do to pull the features

through the process

Fea

ture

sF

eatu

res

10- Make Status visible and understandable

Integration status can be hard to grasp (”nothing works!”)

Green = Running Tested Feature

Features(Requirements)

Integration Environment(un-mocked)

Acceptance Criteria/Requirements

(from PO)

Test cases(checked by PO)

QaTraq

Automated

Integration tests

Automated

Integration tests

Report: how many features are passing their acceptance tests

Req.Test cases

Req.Test cases

Testers use QaTraq regs (tags)In Automated tests

Test results

Dashboard

Running Tested Features @ PPM

Manual Integration

tests

Manual Integration

tests

RunningTested

Features

10 Key success factors for the integration team @ PPM

1. Integrate early2. Team has the necessary resources3. Continuous integration4. Automated tests5. At least two test environments6. Early performance tests7. Stop the line8. Clear contract between Dev. and Integration teams9. Make the process visible10. Make the status visible (and understandable)

Next Steps

• Hunt waste, waiting time, etc. in the integration process– Optimize value flow

• Pull through the integration process instead of push– Kanban

• Release possible after every sprint (instead of a few times per year)!

christophe.achouiantz@avega.se