'You Say "Kvalitet", I Say Quality' by Colin Cherry

You say , I say Stories behind establishing Quality Benchmarks

© 2011 Capgemini. All rights reserved. 1

Colin CherryDirector, Testing ServicesCapgemini Australia

Content

The Background Story

The Story about National & Corporate Cultures

The Story about meeting the Players

The Story about Gathering the Evidence

The Story about Analysing the Risk

The Story about Root Cause Analysis

The Story about a new Belief System

The Story about achieving the Goal

The Story about Changed Perceptions

The Story about a Changed Reality

The Story about thanking my (new) friends


1. A finely balanced project involving a Swedish Transport Authority & a

major Australian Software provider in a strained relationship

2. A Steering Committee running out of patience & a CEO who wanted

one last chance to implement a new SMART transport ticketing system

3. A press highlighting every new delay

4. A project with more than 16,000 (fully documented) requirements that

had been running for over 7 years

5. A solution that had almost gone live twice before

6. A team of 5 (external) Senior Consultants from a leading Systems

Integrator, given 6 months to get the ticketing solution LIVE

7. A phone call from Sydney, a flight to Perth (followed by a slightly longer

one to Stockholm)

The Background Story


The Story about National & Corporate Cultures


1. Meet the Vendor, the Client, the Consultants & Identify the Key Players

2. Sort through the (last 7 years) information – what’s important & what’s not

3. Perform a SWAT followed by a Root Cause Analysis

4. Assess the gathered information

5. Develop a Plan of Attack

The Story about meeting the Players




July 15

Compare Vendor &

Customer Metrics



HPQC Structure

The Story about Analysing the Risk


July 29

Software Version 3.2.3



Triage

Process

Install

Process

Variance found &

confirmed by the client

Variance logged in HPQC;

Type =

Observation, Status =

New

Client Vendor Client

HPQC automatically

generates EMAIL to Vendor

DEV MGR (with enough

detail for Vendor assessment

& fix)

Vendor updates HPQC

with comments to accept or

reject Observation; Status

= Rejected or Open

If client disagrees with

rejection, Type = Bug

Status = Open

If Vendor has (existing) Bug

with same

symptoms, Vendor adds

comments to HPQC Bug

record

Rejected

Rejection overturned

Accepted

Vendor Internal Bug /

Fix Management

Process

Vendor fix tested

OK, Status = Vendor

Retest Successful +

HPQC generates EMAIL

(PC) to client Test

Manager

SL & Vendor conduct

daily Bug Status

reviews & update

HPQC accordingly

XX retest; if OK, Status =

XX Retest Successful;

else Status = Retest

Failed (HPQC generates

EMAIL PC to Vendor DEV

MGR)Retest

Failed

Triage Observation; assign

Critical & Major Bugs to

Vendor DEV MGR; Minor +

Trivial Bugs + Observations

stay at SL

Critical Major

Minor & Trivial Bugs

monitored internally until

released to Vendor.

Observations “watched”

Min

or

&T

riv

ialB

ug

or

Ob

serv

ati

on

Client sets Status to

Closed

Rejection

Confirmed



Trend: More Re-test Failures as we close in on LIVE Date

The Story about a new Belief System


Probability

6 Frequent The risk is likely to occur frequently. The risk

will be experienced hourly or more often.

5 Probable The risk will occur several times. The risk is

expected to occur at least once a week.

4 Occasional The risk is likely to occur several times. The

risk is likely to occur once a month.

3 Remote The risk is likely to occur sometime in the

lifetime of the product; maybe once per 10

years.

2 Improbable The risk is unlikely to occur but is possible;

maybe once per 100 years.

1 Incredible The risk is extremely unlikely to occur; maybe

once in a 1,000 years.

Consequence

4 Significant Immobilizing failure that prevents

a service being provided

3 Major A service failure that must be

rectified to ensure a significant

service outage does not occur

2 Minor A service failure that reduces the

effectiveness of a service

1 Insignificant A failure that does not impact the

provision of a service



Co

de

Fre

eze

Production Today

Release 3.2.3.1

Production Nov 15

Release “3.2.3.X”

STEP ZERO

Production Sept 1

Release 3.2.3.2Production Sept 15

Release 3.2.3.3

Acceptance Sept 1

Release 3.2.3.2

+ Patches

Acceptance Sept 15

Release “3.2.3.X-1”

Acceptance Oct 1

Release “3.2.3.X”

Acceptance Today

Release 3.2.3.2

Acceptance Nov 15

Release 3.2.3.X

Fin

al S

AT

Regre

ssio

n T

est

Consolidate Patches

into Single Release

The Story about Changed Perceptions


We have been LIVE for almost 1 week, but are still finding new Bugs

The Story about a Changed Reality


1. We realised that the Vendor & the Client were not comparing Apples with Apples

• The vendor was producing Release Notes, but not explaining to the customer how to

understand & use them (JIRA was the vendor Release Management tool)

• The customer wasn’t forceful enough in asking for the Release Notes to be provided in

a way that was usable (HPQC was the customer Test Management tool)

2. We improved visibility of what was really happening

• We got the Vendor & the Customer to agree on a single source of truth – HPQC

• We turned HPQC into a Release Management tool

3. We got the Vendor to send Testers to the Customer’s site & the Customer’s Test

Manager to visit the Vendor (for the first time)

4. We created a joint vision of what was required for Go Live, built a consolidated

Plan to achieve it & then executed the Plan

5. We provided the GLUE

The Story about Thanking my (new) Friends


1. The Consultants:

• Magnus Wikholm, Juraj Lesko, Martin Torebring, Maria Ander

2. The Customer:

• Lars Bromander, Anders Nillson, Thomas Enjin, Torkel Strahle, Milena Haykowska

3. The Vendor:

• Kim Fitzpatrick, Russell Ascott, Sonia Libao, Simon Morley, Brian Roberts, Jerico

Ubalde

Thank You…


Colin CherryDirector

Testing Services

Capgemini Australia Pty LtdLevel 2, 477 Collins Street

Melbourne VIC 3000 Australia

Tel: +61 3 9613 3138Mob: +61 412 214 240

[email protected]

mailto:[email protected]

'You Say "Kvalitet", I Say Quality' by Colin Cherry

Technology

Transcript of 'You Say "Kvalitet", I Say Quality' by Colin Cherry