Changing the Testing Conversation from Cost to Value

Why testing has become a commodity, and what you can do about it

Iain McCowatthttp://exploringuncertainty.com

iain@exploringuncertainty.com

@imccowatt

imccowatt

© Can Stock Photo Inc. / frenta

• What this presentation is

• What this presentation is not

Introduction

Project #1

• An opportunity

• Staffing decisions

• Things start to unravel

• Micromanagement, master/slave

• Focus on process, KPIs galore

• And ultimately…

Project #1

GAME OVER

• This wasn’t the first testing vendor

• Nor were they the last

• There will be more

Project #1

• Investopedia:

– When a product becomes indistinguishable from others like it and consumers buy on price alone, it becomes a commodity

Commoditization: Introduction

• More common in the enterprise, particularly in organizations with global buying power

• Endemic in the consulting industry

• May be less common amongst software vendors

Commoditization: Where?

• How to run fast: put one foot in front of the other, quickly

• Who here could win Olympic gold?

Commoditization: Origins

• A lot of people see testing like this:

Commoditization: Origins

Test Plan

Write scripts

Execute Cycle 1

Execute Cycle 2

Execute Cycle 3

Sign off

Commoditization: Dynamics

Perceivedas low skill

Perceivedas commodity

Supply ofcheap testing

Demand forcheap testing

Demand forskilled testers

Supply ofskilled testers

Collaboration

Testingfailures

Negative effectChoice of effect

Commoditization: Dynamics

Vendor B competes on value

Vendor B competes on price

Vendor A competes on value

Vendor A competes on price

Both stand to win long term relationshipsCustomers get better testing

Vendor A: grows market shareVendor B: loses market share

Vendor A: loses market shareVendor B: grows market share

Race to the bottom: short term revenue, short term relationships Everyone loses

• Master/Slave, chilling effects on collaboration

• Information starvation

• Fungibility and economic incentive to juniorize

• Focus on control, process and method

• Demand for skill suppressed

• …and ultimately the projects suffer

Commoditization: Consequences

Project #2

• What’s the problem?

• Your competition can do it cheaper

• No budget!

• If IBM said it, it must be true

• Let’s give it a try

• That was a leap of faith, but…

Project #2

A Happy Client

• A second year, a second release

• A growing portfolio

Project #2

• This was not a commodity testing service.

• But what was different?

Spot the Difference

http://freear.org.uk/nick

• People

– Passion

– Skill

– Creativity

• Relationships and trust

• New ideas, new technologies

• Location

Sources of Differentiation

• Process

• Methodology, branded or otherwise

• Templates and other “accelerators”

• Tools and technology, unless new and unique

• ISO, CMMI etc.

• Just about anything easily replicated or at home in buzzword bingo

NOT Sources of Differentiation

Project #3

• Passion speaks volumes

• The managed service misnomer

• Early conversations, aspects of quality

• Where’s the code?

• Tests as experiments

• Just getting started, but…

Project #3

The Future Looks Bright

• Taking a very different approach predecessor

• Building on a solid relationships

• And pricing? Part of the conversation, but not defining it

Project #3

• We focus on software quality but rarely discuss the quality of testing

• Perhaps a quality model might help?

• This model formed part of the conversation on Project #3…

Reframing the Conversation

A Model of Testing Quality

What can you do?

• Observe vendor behavior: are they interested in you and your problem?

• Do they understand your problem?

• Do they care?

• Do they have any ideas?

• Do they have the skills to deliver?

Customer

• Dare to be different

• Sack the salesmen, consult

• Be prepared to say “No”:

– If they want it cheap, let them go somewhere else

– Don’t follow anyone off a ledge

• Education, education, education

• Shared values => relationship => business worth doing

Vendor

• Empowerment starts with YOU

• Don’t wait to be given a learning agenda

• Learn how to learn

• Learn anything and everything that might help you

• Select, remix, invent

Tester

• Commodity testing isn’t going anywhere: both demand and supply will remain

• Some of us (vendors) are successfully exploiting this to build differentiated services

• This shows there is a demand for something better

• Skilled testers can find fulfilling roles

A Final Word

Questions?