The Future of Testing

The Future of Testing in Finance

Paul [email protected]: @paul_gerrardWeb: gerrardconsulting.com

Download from businessstorymethod.com/Intelligent Testing, Improvement and

AssuranceSlide 1

mailto:[email protected]

https://twitter.com/


http://gerrardconsulting.com/

http://businessstorymethod.com/

http://testaxioms.com/?q=node/25

Thank-you

• Thank-you to Dr Richard Sykes and Susan Windsor for allowing me to pick their brains

• This PowerPoint can be downloaded from:

gerrardconsulting.comIntelligent Testing, Improvement and Assurance

Slide 3

There have been some negative waves recently• Testing is Dead• Death to the Testing Phase• Testing has contributed little to

quality improvement in the last ten years

• Being a tester is a ‘bad thing’

• Is testing really DEAD?

Intelligent Testing, Improvement and Assurance

Slide 4

My crystal ball

• Changes in our business demand a rethink of testing

• Ten five-year predictions

• Some test lead/manager career suggestions

• It’s personal, partial and fallible!


Slide 5

“You can fool some of the people all the time and you can fool all the

people some of the time but you can’t fool all the people all the time.”

Abraham Lincoln

“If you are right, you’ll be hailed as a visionary.

If you are wrong, no one will remember”Bill Hetzel

“How did I get myself into this?”Me


Slide 6

Theme:Changes in our

business demand a re-think of where

testing fits in systems projects

Software product and service companies and user companies are all

affected


Slide 7

Technology, Hype and How

Innovations Become

Mainstream


Slide 8


Slide 9

Not all innovations make it across the “chasm”


Slide 10

The Hype Cycle


Slide 11

Agile, The Cloud and Mobile computing have

made it across the ChasmWhere are YOU in the

Adoption Cycle?

http://en.wikipedia.org/wiki/Image:Gartner_Hype_Cycle.svg

The Cloud


Slide 12

Software as a Service dominates the growth of public cloud markets


Slide 13

http://www.ricespoonandfork.com/wp-content/uploads/2011/04/CloudComputingMktin2020.png

The industry changes its mind – again• Moving to Infrastructure as a Service

will save some money• The IT Director can glory in the

permanent cost savings for a year – and then what?

• The “Business” will want to take advantage of the flexibility that the move to the cloud offers.


Slide 14

Massive re-architecting of systems and companies – work in progress• Many (most?) public websites will be

replaced by mobile apps – depends on who you talk to

• SaaS will represent 26% of all packaged software (so what?)– To re-architect an installed base of software

and create new systems will be a monumental effort

– The challenges of integration, security, performance and reliability will be daunting.http://forrester.com/rb/Research/sizing_cloud/q/id/58161/t/2


Slide 15

http://forrester.com/rb/Research/sizing_cloud/q/id/58161/t/2



The Effect on Suppliers


Slide 16

Out with the old…

• “Old model” hardware suppliers–Motivated to sell more hardware; not

motivated to utilise; requires high labour support

–Motivated to sell new licences (e.g. operating systems); not motivated to sell SaaS; open source.

– Commercial model and company culture relies on this mindset

• Can they change quickly enough? Or at all?• Where are their new markets?


Slide 17

In with the new?

• New model cloud suppliers– Motivated to sell infrastructure as a service– Have tools in place to manage set-up and tear down

with very little manual intervention; very low labour costs

– Rely on high capacity, utilisation, open source– Motivated to sell flexibility

• Small number of very highly skilled people• A cheaper business model, they can reduce

current costs by 75-90% and still make a profit• Expect prices to fall as competition hots up, but

can the oldies, being late to market, compete?


Slide 18

“Bottoms on seats” service models• Traditional IT service suppliers -

applies to all IT, not just testing• Operational costs are HIGH (people)• Low technology – high human

support• No incentive to be efficient• High utilisation equals high profits• Moving towards …


Slide 19

Brains, not bottoms

• High technology – fewer resources• Moore’s law applies – smart people doing more

with less• Companies are used to business process

outsourcing• Development and test is becoming a

commodity supported by lots of tools• Companies may outsource to Service Factories• Their incentive is always to be efficient• Niche specialist skills always in demand.


Slide 20

Impact on selecting and managing supplier relationships?• Even our contracts and purchasing

processes have to change• Testing has an important role to play

of course.


Slide 21

A Methodological Shift

From staged, structured projects to iterative and Agile and now, towards ‘continuous delivery’


Slide 22

Changes ahead

• System users and service vendors moving to new business models based on new architectures

• SaaS is an enabler for very rapid deployment of new functionality onto a range of devices

• SaaS and maturing Agile approaches mean the promise of Continuous Delivery can be achieved.


Slide 23

And Continuous Delivery is...

From requirement to deployment in 24 hours…

every 24 hours?


Slide 24

From traditional delivery…

… to Continuous Delivery


Slide 25

The Deployment Pipeline


Slide 26

• Automated Unit tests

• Automated Acceptance Tests

• Manual User Tests

Structured … Agile?We are in transition• A re-think is in progress• Adopting new architectures and a more

streamlined approach to development• Requirements half-life of weeks or months• Agile approaches are homing in on continuous

delivery• Kanban-Ichiwichi-Hokikoki-Kamikaze-Driven

Development – Whatever!• A hybrid Agile-Structured process:

– Structured Analysis, Agile Delivery?– That would be SAAD!


Slide 27

• Re-distributed functional testing• A focus on continuous test and assurance• Service integration replaces system

integration• Non-functional skills, design and test

particularly UX, security, privacy, performance are in demand (‘twas always so)

• The ability to balance competing non-functional requirements in demand too.

What does this mean for practitioners?


Slide 28

The real squeeze on testing

• From the ‘low-value’ unskilled, downstream test role to upstream, business-savvy, workflow-oriented, UX-aware testing specialists with new tools– Developers absorb the checking – with

tools– Business analysis owns acceptance– New focus on requirements management– Under-skilled testers, relying on clerical

skills to get by will be squeezed out.Intelligent Testing, Improvement and

AssuranceSlide 29

A Redistribution of Testing and not its death - is inevitable.

Board Level Messages

Is testing a Board-Level concern?


Slide 30

The New Testing: A Wider Resource Orchestration• The creation & delivery of systems with

assurance in this new environment calls for new approaches to development and testing

• Assurance is required across the full life cycle, in an increasingly virtualised environment

• This requires orchestration of people, process & technology throughout the lifecycle increasingly in the virtual.

Testing Through the

LifecycleIntelligent Testing, Improvement and

AssuranceSlide 31

The assurance of outcomes, not just deliverables• Testing assures the quality of deliverables, as

ever• But stakeholders commission systems to

achieve business outcomes

• Confidence in the delivery of outcomes is, for a business, an operational and strategic strength

• Testing should, above all, be about delivering confidence in outcomes.

We Test Systems, not

Just Software


Slide 32

My Five-Year Predictions


Slide 33

#150% of in-house testers will be reassigned or outsourced


Slide 34

#2Business test planning will

become part of up-front analysis and design (where it

should be)Intelligent Testing, Improvement and

AssuranceSlide 35

#3Specification by

Example will become the new

buzzword (includes ATDD, BDD, TDD

etc.)Intelligent Testing, Improvement and

AssuranceSlide 36

#4Developers will

adopt behaviour‐driven

development and new tools


Slide 37

#5Some system tests

and most acceptance tests will be business model-driven


Slide 38

#6Business models plus stories will

increasingly become

‘contractual’Intelligent Testing, Improvement and

AssuranceSlide 39

#7System Tests will

be generated from stories or will be

outsourced


Slide 40

#8Manual system test

execution will be outsourced (and in

the cloud)


Slide 41

#9Maybe half of all acceptance tests will be automated

in CI……foreverIntelligent Testing, Improvement and

AssuranceSlide 42

#10New tools that

manage the full requirements,

stories, scenarios, workflows, testing

pipeline will emergeIntelligent Testing, Improvement and

AssuranceSlide 43

Some Doors Close, but Other Doors

are Opening

What are the new opportunities for Test Leads

and Managers?


Slide 44

#1Upstream with

stakeholders and analysis – providing

testing and assurance skills to

businessIntelligent Testing, Improvement and

AssuranceSlide 45

#2Managing Business

Requirements knowledge;

analysing business impact of change


Slide 46

#3Testmaster: Providing an

assurance function to teams, projects

and business stakeholders


Slide 47

#4Managing the

information flows to/from the Continuous

Integration processIntelligent Testing, Improvement and

AssuranceSlide 48

#5Managing/monitoring

outsourced/offshore teams who detail and execute tests


Slide 49

Close


Slide 50

The phase after development is REWORK, not Testing• Managers will never name this phase

correctly• Testing must align with development, not

follow it• Manual test phase will fade away replaced by:– Up-front test design mostly automated– Integrated with code change and software delivery

• Testers must embrace the change, not resist it• Make that leap across the chasm!


Slide 51

Perhaps testing and the role of

testers can at last mature to what they should be?


Slide 52

Thank-You


Slide 53

The Future of Testing

Paul [email protected]: @paul_gerrardWeb: gerrardconsulting.com

Download from businessstorymethod.com/Intelligent Testing, Improvement and

AssuranceSlide 54

mailto:[email protected]



http://gerrardconsulting.com/

http://businessstorymethod.com/

http://testaxioms.com/?q=node/25

The Future of Testing

Business

Transcript of The Future of Testing