Test Design Made Easy (and Fun!)

Rik Marselis 5 November 2015

Let’s walk the testing talk

Question 1

Applying Test design techniques is

easy

Question 2

Applying Test design techniques is

fun

Question 3

How many Test Design Techniques exist?

Test design techniques & coverage typesTMap NEXT• Equivalence classes• Boundary value analysis• Decision points (CC, DC, CDC, MCDC,

MCC)• Orthogonal arrays• Pairwise testing• Data cycle test (CRUD)• Statistical usage (op.prof.• Right paths / Fault paths• Checklist• Decision table• Data Combination Test• Elementary comparison tst• Process Cycle test• Real-life test• Semantic test• Syntactic test• Use case test• Error Guessing• Exploratory testing• Reviews• Walkthroughs• Inspections

ISTQB• Equivalence partitioning• Boundary value analysis• Decision table testing• Cause Effect graphing• State transition testing• Use case testing• User story testing• Structure based (SC, CC, DC, CDC,

MCDC, MCC)• Error Guessing• Exploratory testing• Orthogonal arrays• Pairwise testing• Domain analysis• Defect based techniques• Checklist• Combinatorial / Class. tree• Informal review• Technical review• Walkthrough• Inspection• Static analysis

Torbjorn Ryber• Exploratory testing

(this is an approach, not a technique)• Inspection• Walkthrough• Technical review• Informal review• Modelling• Equivalence partitions• Boundary values• Domain tests• Business process testing• Use case testing• State based testing• Decision tables & trees• Elementary comparison• Combinatorial testing• Data cycle testing• Syntax testing• Time cycle testing• Program Logic (SC, BC, PC)• Data flow testing

Boris Beizer• Control-flow testing

• Loop testing

• Data-flow testing

• Transaction-flow testing

• Domain testing

• Syntax testing

• Finite-state testing

And more…

Question 4

How many Test Design Techniques do you apply in your day-to-day work??

Question 5:

Which test design techniques do you use in yourday-to-day work?

TMap HD test approaches

Why coverage?

What’s your testing challenge?Select from 4 groups of Coverage types

Better know & useone techniqueof each group

thanall techniques

of only one group

Why experience based?

So it needs a skilled tester !

Use results of previous tests to create new tests

Always use a mix

What’s your testing challenge?Select an experience based approach

Largely intuitive & ad hoc

Structured, chartered

Experience brought together

Checklist: Test Heuristics Cheat Sheet

• Abstract (remove details)• Never and Always• Interrupt• Reverse• Some, none, all• Starve• Too few, too many

Source: “Explore It!” by Elisabeth Hendrickson, appendix 2

• Useful approximations• Violate data format rules• Zero, one, many• Zoom• Back, forward, history• Bookmark it

More checklists from Explore it!

Numbers:

• Zero

• One

• Many

• Too many

• Too little

Selecting:

• None

• One

• Some

• Many

• All

Timing:

• Allways

• Sometimes

• Never

CRUD:

• Create

• Read

• Update

• Delete

Source: “Explore It!” by Elisabeth Hendrickson

What is the goal of testing?

• To report about quality and risks

• To build confidence

• So you need to log and report

Example of a test log

How to mix test approaches

• Create an Exploratory Testing charter(this can be done long before the test object is available)

• Use coverage based techniques as “test ideas”(thus your start-set of test cases can be made long before …)

• Execute your test based on prepared test cases complemented with exploratory testing

Charter

Explore <target>

With <resources>

To discover <information>

simple

Extensive:

Source: “Explore It!” by Elisabeth Hendrickson, page 13

Use a tool to get a start

Source: the test design tool “COVER” of Sogeti

Calculate Public Transport tariff

• Based onage, profession and discount card the tariff forpublic transport is calculated.

SPRINT 1: Calculate Transport tariff

• What kind of testing challenge is this?

Assignment:Test the public transport tariff decision

• Think of some test idea’s

• Use the DCT test cases as a starting point

• Execute the test

• Use the logsheetGroups of 2-3

20 minutes

Retrospective

• What is your conclusion about quality & risks?

• How much confidence do you have?

• Have you found defects?

Coffee break

What is the goal?

• With preparing test ideas and some test cases;You have a good start for exploring

– Most people get better coverage than with purely experiencebased test cases

– Proof of coverage at the spots where you need it

– Faster progress of testing compared to only exploring

– It fuels you imagination for the real exploration

Workshop: The Triangle exercise

In his 1979 book “The art of software testing” Glenford J. Myers starts with a simple test to see if you are suited as a tester.

The challenge is to test an application thatgets 3 integers as input which represent the lengths of the sides of a triangle. Then the program tells what type of triangle it is.

So we start thinking…

• What is/are our challenge(s)?

• What possible inputs?

• What possible outputs?

• What’s the process?

• How can we test this?

• How can we be efficient?

Process for triangle application

Input check

start

A = B ?

A = C ?

A = C ? B = C ?

A + B < C ORA + C < B ORB + C < A ?

Input numeric

?

Not a triangle

yes

yes

yesyesyes

yes

no

no

no nono

no

Inspired by Glenford J. Myers

IsoscelesEquilateral Scalene Not a triangle

Exercise 2a: Create a charter

• Include at least 3 test design techniques

Explore <target>With <resources>To discover <information>

Example of an extensive charter

Source: “Explore It!” by Elisabeth Hendrickson, page 13

Example of a simple charter

Groups of 2-35 minutes

Use MCDC for the complex decision

FALSE (0)(scalene triangle)

X Y Z

TRUE (1)(not a triangle)

X Y Z

(X) A + B < C

(Y) A + C < B

1 . .

. 1 .

. . 1

0 . .

. 0 .

. . 0 (Z) B + C < A

0 0

0 0

0 0

0 0

0 0

0 0

D2.1

D2.2

D2.3

D2.4

Result = Condition X OR Condition Y OR Condition Z

Neutral value (AND = 1, OR = 0 )0 0

MCDC =ModifiedConditionDecisionCoverage

Exercise 2b: Execute your charter

Test the triangle application using:

• Your charter

• Your test ideas and tools

• The logsheet

Groups of 2-320 minutes Note:

application and handouts will be provided

Retrospective

• What is your conclusion about quality & risks?

• How much confidence do you have?

• Have you found defects?

• How many of the tests that Glenford Myers has listed, did you do?

Tooling: Of course !But beware…

Source: the test design tool “COVER” of Sogeti

Conclusion

• Exploratory testing can be very well combined with test design techniques

• Exploratory tests can be prepared before test executionstarts

• Combining scripted and exploratory tests can be botheasy and fun!!

The end. Thank you !!

Happy testing

This whole presentation copyright © Sogeti 2015

The next slides contain handoutsand background information.

In this workshop Rik will make you see the beauty of test design by

making it easy and fun. Therefore this workshop is all about actively

testing software and learning while doing so.

The first step in making testing more easy is to determine the type of challenge you are facing and based on that select a proper approach. Early test design alone is not enough to really gain confidence, it always must be complemented with experience based testing. So in this workshop you will do both coverage based testing and exploratory testing and you will experience that proper testing has the following components:

Examine the type of challenge you are facing

Select the proper approach consisting of a mix of coverage based testing and experience based testing

Prepare both coverage based testing (by making test cases) and experience based testing (by making charters)

Execute your tests

Report the results (in a debriefing, supported by some sort of document)

Join Rik in this (inter)active workshop and be inspired to make better use of test design as a means to establish confidence in the system under test and to really enjoy the testing profession.

We’ll be actively testing, so bring your laptop or tablet along.

Logging testcasesNr. Age Soldier Discount

cardExpectedresult

Actualresult

Observations, remarks

1

2

3

4

5

6

7

8

9

10

Use the back of this page if needed

Logging testcasesNr. Side 1 Side 2 Side 3 Expected

resultActualresult

Observations, remarks

1

2

3

4

5

6

7

8

9

10

Use the back of this page if needed

Use MCDC for the complex decision

FALSE (0)(scalene triangle)

X Y Z

TRUE (1)(not a triangle)

X Y Z

(X) A + B < C

(Y) A + C < B

1 . .

. 1 .

. . 1

0 . .

. 0 .

. . 0 (Z) B + C < A

Result = Condition X OR Condition Y OR Condition Z

Neutral value (AND = 1, OR = 0 )0 0