Huib%Schoots%(codecentric)% … testing - Huib Schoots... · Boost your testing power with...

36
Boost your testing power with Exploratory Testing! Exploration Huib Schoots (codecentric) Dutch Tes4ng Day 8 2012

Transcript of Huib%Schoots%(codecentric)% … testing - Huib Schoots... · Boost your testing power with...

Boost your testing power with Exploratory Testing! Exploration

Huib%Schoots%(codecentric)%

Dutch%Tes4ng%Day%8%2012%

•  What%is%exploratory%tes/ng?%

•  Strategy%and%techniques%of%ET%•  SBTM:%plan,%structure%&%traceability%

•  How%more%explora/on%can%boost%the%value%of%your%tes/ng%

•  How%to%master%exploratory%tes/ng?%

“a%style%of%soFware%tes/ng%that%%

emphasizes%the%personal%freedom%%

and%responsibility%of%the%individual%%

tester%to%con/nually%op/mize%the%quality%of%his/

her%work%by%trea/ng%testJrelated%learning,%test%

design,%test%execu/on,%and%test%result%

interpreta/on%as%mutually%suppor/ve%ac/vi/es%

that%run%in%parallel%throughout%the%project”%

Exploratory%Tes/ng%

Exploratory%Tes/ng%

“An%approach%to%soFware%tes/ng%that%emphasizes%the%personal%freedom%and%responsibility%of%each%tester%to%con/nually%op/mize%the%value%of%his%work%by%trea/ng%learning,%test%design%and%test%execu/on%as%mutually%suppor/ve%ac/vi/es%that%run%in%parallel%throughout%the%project."%

"Simultaneously%designing%and%execu/ng%tests%to%learn%about%the%system,%using%your%insights%from%the%last%experiment%to%inform%the%next."%

Source:(�Telling(Your(Exploratory(Story(by(Jon(Bach(

%

Freedom%

Pre8scrip4on%

pure%scripted%

vague%scripts%

fragmentary%test%

case%(scenarios)%

charters% roles%

freestyle%%

exploratory%

Fairy%tales%Do%not%apply%ET%if:%

• Higher%demands%on%demonstrability%/%repor/ng%%(eg.%imposed%standards)%

• Cri/cal%func/onality%(failure%=%big%damage)%

•  Inexperienced%testers%• Test%cases%must%be%able%to%be%executed%by%a%different%tester%

• Test%cases%have%to%be%reJusable%• No%direct%feedback%from%test%execu/on%(the%results%are%not%immediately%available)%

• Tests%require%a%lot%of%prepara/on%• Tes/ng%needs%to%be%on%the%cri/cal%path%as%short%as%possible%%

Source:(TMap(Next,(chapter(14(

%

Fairy%tales%

Source:(Fan(booklet(Exploratory(tesBng((MicrosoD)(

Fairy%tales%

•  What%is%exploratory%tes/ng?%

•  Strategy%and%techniques%of%ET%•  SBTM:%plan,%structure%&%traceability%

•  How%more%explora/on%can%boost%the%value%of%your%tes/ng%

•  How%to%master%exploratory%tes/ng?%

Source(original(picture:(Two(futures(of(tesBng(by(Michael(Bolton%

execution

Exploration: searching for value and risk

discovery

investigation

learning reporting

design

Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission

Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission

Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission

Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission

Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission

Stories%

Product%Story:%%

How%the%product%works%and%how%it%fails...%

%

Tes4ng%Story:%%

What%we%have%tested%and%how,%what%we%do%not%

test%and%how%good%our%tes4ng%was...%

•  What%is%exploratory%tes/ng?%

•  Strategy%and%techniques%of%ET%•  SBTM:%plan,%structure%&%traceability%

•  How%more%explora/on%can%boost%the%value%of%your%tes/ng%

•  How%to%master%exploratory%tes/ng?%

Daily%Test%Sessions%

Project%

%Strategy%

General%

Ongoing%

Planning%

Low%Tech%Dashboard%

Test%

Charters%

(weekly)%

planning%

session%

Wrap8up%/%

Debrief%

Test%

Session%

Issues% Bugs%Risk%Catalog%Heuris4cs% Test%Ideas%

Risks%Test%Plan% Coverage%

Test%Log%

Wrap8up%/%

Debrief%

Test%

Session%

Test%Log%

Wrap8up%/%

Debrief%

Test%

Session%

Test%Log%

Session%

Sheets%

Source:%Ruud%Cox,%Tes/ng%in%a%Medical%Device%Context,%EuroStar%2012%

Daily%Test%Sessions%

Project%

%Strategy%

General%

Ongoing%

Planning%

Low%Tech%Dashboard%

Test%

Charters%

(weekly)%

planning%

session%

Wrap8up%/%

Debrief%

Test%

Session%

Issues% Bugs%Risk%Catalog%Heuris4cs% Test%Ideas%

Risks%Test%Plan% Coverage%

Test%Log%

Wrap8up%/%

Debrief%

Test%

Session%

Test%Log%

Wrap8up%/%

Debrief%

Test%

Session%

Test%Log%

Session%

Sheets%

Test%Charters%

Explore (target) With (resources)

To discover (information)

The%charter(is(a(oneH(to(threeHsentence(mission(for(a(tesBng(session%

My mission is to test "<insert risk here> to "

<insert coverage here>

Explore Application X import menu. Identify important features with the goal of

developing a coverage outline and a risk list.

Read Chapter 4 of the product specification. Prepare a mind

map, and discuss it with Peter (programmer) and David

(architect).

Daily%Test%Sessions%

Project%

%Strategy%

General%

Ongoing%

Planning%

Low%Tech%Dashboard%

Test%

Charters%

(weekly)%

planning%

session%

Wrap8up%/%

Debrief%

Test%

Session%

Issues% Bugs%Risk%Catalog%Heuris4cs% Test%Ideas%

Risks%Test%Plan% Coverage%

Test%Log%

Wrap8up%/%

Debrief%

Test%

Session%

Test%Log%

Wrap8up%/%

Debrief%

Test%

Session%

Test%Log%

Session%

Sheets%

Test%log:%my%notes%

Prepara4on:%talk%to%product%owner%Test%Strategy%&%notes%on%a%bugs%

Issue/ques4on%&%coverage%outline%%Debrief%using%PROOF%

Session%sheet%

• Charter%• Coverage%areas%(product%areas,%product%elements,%quality%criteria%or%test%techniques)%

• Start%Time%+%Session%dura/on%(long,%normal,%or%short)%

• Tester%Name(s)%• Charter/opportunity%/me%,%expressed%as%a%/me%breakdown:%%

– Test%design%and%execu/on%(as%a%percentage%of%the%total%onJcharter%/me)%– Bug%inves/ga/on%and%repor/ng%(as%a%percentage%of%the%total%onJcharter%/me)%– Session%setup%(as%a%percentage%of%the%total%onJcharter%/me)%

• Data%Files%• Test%Log%(notes%the%tester%made%during%tes/ng)%

• Bugs%(a%problem%that%threatens%the%value%of%the%product)%

•  Issues%(a%problem%that%threatens%the%value%of%the%tes/ng%process)%%

Source:(An(Exploratory(Tester’s(Notebook(by(Michael(Bolton%

•  What%is%exploratory%tes/ng?%

•  Strategy%and%techniques%of%ET%•  SBTM:%plan,%structure%&%traceability%

•  How%more%explora/on%can%boost%the%value%of%your%tes/ng%

•  How%to%master%exploratory%tes/ng?%

More%explora/on%can%boost%%the%value%of%tes/ng%

•  Do%what%really%needs%to%be%done%•  Create%engagement:%help%people%use%their%brains%

•  Take%advantage%of%tacit%knowledge%and%skill%•  Use%insights%from%experiments%to%inform%the%next%

•  Beware%of%the%crea/ve%power%of%explora/on%%

•  What%is%exploratory%tes/ng?%

•  Strategy%and%techniques%of%ET%•  SBTM:%plan,%structure%&%traceability%

•  How%more%explora/on%can%boost%the%value%of%your%tes/ng%

•  How%to%master%exploratory%tes/ng?%

Challenges?%

Exploratory%Tes4ng%=%not%easy%and%

needs%(a%lot)%training%and%prac4ce%

%OFen%seen%as%most%difficult:%%

– Note%taking%– Test%ideas%– Coverage%repor/ng%– Managing%ET%

%

•  Just%do%it!%•  Prac/ce,%prac/ce%and%prac/ce%some%more…%•  Pair%with%colleagues%•  Tes/ng%Dojo%•  Debrief%&%retrospect%•  Train%crea/ng%test%ideas%fast%•  Learn%more%about%observa/on,%experiments,%biases,%social%science,%....%

How%to%master%exploratory%tes/ng?%

Source:(GiMe(OMosen(–(Many(flavours(of(exploratory(tesBng((Agile(TesBng(days(2012)%

hfp://trishkhoo.com/%

My%personal%experience%

Image:%hfp://corkboard.me%

“Testing is about questioning & learning under conditions of fundamental uncertainty.”

RST “I've participated and organized many testing competitions. I've never seen someone win such a competition by writing down tests. Doesn't that tell us something?”

James Bach

“A tester knows that things can be different”

Jerry Weinberg

If you cannot trust your testers, you do not make them write more detailed test case. But you train them!

Rikard Edgren – EuroStar 2012 Gitte Ottosen – ATD 2012

More%informa/on%

hXp://www.huibschoots.nl/links%

[email protected]%

+31%6%8%24641033%

@huibschoots%

www.magnifiant.com%