SAST 2002-09-19 Managing ET

download SAST 2002-09-19 Managing ET

of 55

  • date post

    06-Apr-2018
  • Category

    Documents

  • view

    215
  • download

    0

Embed Size (px)

Transcript of SAST 2002-09-19 Managing ET

  • 8/3/2019 SAST 2002-09-19 Managing ET

    1/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 1

    Managing Exploratory Testing

    Parts of this presentation have been excerpted from:Rapid Software Testing, by James Bach,[email protected]

    Black Box Software Testing, by Cem Kaner, [email protected]

    High Speed WEB Testing, by Jarle Vga,[email protected]

    Stle Amland, Amland Consulting,

    Hulda Garborgsv. 2, 4020 STAVANGER, NORWAY

    Mob:+47 905 28 930, Phone: +47 51580587, Fax: +4751585524

    e-mail: [email protected] WEB: www.amland.no

    SAST meeting September 19, 2002, Stockholm, Sweden

    mailto:[email protected]:[email protected]:[email protected]:[email protected]://www.amland.no/http://www.amland.no/mailto:[email protected]:[email protected]:[email protected]:[email protected]
  • 8/3/2019 SAST 2002-09-19 Managing ET

    2/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 2

    Software Testing:

    There is artistry. There is craft. There isscience. It is all threeas with all

    technologies.

    If you understand the science and havemastered the craft, then it is meaningful toinvestigate, espouse, and perfect the art.

    Boris Beizer

  • 8/3/2019 SAST 2002-09-19 Managing ET

    3/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 3

    Software Testing:

    Testing is the process of executing a programwith the intent of finding errors.

    Testing is an extremely creative andintellectually challenging task.

    Glenford Myers inThe Art of Software Testing

  • 8/3/2019 SAST 2002-09-19 Managing ET

    4/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 4

    Presentation Outline

    What is Exploratory Testing?

    How to do Exploratory Testing?

    How to Manage Exploratory Testing?

    High Speed WEB Testing Case Study

  • 8/3/2019 SAST 2002-09-19 Managing ET

    5/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 5

    Different testing approachesSkeptical approaches

    Analytical Approaches

    Information-driven approaches

    Time-honored but less effective approaches

    Experiential and intuitive approaches:Lets think blue-sky, speculate and follow our intuition.

    We have good hunches about where the bugs are lurking.

    Lets jump in an explore the systems behavior hands-on, sowe can decide how to test it.

    Lets find the important bugs fast, and worry about the testpaperwork later.

    Ross Collard (2002)

  • 8/3/2019 SAST 2002-09-19 Managing ET

    6/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 6

    What is Exploratory Testing?"Exploratory testing involves simultaneously learning,

    planning, running tests, and reporting / troubleshootingresults."

    Dr. Cem Kaner (2001)

    "Exploratory testing is an interactive process of concurrentproduct exploration, test design and test execution.

    To the extent that the next test we do is influenced bythe result of the last test we did, we are doing

    exploratory testing.

    James Bach, Satisfice (2001)

    http://www.kaner.com/
  • 8/3/2019 SAST 2002-09-19 Managing ET

    7/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 7

    Said about eXtreme Programming

    Agile software development is not conventional software developmentdone more quickly or done on tippie-toe. Agile software development issoftware done differently.

    Ron Jeffries, (e-mail on agile-testing list, April 24, 2002)

    proven (no single technique is new)application oriented

    planned and disciplined

    controllable and reliable

    risk minimizingTwo sides of extreme programming:

    for the developer: freedom, flexibility, fun

    for the manager: controllability, reliability, high qualityMartin Lippert (University of Hamburg), ICSTEST 2002

  • 8/3/2019 SAST 2002-09-19 Managing ET

    8/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 8

    The eXtreme Programmingand Exploratory Testing Analogy:

    Agile software testing is not conventional (scripted) software testingdone more quickly or done on tippie-toe.

    Exploratory Testing:proven (no single technique is new)

    application oriented

    planned and disciplined

    controllable and reliable

    risk minimizingTwo sides of Exploratory Testing:

    for the tester: freedom, flexibility, fun

    for the manager: controllability, reliability, high quality

  • 8/3/2019 SAST 2002-09-19 Managing ET

    9/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 9

    ET vs. Scripted Testing

    Fully

    Scripted

    Testing

    Ad-hoc

    Testing

    AutomatedTests

    Bug

    Hunting

    Exploratory Testing

    Jarle Vga (2002)

  • 8/3/2019 SAST 2002-09-19 Managing ET

    10/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 10

    What is Scripted Testing?Small (but realistic) example:

    How to script and test this login?(Functional tests only not security!)

  • 8/3/2019 SAST 2002-09-19 Managing ET

    11/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 11

    When to use Exploratory Testing? (1)There is little or no specifications and /or requirements

    You have little or no domain knowledgeYou dont have time to specify, scriptand test

    Uncertainty and Time Pressure!

  • 8/3/2019 SAST 2002-09-19 Managing ET

    12/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 12

    When to use Exploratory Testing? (2)A common goal of exploration is to probeforweakareasof the program.

    Test teams resource consumption per week:25% of the groups time developing new tests

    50% executing old tests (including bugregression)

    25% on exploratory testing

    Cem Kaner (2001a)

  • 8/3/2019 SAST 2002-09-19 Managing ET

    13/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 13

    Testing is done in Context1. The value of any practice depends on its context.2. There are good practices in context, but there

    are no best practices.

    3. People, working together, are the most importantpart of any project's context.

    4. Projects unfold over time in ways that are often notpredictable.

    5. The product is a solution. If the problem isn't solved, theproduct doesn't work.

    6. Good software testing is a challenging intellectual process.7. Only through judgment and skill, exercised

    cooperatively throughout the entire project,are we able to do the right things at the righttimes to effectively test our products.

    http://www.context-driven-testing.com/

  • 8/3/2019 SAST 2002-09-19 Managing ET

    14/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 14

    Presentation Outline

    What is Exploratory Testing?

    How to do Exploratory Testing?

    How to Manage Exploratory Testing?

    High Speed WEB Testing Case Study

  • 8/3/2019 SAST 2002-09-19 Managing ET

    15/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 15

    Doing Exploratory TestingKeep your mission clearly in mind.

    Keep notes that help you report what you did,

    why you did it, and support your assessmentof product quality.

    Keep track of questions and issues raised inyour exploration.

    To supercharge your testing, pair upwith another tester and test the samething on the same computer at thesame time.

    From Rapid Software Testing, copyright 1996-2002 James Bach

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

  • 8/3/2019 SAST 2002-09-19 Managing ET

    16/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 16

    Charter: A clear mission for the sessionA charter may suggest what should be tested, how itshould be tested, and what problems to look for.

    A charter is not meant to be a detailed plan.

    General charters may be necessary at first:Analyze the Insert Picture function

    Specific charters provide better focus, but take moreeffort to design:

    Test clip art insertion. Focus on stress and flow techniques,and make sure to insert into a variety of documents. Wereconcerned about resource leaks or anything else that mightdegrade performance over time.

    From Rapid Software Testing, copyright 1996-2002 James Bach

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

  • 8/3/2019 SAST 2002-09-19 Managing ET

    17/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 17

    Taking NotesTest Coverage Outline/Matrix

    Evaluation Notes

    Risk/Strategy ListTest Execution Log

    Issues, Questions & AnomaliesIt would be easier to test if you changed/added

    How does work?Is this important to test? How should I test it?

    I saw something strange

    From Rapid Software Testing, copyright 1996-2002 James Bach

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

  • 8/3/2019 SAST 2002-09-19 Managing ET

    18/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 18

    ET DocumentationPlanning:

    Charter overview of what to test (plan)

    Migh be a flip chart on the wall

    Mission What are we looking for?

    ExecutionNotes what happened during testing?

    What did I do? Why did I do it?

    Used to assess product quality after test.Data files input data used for testing

    Bug reports enough details to recreate the test / bug

    Track questions and Issues

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

  • 8/3/2019 SAST 2002-09-19 Managing ET

    19/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 19

    Approaches / Styles of ETAt the heart of all ET styles:

    Questionsand Questioning Skills

    Characterize the styles with respect to each other:Do they focus on:

    Method of questioning?

    Method of describing or analysing the product?

    The details of the product?

    The patterns of use of the product?The environment in which the product is run?

    To what extent would this style benefit from groupinteraction?

    From Black Box Software Testing, copyright 19962002 Cem Kaner

  • 8/3/2019 SAST 2002-09-19 Managing ET

    20/55

  • 8/3/2019 SAST 2002-09-19 Managing ET

    21/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 21

    Exploratory Testing SkillsGeneral systems modelling and dimensional analysis

    Inferencing out loud

    Experiment designTechnical story telling

    Use of mnemonics and heuristics

    De-biasing (individual or team)

    Exploratory investigationRisk analysis

    Bug advocacy

  • 8/3/2019 SAST 2002-09-19 Managing ET

    22/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 22

    I thought skill is the ability to something,more or less.

    Skill varies from person to person.Its distinct from talent and knowledge.

    A technique, by contrast, is a way of doingsomething; a sort of recipe.

    Skill belongs to a person, technique isuniversal.

    James Bach, Satisfice Inc.

    [email protected]

  • 8/3/2019 SAST 2002-09-19 Managing ET

    23/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 23

    Heuristics (and rules) and Skills we relate to heuristics as a tool to apply; something that might

    help us do the right thing in a given situation,

    whereas we relate to a rule as something to

    comply with; something that defines right behavior.

    Using heuristics properly requires that you exercise

    discretion and judgment, on some level; whereas judgment may get

    in the way of rules. It's helpful to have contradictory heuristics,because that's like having a variety of advice available before

    making a decision; whereas contradictory rules make compliance impossible.

    [email protected]

  • 8/3/2019 SAST 2002-09-19 Managing ET

    24/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 24

    Heuristic Risk Based TestingThe purpose of finding errors

    Make a prioritised list of risks.

    Perform testing that exploreseach risk

    As risks evaporate and newones emerge, adjust your testeffort to stay focused on thecurrent crop. Ref.: James Bach

  • 8/3/2019 SAST 2002-09-19 Managing ET

    25/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 25

    Summarizing ETThe object of your mission is to explore theMissouri river, & such principal streams of it,

    as, by its course and communication with thewaters of the Pacific ocean...may offer themost direct & practicable water

    communication across this continent for the

    purposes of commerce.

    - Thomas Jefferson's letter to Meriwether Lewis, June 1803

    Robinson, H., Microsoft, Exploratory Modeling

  • 8/3/2019 SAST 2002-09-19 Managing ET

    26/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 26

    Presentation Outline

    What is Exploratory Testing?

    How to do Exploratory Testing?

    How to Manage Exploratory Testing?

    High Speed WEB Testing Case Study

    E i

  • 8/3/2019 SAST 2002-09-19 Managing ET

    27/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 27

    The Test Session

    1) Charter

    2) Time Box3) Reviewable Result

    4) Debriefing vs.

    From Rapid Software Testing, copyright 1996-2002 James Bach

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

    Debriefing

    E ti

  • 8/3/2019 SAST 2002-09-19 Managing ET

    28/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 28

    ET Execution - SessionsSession

    Basic testing work unit

    Uninterruptede-mail, meetings, telephone calls etc.

    ReviewableA report should be produced

    CharteredMission associated with this session; What arewe testing? What are we looking for?

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

    Debriefing

  • 8/3/2019 SAST 2002-09-19 Managing ET

    29/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 29

    Charter:

    A clear mission for the sessionA charter may suggest what should be tested, how itshould be tested, and what problems to look for.

    A charter is not meant to be a detailed plan.

    General charters may be necessary at first:Analyze the Insert Picture function

    Specific charters provide better focus, but take moreeffort to design:

    Test clip art insertion. Focus on stress and flow techniques,and make sure to insert into a variety of documents. Wereconcerned about resource leaks or anything else that mightdegrade performance over time.

    From Rapid Software Testing, copyright 1996-2002 James Bach

  • 8/3/2019 SAST 2002-09-19 Managing ET

    30/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 30

    Time Box:

    Focused test effort of fixed duration

    Brief enough for accurate reporting.

    Brief enough to allow flexible scheduling.

    Brief enough to allow course correction.

    Long enough to get solid testing done.Long enough for efficient debriefings.

    Beware of overly precise timing.

    Short: 60 minutes (+-15)Normal: 90 minutes (+-15)

    Long: 120 minutes (+-15)

    From Rapid Software Testing, copyright 1996-2002 James Bach

  • 8/3/2019 SAST 2002-09-19 Managing ET

    31/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 31

    Reviewable Result: The session sheetCharter

    #AREAS

    Start Time

    Tester Name(s)Breakdown

    #DURATION

    #TEST DESIGN AND EXECUTION

    #BUG INVESTIGATION AND REPORTING#SESSION SETUP

    #CHARTER/OPPORTUNITY

    Data Files

    Test Notes

    Bugs#BUG

    Issues#ISSUE

    CHARTER-----------------------------------------------

    Analyze MapMakers View menu functionality and

    report on areas of potential risk.#AREAS

    OS | Windows 2000Menu | ViewStrategy | Function TestingStrategy | Functional AnalysisSTART-----------------------------------------------5/30/00 03:20 pm

    TESTER-----------------------------------------------Jonathan BachTASK BREAKDOWN-----------------------------------------------#DURATIONshort#TEST DESIGN AND EXECUTION65#BUG INVESTIGATION AND REPORTING 25#SESSION SETUP20

    From Rapid Software Testing, copyright 1996-2002 James Bach

  • 8/3/2019 SAST 2002-09-19 Managing ET

    32/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 32

    Debriefing:

    Measurement begins with observationThe manager reviews session sheet to assurethat he understands it and that it follows theprotocol.

    The tester answers any questions.

    Session metrics are checked.

    Charter may be adjusted.

    Session may be extended.New sessions may be chartered.

    Coaching happens.

    From Rapid Software Testing, copyright 1996-2002 James Bach

    Ch / i k Execution:

  • 8/3/2019 SAST 2002-09-19 Managing ET

    33/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 33

    SummaryExploratory Test Management

    Requires skills and experience practice

    Be prepared! Improve your skills!Charter Define your mission

    Take notes

    Test in SessionsTrack Session Metrics

    Charter/Mission

    Notes, Risks, Issues,Questions and Errors

    Execution:Pairs in Sessions

    Debriefing

  • 8/3/2019 SAST 2002-09-19 Managing ET

    34/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 34

    Presentation Outline

    What is Exploratory Testing?

    How to do Exploratory Testing?

    How to Manage Exploratory Testing?

    High Speed WEB Testing Case Study

  • 8/3/2019 SAST 2002-09-19 Managing ET

    35/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 35

    High Speed WEB Testing*

    A Case Study

    With permission from Jarle Vga, XIT AS,[email protected],

    Oslo, Norway* contribution to the book Software Quality and Testing in Internet Times, see references

  • 8/3/2019 SAST 2002-09-19 Managing ET

    36/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 36

    Presentation OutlineAbout the WEB application

    Preparation

    Environment and people

    Test Planning

    Test execution

    The web testing experience

    Its on the Web. Go find the bugs before the client does!

  • 8/3/2019 SAST 2002-09-19 Managing ET

    37/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 37

    Application: Basic functionalityWEB publishing tool:

    Content Creator creates pages

    Multi dimension / organisational viewsMulti language

    Topic Manager have to approve eachtopic before publication

    One company-One message

  • 8/3/2019 SAST 2002-09-19 Managing ET

    38/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 38

    Preparation...DocumentationSystem documentation not available onlyhigh level standard product description

    Changes on the test environmentConstant changes during planning period

    Browser difficulties

    Not possible to downgradeMicrosoft software, or toinstall two versions of InternetExplorer on the same PC

  • 8/3/2019 SAST 2002-09-19 Managing ET

    39/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 39

    What to doFeatures to be tested:Functional test

    Search facilities

    Browser testsInternet Explorer 4.0 and up

    Netscape 4.0 and up

    Operating Systems:

    Windows NTWindows 98

    Windows 2000

    would this be enough? And how to do it??

  • 8/3/2019 SAST 2002-09-19 Managing ET

    40/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 40

    Who?12 testers

    Different business areas

    Different countriesMarketing and publishing people, a few ITliterates

    2 test coordinatorsTest Environment manager

  • 8/3/2019 SAST 2002-09-19 Managing ET

    41/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 41

    Test Environment20 PCs

    5 browsers (versions)

    3 Operating systems12 people

    2 user groups simultaneously:Publishers / Content CreatorContent Approvers / Topic Manager

  • 8/3/2019 SAST 2002-09-19 Managing ET

    42/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 42

    How to...

    Test scripts - abandonedUnstable system

    Unavailable documentationPair testingTo motiovate the tesers andovercome any problems related to

    inexperienced peopleExploratory testing

    Will depend on the testers ability toexplore the system to find bugs

  • 8/3/2019 SAST 2002-09-19 Managing ET

    43/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 43

    The Test Plan (Chart)

    Features to be testedImportant to be precise, this helps focusing the pair testing

    Features not to be testedJust as important, used to limit the test

    ApproachPreparations and requirements

    No specific test script (but rather guidelines)

    Use of role-play and test matrixes

    Item pass/ fail criteriain order to judge the outcome of the test.

  • 8/3/2019 SAST 2002-09-19 Managing ET

    44/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 44

    Test ExecutionMajor difficulties with the set-up of the test environment

    Browsers not installed

    Win2000 pcs not available

    Functionality test according to guidelinesCreation of pages in the portal using role-play with pairs (ContentCreator/Topic Manager)

    Using a matrix, where roles, OS and browsers were integrated.

    Browser and op.sys. testMatrix with 5 browsers and 3 op.sys.

    Yellow pages (search functionality)re-use of pages published

    Exploratory testwhat happens if...

    Th t i i di t h

  • 8/3/2019 SAST 2002-09-19 Managing ET

    45/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 45

    ExecutionThe Matrix

    GroupsUser group

    1 2 3 4 5 6

    Content Creator

    Topic Manager

    Group # Operating System

    W-NT W98 W2000

    Netscape 4 6 1

    Netscape 6 3 4/6 5

    Iexplore 4 1

    Iexplore 5 4/5 2/3Browser

    Iexplore 5.5 2 5 6

    Day 1 10:00-12:00 13:00-15:00 15:00-16:30

    Time

    Day 2 09:00-17:00 09:00-17:00 13:00-16:00

    The upper matrix indicate howthe groups worked on differentcombinations of browsers andOS. The lower how the groups

    were divided into different usergroups at the start of the test, foreach change of operatingsystem they were alternated.

    I.e. Group 3 started as ContentCreator working on NT with

    Netscape 6, and moved to W98with Explorer 5 after lunch,working as Topic Manager.Different test guidelines was

    developed for each combination,which gave us

    OS*Browser = 3*5 = 15 differentguidelines.

  • 8/3/2019 SAST 2002-09-19 Managing ET

    46/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 46

    Execution Exploratory pair testing

    Plan increase coverage by using thetesters creativity

    which is what happenedIt can be difficult to keep testers ontrack

    but easy to motivate, and many bugsfound.

  • 8/3/2019 SAST 2002-09-19 Managing ET

    47/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 47

    Execution Debriefing and Notes

    Continous guiding / coordination required

    Taking Notes did not seem to be a problem

    (due to testers working in pairs)High number of duplicate error messages dueto no clear mission set for each pair

    Debriefing takes time! Need to be planned

  • 8/3/2019 SAST 2002-09-19 Managing ET

    48/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 48

    Acceptance Test SummaryApprox. 150+ bugs reported (incl. duplicates)

    Mainly due to lack of limitations in the system, youcould do anything.

    Developer accepted only approx. 60Corrupted pages: Web portal made inaccessible

    Menu disappeared when usingsigns like (#*\/@ etc ) in

    page titleCustomer: Severe safety riskDeveloper: Tester have misusedthe system

    TestManager

  • 8/3/2019 SAST 2002-09-19 Managing ET

    49/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 49

    Case Summary

    The portal was NOT recommended forpilot production after the test.

    Using pair- and exploratory methods

    proved successful

    Has happened since then:The two test coordinators scripted tests for the 60 bugs

    fixed and participated in a 2 days re-test of the applicationApplication went live after this total of 4 days testing andhas been in production without significant problems for morethen a year

  • 8/3/2019 SAST 2002-09-19 Managing ET

    50/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 50

    Lessons learned...(1)

    Pair testingSpeeded up reporting errors

    Excellent to promote creativity

    Encouraged discussionDemanded a strong voice to keep everyonefocused!

    Enables effective testing of short-lived, intense

    projects...but has also proved very useful on many testprojects, also larger (i.e. at Microsoft, ref. J.Bach)

  • 8/3/2019 SAST 2002-09-19 Managing ET

    51/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 51

    Lessons Learned...(2)

    Exploratory testingRequires skilled testers

    Requires some pre-set restraints, and thorough planning

    Requires a system for briefing and reporting/ documenting(ref. J.Bach -Session based test management)

    Requires guidance from test manager

    ..but when the requirements are fulfilled, it proved to bevery successful.

    Easy to motivate

    MANY deviations found

    and it gave testers a greater feeling of participation

  • 8/3/2019 SAST 2002-09-19 Managing ET

    52/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 52

    Learn moreExploratory Testing:

    Rapid Software Testing, by James Bach,[email protected]

    Black Box Software Testing, by Cem Kaner,

    [email protected]

    Amland Consulting; London, Copenhagen and Stockholm:Stockholm, October 30 31, 2002 (www.enea.se)

    http://www.amland.no/training_schedule.htm

    People Issues, test techniques, inspections:Grove Consultants, www.grove.co.uk

    + many, many more

    mailto:[email protected]:[email protected]://www.grove.co.uk/http://www.grove.co.uk/mailto:[email protected]:[email protected]
  • 8/3/2019 SAST 2002-09-19 Managing ET

    53/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 53

    References/Bibliography (1)Amland, S., 1999. Risk Based Testing and Metrics, EuroSTAR '99Preceding, Barcelona, Spain, http://www.amland.no/articles

    Argus, C. & Johnson, B., Ad Hoc Software Testing; Exploring theControversy of Unstructured Testing,

    http://www.testingcraft.com/ad_hoc_testing.pdfBach, James, 1999a. Risk-Based Testing. How to conduct heuristic riskanalysis, Software Testing & Quality Engineering Magazine,November/December 1999, vol. 1, issue 6,http://www.stqemagazine.com. See also www.satisfice.com

    Bach, James , 1999b. What is Exploratory Testing? Satisfice.com

    Bach, Jonathan, 2000, Session Based Test Management, STQEMagazine, 11/00, www.satisfice.com

    Beizer, Boris, Software Testing Techniques, Van Nostrand Reinhold.NY. 1990.

    Collard, Ross, 2002, to be published, [email protected]

    http://www.amland.no/articleshttp://www.testingcraft.com/ad_hoc_testing.pdfhttp://www.satisfice.com/http://www.satisfice.com/mailto:[email protected]:[email protected]://www.satisfice.com/http://www.satisfice.com/http://www.testingcraft.com/ad_hoc_testing.pdfhttp://www.amland.no/articles
  • 8/3/2019 SAST 2002-09-19 Managing ET

    54/55

    SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 54

    References/Bibliography (2)Kaner, C., Falk, Nguyen, 1999, Testing Computer Software, John Wiley & Sons

    Kaner, C., 2000. Rethinking Software Metrics, Evaluating measurement schemes,Software Testing & Quality Engineering Magazine, March/April 2000, vol. 2,issue 2.

    Kaner, C., 2001a, Black-Box Software Testing, training at Satisfice July 2001.

    Kaner, C., Bach, J., Pettichord, B., 2001b, Lessons Learned in Software Testing,John Wiley & Sons; http://www.testinglessons.com/, ISBN: 0471081124

    Kaner, C., Bach, J. 2001c, Exploratory Testing in Pairs, presentation atSTAREast, Orlando, FL, www.kaner.com

    Keith, Geordie, 2002, All Hands on Deck: How we used our whole company totest, STQE Magazine, vol. 4, issue 4, July / August 2002.

    Lyndsay, James and Eeden, Niel van, 2002, Adventures in Session-BasedTesting, paper at StickyMinds.com, will also be presented at EuroSTAR 2002

    Vga, J., Amland, S., 2002. High Speed Web Testing, contribution to the book Software Quality and Testing in Internet Times, Editor: Dirk Meyerhoff et al,Springer-Verlag, 2002, ISBN: 3-540-42632-9

    http://www.testinglessons.com/http://www.kaner.com/http://www.kaner.com/http://www.testinglessons.com/
  • 8/3/2019 SAST 2002-09-19 Managing ET

    55/55

    Stle Amland

    Amland Consulting

    Hulda Garborgsv. 2

    N-4020 STAVANGERNorway

    Phone: +47 905 28 930

    Fax: +47 51 58 55 24

    e-mail: [email protected]

    WEB: www.amland.no

    Contact Details

    mailto:[email protected]://www.amland.no/http://www.amland.no/mailto:[email protected]