Scrum Gathering 2012...

© 2011 SAP AG. All rights reserved. 1 Public

Our Exploratory Testing Journey

Zhang Kebing

SAP Labs China

June, 2012 Public


Disclaimer:

This presentation outlines our general product direction and should not be relied on

in making a purchase decision. This presentation is not subject to your license

agreement or any other agreement with SAP. SAP has no obligation to pursue any

course of business outlined in this presentation or to develop or release any

functionality mentioned in this presentation. This presentation and SAP's strategy

and possible future developments are subject to change and may be changed by

SAP at any time for any reason without notice. This document is provided without a

warranty of any kind, either express or implied, including but not limited to, the

implied warranties of merchantability, fitness for a particular purpose, or non-

infringement. SAP assumes no responsibility for errors or omissions in this

document, except if such damages were caused by SAP intentionally or grossly

negligent.


The role of manual testing in Scrum

We use manual test when

1. Some features are too costly to automate.

2. Some features are not automatable.

3. Find the complicated bugs.

4. Explore new ways of testing a feature.

Shipping new features after each sprint


Ad hoc vs. Scripted Test Case Design

Scripted

Testing

Ad hoc

Testing Design effort Low High

• No test strategy

• No focus

• No traceability

• High

planning/maintenance

effort

• No variations

• Kills creativity


Exploratory Software Testing with Tours

A lightweight approach to test case design and execution

[1]


Exploratory Testing

Interaction

Creativity

[2]

[3]


Analogy: A software tester is like a tourist…

• You can not see everything

• There are lots of different ways of exploration

• You want to make sure to see the most

important things

[4]


Feature: Campaign Management in Real Estate


Garbage Collector Tour Garbage men go street by street, house by house. They stay only

a few moments but they crisscross the neighborhood in a

methodical manner.

[5]


Garbage Collector Tour 1

Example for Campaign in Real Estate

1. Open the UI of add a new Campaign.

2. Execute all main sub-frames.

3. Make sure you go to each sub-frame and

create a new item.

4. Don’t test every detail.

5. Test document

Jason&Sean_Campaign_Garbage.docx


Garbage Collector Tour 2

Example for Campaign in Real Estate

1. Open the UI of add a new Campaign.

2. Only test one campaign type, but check all the buttons on the menu.

3. Don’t touch other sub features.

4. Test Document

Template_Campaign_Garbage_ella&chloe.docx

Anti-Social Tour

You are not like everybody else.

[6]


Anti-Social Tour

[7]

Example

If valid input is a number enter a

character

if valid input is a picture file use a word

file

if you are on the last page of a wizard

hit “Back” and change something, …


Supermodel Tour

It’s all about the look…

[8]


Supermodel Tour

Design and execution of the tour in general:

1. Pick a feature

2. Make sure it follows the UI guidelines

3. All icons are correct

4. All texts contain no spelling mistakes

5. …


Standard Tours

Business District • Guidebook Tour (F1

Tour) • Money Tour • Landmark Tour • Intellectual Tour • FedEx Tour • After-Hours-Tour • Garbage Collector's

Tour1

Tourist District • Collector's Tour • Lonely Businessman

Tour • Supermodel Tour • TOGOF Tour

(TestOneGetOneFree) • Scottish Pub Tour

Seedy District • Saboteur • Antisocial Tour

• Opposite • Crime Spree • Wrong Turn

• Obsessive-Compulsive

Historical District • Bad-Neighborhood • Museum Tour • Prior Version Tour

Entertainment District • Supporting Actor Tour • Back Alley Tour • All-Nighter Tour

Hotel District • Rained-Out Tour • Couch Potato Tour

James Whittaker

Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design

Addison Wesley Pub Co Inc, 2009


Exploratory Testing with Tours!

Failure!

Failure!

Failure!

Failure!

[14]


Exploratory Software Testing Ingredients

Test Tour Time Box Protocol


Time Box

Helps you to

stay focused

It should

between 30 – 60

minutes per tour

You must take

note, record

your test within

time box

Prepare the test

before you start

to test


Protocol – Test Document

Written on the fly by the

tester

Documents what has

been tested

Documents which bugs

have been found

Documents which test

opportunities are still

available

Documents open

questions (bug or

feature?)

Examples

Template_Campaign_Garbage_ella&chloe.docx


Test Tour

• Defines test approach

(How)

• Leaves the tester enough

freedom to explore

• Ensures tester stays

focused, not testing

everything

[9]


Integration into the Scrum process

- Test Strategy

Team A:

Scripted Test

ET

Team B:

Unit Test

Auto-System Test

ET



- Which tour?

Reports

• Intellectual Tours : ask the hardest

questions to the reports.

• Monthly close?

• Enter transactions with huge

amount?

• View reports with multiple

currencies?

• …



- Which tour？

Invest more than one tours for a feature.

• Garbage collector tour - Test

Coverage

• Anti-social tour - invalid inputs or

operations.

• Landmark tour - Features in different

sequences.



1. Define Tasks – Planning Meeting

2. Test Prepare - Tester

3. Who perform – any team member

4. Rules – Cross test, start test when all development tasks is done.

Story Tasks To-Do In Process Completed Done

Story

Bug

Story


What ET bring to our scrum team?

Long document Just Enough VS

[10]



• Enlarge testing resource

• Involve everyone

[11]



• Better test coverage

• Each of tester has their own way

to explore the software

[12]


Current Challenges

• Try more tours with scrum team

• Try more different approaches

• Follow up on bug reports and find out whether your test strategy

works or not.


Try!

Contact information:

Zhang Kebing

Quality Engineer/ Scrum Master

[email protected]


References

[1] http://www.flickr.com/photos/statelibraryofnsw/2876551803/

[2] http://www.quardev.com/articles/exploratory_testing

[3] http://www.flickr.com/photos/bobcatrock/2862617356/

[4] http://www.flickr.com/photos/gamene/4341746457/

[5] http://www.flickr.com/photos/nasacommons/4858566574/

[6] http://www.flickr.com/photos/foxypar4/1852428153/

[7] http://www.flickr.com/photos/judepics/1950452493/

[8] http://www.flickr.com/photos/istolethetv/4045140609/

[9] http://www.nipic.com/show/1/66/71cbc36ac3b50aa9.html

[10] http://www.superstock.com/stock-photos-images/1775R-6937

[11] http://www.allmoviephoto.com/photo/2003_poster_smith_international.html

[12] http://www.fs3000.com.cn/bbs/read.php?tid=610

[14] http://blog.sysomos.com/2010/05/19/twittera-as-a-resource-tool/question-

mark/

http://www.flickr.com/photos/statelibraryofnsw/2876551803/

http://www.quardev.com/articles/exploratory_testing

http://www.flickr.com/photos/bobcatrock/2862617356/

http://www.flickr.com/photos/gamene/4341746457/

http://www.flickr.com/photos/nasacommons/4858566574/

http://www.flickr.com/photos/foxypar4/1852428153/

http://www.flickr.com/photos/judepics/1950452493/

http://www.flickr.com/photos/istolethetv/4045140609/

http://www.nipic.com/show/1/66/71cbc36ac3b50aa9.html

http://www.nipic.com/show/1/66/71cbc36ac3b50aa9.html

http://www.superstock.com/stock-photos-images/1775R-6937









http://www.allmoviephoto.com/photo/2003_poster_smith_international.html

http://www.allmoviephoto.com/photo/2003_poster_smith_international.html

http://www.fs3000.com.cn/bbs/read.php?tid=610

http://www.fs3000.com.cn/bbs/read.php?tid=610

http://blog.sysomos.com/2010/05/19/twittera-as-a-resource-tool/question-mark/














No part of this publication may be reproduced or transmitted in any form or for any

purpose without the express permission of SAP AG. The information contained

herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain

proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of

Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5,

System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries,

zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390

Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6,

POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes,

BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF,

Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere,

Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM

Corporation.

Linux is the registered trademark of Linus Torvalds in the U.S. and other

countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or

registered trademarks of Adobe Systems Incorporated in the United States and/or

other countries.

Oracle and Java are registered trademarks of Oracle and/or its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and

MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.

HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®,

World Wide Web Consortium, Massachusetts Institute of Technology.

© 2011 SAP AG. All rights reserved.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects

Explorer, StreamWork, and other SAP products and services mentioned herein as

well as their respective logos are trademarks or registered trademarks of SAP AG

in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal

Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business

Objects products and services mentioned herein as well as their respective logos

are trademarks or registered trademarks of Business Objects Software Ltd.

Business Objects is an

SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other

Sybase products and services mentioned herein as well as their respective logos

are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP

company.

All other product and service names mentioned are the trademarks of their

respective companies. Data contained in this document serves informational

purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document

may be reproduced, copied, or transmitted in any form or for any purpose without

the express prior written permission of SAP AG.

Scrum Gathering 2012...

Technology

Transcript of Scrum Gathering 2012...