The Test Coverage Outline: Your Testing Road Map

W2 Test Techniques

5/1/2013 11:30:00 AM

The Test Coverage Outline: Your

Testing Road Map

Presented by:

Paul Holland

Testing Thoughts

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Paul Holland

An independent software test consultant and teacher, Paul Holland has more than sixteen years of hands-on testing and test management experience, primarily at Alcatel-Lucent where he led a transformation of the testing approach for two product divisions, making them more efficient and effective. As a test manager and tester, Paul focused on exploratory testing, test automation, and improving testing techniques. For the past five years, he has been consulting and delivering training within Alcatel-Lucent and externally to companies such as Intel, Intuit, Progressive Insurance, HP, RIM, and General Dynamics. Paul teaches the Rapid Software Testing course for Satisfice. For more information visit www.testingthoughts.com.

1

Product Coverage Outlines

and How to Use Them

Paul HollandTest Consultant and Teacherat Testing Thoughts

My Background

• Independent S/W Testing consultant since Apr 2012

• 16+ years testing telecommunications equipment and reworking test methodologies at Alcatel-Lucent

• 10+ years as a test manager

• Presenter at, STAREast, STARWest, Let’s Test, CAST, STARCanada, and EuroStar

• Keynote at KWSQA conference in 2012

• Facilitator at 25+ peer conferences and workshops

• Teacher of S/W testing for the past 5 years

• Teacher of Rapid Software Testing– through Satisfice (James Bach): www.satisfice.com

• Military Helicopter pilot – Canadian Sea Kings

April, 2013 ©2013 Testing Thoughts 2

2

Attributions

• Many of the concepts that I am presenting

come from the Rapid Software Testing

class developed by James Bach and

Michael Bolton


Test Strategy Development

• A Test Strategy is often developed by:

– Following a template

– Envisioning the entire product as a whole

– Using elements from previous features/products

– Using estimates from the past

– Some use development estimates of complexity

and effort to determine test effort


3

An Alternative

• Each feature/product is different from other

features/products

• Start with “raw” project specific information:

– Detailed description of the elements of the

feature/product

– Identify specific risks related to the project

– Develop a list of “test ideas”


Product Coverage Outline

• A Product Coverage Outline can be

described as:

– A parts list

– An inventory of elements

– Components of a product

• Essentially a list of items of a project that can

be tested in some way


4

Benefits of a PCO

• Helps eliminate tunnel vision

• Will help display your thoroughness to others

– Help you gain respect

• Will allow you to get “buy-in” at a high level of

what you are going to consider when testing

• Helps prioritize your testing

• Helps in reporting



How to create a PCO


5


• Take a tour of the feature/product

– Investigate every menu item

– List sub-menus

– Context specific items

– Text boxes

– Radio buttons / Check boxes

– Visual elements (pictures, background, logos)

– Pop-up boxes



• Talk with designers, business people,

customers, customer service, etc.

• Look at previous versions

• Look at marketing information

• Read the functional specification document

(*although, I would do this later – seriously)


6


• Consider the following (SFDIPOT):

– Structure of the program (smallest components)

– Functionality (individual features)

– Data (I/O, create, store, manipulate, backup, F)

– Interface (User interfaces, APIs, F)

– Platform (Computer, CPU, OS, browser, F)

– Operations (How is it used by customers)

– Timings (Race conds, time of day/wk/mth/yr, F)

(Taken from the Rapid Software Testing Course, Bach and Bolton)



• As you are investigating the feature/product

– Create a list of ALL the elements that MAY need

to be tested in some way

• Make a mind map (I use XMind)

• Enter the elements into Excel (or a word processor)

• Create a list on paper, flip chart, or whiteboard

• Make a bunch of sticky notes

(I would encourage electronic versions for ease of

sharing and re-use – but it is not mandatory)


7

Risk List

• Create a list of project specific risks

• These are in addition to “generic” risks

• Talk with designers, business managers,

customers, customer support, etc.

• Try to identify risks during the creation of the



Test Ideas

• This is NOT a list of specific test charters

• List ways to test that you came across during investigation. For example:

– Overflow input buffers

– Check interop between two components

– Verify information in text boxes/help screens

– Modify values, change all values from default

(These are only a very small set of examples)


8

3 Raw Lists: Now What?

• Use these three lists as the starting point

for your test strategy

• Get agreement from stakeholders of the

components and their importance

• They are important in “traditional” testing

environments as well as for agile teams


3 Raw Lists: agile testing

Use these 3 lists to:

• Create your list of Charters

– Combine the test ideas, risks and components to

create specific charters

– Add “test ideas” under each Charter

• Help with the initial prioritization

• Assist in creation of new Charters during testing

• Help generate a better definition of done


9

3 Raw Lists: Traditional

• Create your list of Test Cases (shudder)

• If you must create scripted test cases,

using these three lists will help:

– Give you better coverage (not just using the

requirements)

– Reduce rework (because you toured SUT)

– Prioritize the test cases for execution and

reuse


Hands on

Creation of a PCO for a simple program

• Lego MindstormsTM robot which changes

behavior depending on “sensed” color

• A Flash program that simulates the robot

• Generation of a PCO for the robot

• How would PCO differ for the Flash

simulation?


10


The Test Coverage Outline: Your Testing Road Map

Technology

Transcript of The Test Coverage Outline: Your Testing Road Map