Reporting principles for every QA manager

Joel MontveliskyChief Solution

Architect

Simplify Test Management

PractiTest Webinar:Reporting Principles for Every QA Manager

What do we want to cover today?

• DDTM – Decision Driven Test Management

• The need for repetition & multiple reporting channels

• Numberless reporting

• Do’s & Don’ts for successful reporting

• Questions and AnswersSimplify Test Management

A little about me…Testing:

- Tester- QA Manager- Blogger- Speaker- Solution

Architect

Some numbers:- 1 wife- 3 kids- 1 dog & 2 fish- 18 years testing

PractiTest- Chief Solution Architect- Test Manager


About SaaS ALM Solution Simple & Intuitive – yet – Methodological & Flexible Integrates with Jira, Selenium, Pivotal Tracker, Jenkins,

Redmine, Bugzilla, QTP, Ranorex, SoapUI, etc… Best Darn Support in the Business


DDTMDecision-Driven

Test Management


Why Do We Test?


To provide our stakeholders with visibility

into the status of the product and process,

so they can make the correct decisions.

DDTM Decision-Driven Test ManagementPlanning and running your tests based on the information needed by your stakeholders to make their decisions.

DDTM –Reverse Engineering your Testing Process:

1. Identify your stakeholders and their decisions

2. Understand the information needed to answer questions

3. Plan your reports, metrics and statistics

4. Define the tests to capture data

5. Run your tests based on timelines required for data capturing

6. Continuously monitor for new stakeholders & their changing needs


A Word of Caution About Stakeholders’ Decisions

Stakeholders come in many types & sizes.

Decisions tend to happen at the beginning or the end of the project.

Decisions are not only based on user stories or requirements.

Each stakeholder will make different

decisions, at different times,

based on different information.


The Need for Repetition&

Multiple Channels


Question:


Which of the following reporting channels do you use?

Web-based Dashboards Live Monitors High or Low Level Testing Reports Emails with Status Updates Update Meetings Periodic Management Presentations Other

Need for Repetition(Theory of Effective Frequency)


Herbert E. Krugman (GE 1960’s):

“People need to be exposed 3 times to act on an

idea.”

1st time: What is it? (Curiosity)

2nd time: What of it? (Recognition)

3rd time: Take action. (Decision)

Using Multiple Channels


People absorb information differently:

Graphically Textually Statistically Comparatively

etc.

People access information via multiple media:

Computer Phone Verbally Presentations

etc.

It is more effective to combine multiplechannels & formats to communicate information


And so...In the case of Reports, More is actually Better.

More Repetitions x More Formats =

Better Results!!

Numberless Reporting


Question:


Do you use exact numbers as part of your reports?

e.g. Number of Tests, Number of Bugs, etc



ReminderDon’t make

me think!Cheers,

The Boss

If you already know what interests your stakeholders, why not provide them with the processed information?

Numberless Reporting - example


Version 7.2 Status Overview:

Feature A.1This

week Last

week

Feature B.2Feature C.3Feature D.4Platform E.5Platform F.6Stability & Load G.7

Overall project projectionStabilityNew features

Version Projection

Main risks Major delays on feature D.4 Lack of stability in Feature

C.3 4 Team members moving to

project XYZ next week



Color code the main areas (features?) of your project

List the most important things to know

Provide projections

Make sure people know where things are good and where they are not good

Be ready to provide numbers to back your numberless conclusions.

Reporting Do’s & Don’ts


Do 1: Always start from the bottom line then you can explain


Place the most important information on the top and center.

Add data to support your conclusions, but always at the end.

You telling the News, and not writing a Novel!

Do 2: Work S.M.A.R.T.


Simple

Measurable

Actionable

Repeatable

Timely

Do 3: Trust your gut


Do 4: Focus

Express your feelings even if you sometimes lack hard data to back them up.

Be concrete and focused on what you are trying to say.

Don’t beat around the bush.

Don’t 1: Don’t fall in love with your data


Don’t 2: Don’t be afraid to correct yourself

Don’t 3: Don’t be afraid to say I don’t know(and get back to people with the answer later)

Questions & Answers

About SaaS ALM Solution Simple & Intuitive – yet – Methodological & Flexible Integrates with Jira, Selenium, Jenkins, Redmine, Bugzilla,

QTP, Ranorex, SoapUI, etc… Best Darn Support in the Business


Thank [email protected]

http://qablog.practitest.com


Reporting principles for every QA manager

Software

Transcript of Reporting principles for every QA manager