W13

Test Management

10/15/2014 3:00:00 PM

The Test Manager’s Role in

Agile: Balancing the Old and the

New

Presented by:

Mary Thorn

ChannelAdvisor

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Mary Thorn

ChannelAdvisor A QA director at ChannelAdvisor in Morrisville, NC, Mary Thorn has a broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques. During her more than seventeen years of experience in healthcare, HR, financial, and SaaS-based products, Mary has held manager and contributor level positions in software development organizations. She is a strong leader in agile testing methodologies and has direct experience leading teams through agile adoption and beyond.

The Test Manager’s Role in Agile

Balancing the Old and the New

By Mary Thorn

Director of QA at ChannelAdvisor in Morrisville, North Carolina,

A broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques.

During her more than fifteen years of experience in healthcare, HR, agriculture, and SaaS-based products,

Mary has held manager and contributor level positions in software development organizations.

She has a strong interest in agile testing methodologies and direct experience leading agile teams through Scrum adoption & beyond.

About Mary

Mission & Vision

3-Pillars Strategy

Converting Strategy to Roadmaps (Backlogs)

"Selling" the Roadmaps –

Metrics & Assessments

Agile Champion:

Continuous Improvement

Outline

Mission

The QA team members are functioning as an equal part of the overall Scrum Team

They are Subject Matter Experts (SMEs) in the area of the application.

They are technically competent.

QA members are accountable, committed, trust their teammates, driven to results, and don’t fear conflict.

They are empowered to be change agents.

Vision

3 Pillars

Development & Test Automation

Software Testing

Cross Functional Practices

What does success look like?

Roadmap

Transparency around areas where improvement is needed within QA, especially around areas of automation.

It helps define what success looks like for the QA personnel for the year, and it helps define individual objectives around this.

Defines the strategy discussed above.

It actually makes you look like you know what you are doing.

Roadmap

Your QA team(s) or Scrum team

You as a leader (your experience, instincts, etc.)

Your organizational directives

Retro

Roadmap – Who inputs to them?

What are examples of what would be in a roadmap?

Build automation framework

Automate X smoke test

Automate X regression test

Define/implement performance testing strategy

Train/implement BDD

Train/implement exploratory testing

Write manual regression test cases for x

Create QA Standard Operating Procedure document

Migrate away from Quality Center/ALM and save the company$100k/year(BEST roadmap item ever)

Roadmap cont.

I have one now what?

Once you have your roadmap, you need to work with your product owner organization to create stories, and get them prioritized and injected across your entire team’s product backlogs. Make sure that you put all of your work in a “business context”, explaining what it will do for “them” or the “customer”.

Another approach is to allocate a specific percentage of each backlog to this sort of work.

If you do not have a global agreement of some kind, you have to allocate your work to the product backlogs on a situation-by-situation basis.

I have one now what?

Roadmap for Sale

Cross Cutting Strategy

The team has to get it done, do it incrementally, measure it, and make adjustments along the way.

QA Manager should collaborate with your team in the execution phase

This gives them ownership and accountability, and creates trust. Without team buy-in and the QA Manager’s leadership of the strategy, it is no more than writing on a piece of paper.

Strategy - Implementation

Measures something only if you plan to act on the results

Measures only what can be measured

Measures at the correct level and in the correct units

Is easy to collect

What makes a Metric “good”?

All the information they need to make decisions, and no more

Information at the level of detail they can use

Information at the scope they care about

Information pertaining to the time frame they care about

What do your Stakeholders want?

Name

Question being Answered

Basis of Measurement

Level & Usage

Expected Trend

When to Use It

When to Stop Using It

How to Game It

Warnings

Metrics Checklist

Defects – Sev 1/Sev trend

0

10

20

30

40

50

60

Total Sev 1 &2 per month hotrolled

Total Sev 1 &2 per month

Bug Score by Team

Unit Tests– a test that has no dependencies (do our objects do the right thing, are they convenient to work with?) Integration Tests– a test that has only one dependency and tests one interaction (usually, does our code work against code that we can’t change?) UI Tests– a test that tests the UI and work flow.

Unit = 12830

(10% Increase)

Int = 1071

(2% increase)

UI= 671

(5% change)

Automation Pyramid

Forrester Agile Testing Tool

Forrester Agile Testing Tool

0.00

1.00

2.00

3.00

4.00

5.00

Testing behavior

Agile testing organization

Agile testing practices Automation

Agile adoption

Agile Assessment Tool

Agile Assessment Tool

0%

25%

50%

75%

100%

Product Health

Release Health

Sprint Health

Team Health

Debt and Done Health

Product Quality

Affirm and reinforce lean / agile practices

Measure results, not output

Follow trends not numbers

Belong to a small set of metrics / diagnostics

Provide fuel for meaningful conversation

Diagnose & improve the processes that produce business value

Provide feedback on a frequent and regular basis

Are easy to collect

Good Agile Metrics

Agile Champion

Doneness/Policies

Removing Impediments

• Test Data Strategy

• Environments Strategy

• Continuous Integration Strategy

• Defect Cleanup Strategy

Removing Impediments

• Discovery Retro

• Quarterly QA Retro

• Technical Debt Retro

• Post Prod Sev 1 or 2 Retro

Retrospectives

Servant leader shares power, puts the needs of others first and helps people develop and perform as highly as possible.

Servant Leadership

Project Management – Scrum Masters

Architecture and Business Analysis

Development

Testing

UX Design

Product Owners

DevOps or Technical Operational Team(s)

Documentation

Customer Facing Operational Team(s)

Relationship building

Hiring

Hiring

First Who, Then What: Get the right people on the bus, then figure out where to go. Finding the right people and trying them out in different positions.

Hire by traits in order of:

1. Attitude

2. Aptitude

3. Skillset

NO ROCKSTARS NO HEROES

Hiring

People Management

Trainings – All 3 Pillars

Automation

Testing Practices

Team(IE Five Dysfunctions)

Agile

10% time

Management and Leadership

Continuous Improvement

Wrapping it Up