Agile Test Management -

SoftTest Agile Test Management 21/04/2011

© Copyright Inspire Quality Services 2011

Slide 1

SoftTest Ireland

Today’s Event

- kindly sponsored by

Sogeti Ireland

InterTradeIreland

Software Skillnet

HP Ireland

Delivery Partners

2

Soft Test Events 13th/14th April

� 1.30 – 2.00 pm Registration

� 2.00 – 3.15 pm Agile Test Management – part 1

� 3.15 – 3.30 pm Coffee

� 3.30 – 5.00 pm Agile Test Management – part 2

� Followed by Networking



3

Agile Test ManagementSoftTest Workshop

April 2011Fran O’Hara

Inspire Quality Services

(formerly Insight Consulting)

e: [email protected]: www.inspireqs.iet: +353 (0)1 2854510

© 2011 Inspire Quality Services

4

Inspire Quality Services

Inspire (formerly Insight Consulting) provides Agile, Quality and Process Improvement Services such as

� Consulting/Coaching:– Strategic advice and hands-on Coaching/mentoring in areas such as agile, testing, process

improvement, etc.

� Training public/inhouse:– Agile: Scrum, Agile project management, Agile Testing, etc.

– Testing (ISTQB Foundation and Advanced Test Manager/Analyst, Risk-based testing, Test design techniques, Testing for developers, TMap®, Peer Reviews, UAT, etc.)

– Requirements/Business analysis

– Software project management

� Assessments– Agile practices

– Industry standards and models such as CMMI®, TPI®, TMMi®, etc.



5

Workshop Description

Much of the focus with testing in an agile environment has been on practices/techniques and tools but what about Test Management? Test Managers want to know what happens to their role when teams are self-empowered and what a test management process looks like in an agile context. Using Scrum as the main agile method (but with discussion of participants own approaches/variations), this workshop will through presentation, exercises and discussion cover topics such as

– Transitioning the test manager role to agile… both from the project test manager and line

manager perspective

– Test strategy in agile in differing contexts

– Test management process – estimation and planning in agile

– Test management issues and their place in agile – metrics, process improvement, tooling,

etc.

Participants, particularly those already working with agile, will be encouraged to share their challenges and experiences.

6

Agenda

� Introduction to Agile/Scrum

� Test Management challenges with Agile/Scrum

� Some key agile test management considerations

– Transitioning the TM role

– Agile Test Strategy

– Test Management Process – estimating and planning in Agile

– Test Management issues – metrics, process improvement, etc.

� Conclusions



7

Agile Manifesto - TM

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

– Individuals and interactions over processes and tools

– Working software over comprehensive documentation

– Customer collaboration over contract negotiation

– Responding to change over following a plan

“That is, while there is value in the items on the right, we value the items on the left more.”

8

Six principles of Agile Development

Promised ShippingDate

RELEASE

1 – 4 weeks

All

features

time

Working Software = Potentially shippable

2: Deliver chunks of high-value, well engineered, Working software often

4: The Customer can add, delete or reprioritise features at any time. i.e. this is how we “embrace change”

1: Customer lists known requirements (to a high level), then prioritises them.

prioritised

3: The Customer can release the software at any time they want.

££££££££££££££learn from the market

5: We protect schedule commitments, despite change

Backlog

6. We can review the project and the value it delivers at the end of each increment

Clarke Ching - www.clarkeching.com



9

Scrum

See www.controlchaos.com

Roles:

-Scrum master

-Scrum team

-Product owner

Retrospective

10

ScrumMaster

Product Owner

►Defines the features of the product, decides on release date and content►Is responsible for the profitability of the product (ROI)►Prioritizes features according to market value►Can change features and priority every 30 days►Accepts or rejects work results►An active role pointing the work in a particular direction, evaluating the results and adjusting direction based on the reality of what the last piece of work produced. Typically a Product Manager, Marketing, Internal Customer, etc.

►Cross-functional, seven plus/minus two members►Selects the Sprint goal and specifies work results►Has the right to do everything within the boundaries of the project guidelines to reach the Sprint goal

►Organizes itself and its work►Demos work results to the Product Owner

►Ensures that the team is fully functional and productive►Enables close cooperation across all roles and functions and removes barriers►Shields the team from external interferences►Ensures that the process is followed. Invites to daily scrum, Sprint review and planning meetings. Typically filled by a Project Manager or Team Leader

►Is NOT manager of the team

Key roles and responsibilities



11

� What does “Done” mean for the project?...

– Spec written

– Code checked in

– Builds

– Unit tests complete successfully

– 80% code branch coverage on unit tests

– Usual coverage of boundaries, story conditions/requirements, risks, etc.

– Within acceptable defect levels

– Non functionally tested (performance, security…?)

– Etc.

– Accepted by product owner

� If stories not ‘done’ by end of Sprint then they are put back on backlog for the next Sprint planning

� Can there be more than one level of ‘Done’?

Done

12

Discussion

� A key Scrum principle is: “Trust the Team”. A Scrum sprint delivers “working software, ready for deployment”.

� So “no need for independent testing” may be the viewpoint of scrum people you are collaborating with. And the problem is: it makes sense! To a certain extent at least.

� What’s YOUR position here? Do you agree, even in complex IT landscapes and critical business chains? Or do you oppose? Or something in between?



13

Agenda








� Conclusions

14

Typical general quality related

challenges

Non-functional

issues

Developer buy-in for

shared quality

ownership

‘Agile’ without sufficient customer

involvement

Insufficient focus on working software –

stories not ‘done’

Lack of technical expertise in test

team

Quality of unit tests

Testing bottleneck

Ineffective incremental test

strategy

Hybrid implementations

Integration Testing

Effectiveness of automated

tests

Role of the tester and

test manager

Distributed development



15

Your Agile Test Management

challenges?� …

� …

� …

16

Agenda








� Conclusions



17

Test Line/functional Management responsibilities in agile – some

considerations � Support tester capability within agile teams

– Hiring/firing, overall test resource management

– Career development/Performance management

– Training/coaching/mentoring

� In domain/product

� In agile test process/culture

� In technology

– Specialist test support for agile teams

� Performance, security, etc.

� Remove organizational obstacles� Agile test strategy?

– Discussed later

– Supporting overall quality e.g. supporting developers, automation approaches, etc.

� Agile process definition?– Test practices such as TDD

– Test role definition

– Balancing standardisation/organisational improvement with team empowerment?

� Test Tool standardisation?

Note: for programmes/large projects a Project Test Manager might be responsible for much of the above (rather than an organisational test line/functional manager)

18

Project Test Management in agile –

some considerations

� Traditional test manager responsibilities now done by agile team:– Estimation and Planning– Monitoring/control

� Direct involvement in agile teams?– ScrumMaster– Team member for key developments – leading by example

� Larger more complex projects– Higher level involvement in planning and co-ordination– Test Integration planning and co-ordination– Reporting



19

Agenda








� Conclusions

20

Scrum Test Strategy

� Risks– Similar product risks

– Regression risk with high level of change

� How many test levels?– XP appears to advocate two as part of a

predefined test strategy� Unit and Acceptance (both automated as part of Test Driven

Development)

� Is system test no longer required?

– Automation reduces regression risk

– Developers doing testing reduces risk of poor quality code

– But how can a test strategy/approach be method rather than product based?



21

‘Acceptance’ Testing – is it enough?

� Usually not!…context/risk/strategy issue…

– May not be fully automated – partial regression strategy needed

– Expand to fuller ‘system’ tests� Functional testing

� Non-functional testing – performance, usability, etc.

– May still need end-to-end business scenario focused User Acceptance test, user story interaction tests, etc.

– System integration testing issues

– Etc.

22

Agile Testing Quadrants



23

Testing within a Sprint

Automated Acceptance/Story

basedTests

Automated Unit Tests

Manual Exploratory

Tests

Represent Executable requirements

Represent Executable Design specifications

Provides Supplementary

feedback

24

Agenda








� Conclusions



25

Planning in agile

� Release and iteration/sprint planning

� Plan as you go versus full upfront

� Adaptive based on feedback– Estimation : ‘Velocity’

– Test results available early so use to adapt strategy/plan

� Iteration planning based on short term upcoming work

� Informal planning tools – whiteboards, stickies, wikis, checklists, etc. (versus Gantt charts, comprehensive documents)

26

Test Estimation� Done as part of story estimation and sprint planning

� Story estimation in backlog

– Typically story points

� Sprint planning – task estimation

– Typically effort in hours

� Both can use planning poker (delphi) technique for a team based estimate

– E.g. using fibonacci sequence 1, 2, 3, 5, 8, 13



27

Exercise – test strategy and

planning� In small groups, review the scenario provided in the

handout and define the key elements of a test strategy/plan as outlined

� Followed by group discussion/feedback

28

Agenda








� Conclusions



29

Metrics� Burn rate. How much money/time/resources are we

investing in this effort? Like it or not, someone always wants to know this number.

� Delivered functionality. The delivery of working software is the only true measure of progress on a software development project. Simple ways to measure this are simply tracking – Velocity. This is a measure of how much functionality

can the team deliver per iteration. This measure is often captured in the form of "use case points" or "story points" which are specific to the team.

– Defects. Knowing how many defects, and the severity of those defects, is an important determinant of whether you're in a position to deploy your system.

� Scott W. Ambler

30

Other issues� Test Process improvement in agile

– Retrospectives– Is there a role for guidance, organisational

improvement, etc.?– Suitability of TPI, TMMi, etc?

� Similarly re Tooling– Balancing empowerment with Organisational support

to avoid tool proliferation and associated inefficiencies

� More?



31

Summary

� Incremental & Working Software!!!� Agility requires a particular mindset compared

to traditional plan driven approaches – can be a significant cultural change

� Be proactive - Define your role and an appropriate test strategy

� Need highly automated regression testing� Collaborate closely with developers and

customers� Business value drives priority� Testing provides continuous feedback and

decision support

32

Conclusions

� In small groups list your top 3 Do’s and Don’ts for Agile Test Management

� Facilitator to collate



33

Questions/discussion-

any outstanding challenges, etc.

Contact Details:

[email protected]

Agile Test Management -

Documents

Transcript of Agile Test Management -