Moving to an Agile Testing Environment

What Went Right?

1

What went Wro

Ray ArellSr. Engineering Manager, Intel

Corporationray.arell@intel.com

ng?

Sprint 6 Customer Review

2

Career user of the waterfall product life cycle…Prior owner of the platform level waterfall corporate specification…Contributor to a number of other waterfall standards…

3

What Happened?

Even spoke at conferences about my brainchild the “Framework of Quality”!

Yep, that’s waterfall

Background: Prior to the Move

Lots of milestones, checkpoints, and processes

Waterfall of Complexity

My team

Typical Project

Exponential Complexity

©2005 Intel Corporation

Moore's Law states that the number of transistors on a chip doubles about

every two years. The same holds true

for software!

Flexibility

Observation of Waterfall Delivery

Planning Production

DevelopmentExploration

No! No! No!

3-6 Months

7

“The basic framework described in the waterfall model is risky and invites failure.”–Winston Royce, creator of the original waterfall model

Customer

“There is at least one point in the history of any company when you

have to change dramatically to rise to the next level of performance. Miss

that moment - and you start to decline.” -- Andy Grove

8

9

What Sold Me on AgileFocuses on delivering high customer value with every releaseWork culture that promotes:

TeamworkJust enough process to get stuff doneFrequent customer feedbackHigh level of empowerment

Welcomes change and allows the product to evolve to meet the customer’s needs

This is not Agile….

10

Cowboy Coding

Ad hoc processes focused on doing things people want to do vs. need to do.

Wagile

Doing short waterfall delivery and calling it Agile

Agile Nucleus

11

Scrum Project Framework

12

Each sprint is a

fixed duration

Team works from a prioritized

product backlog

Short daily team

meetings

Must deliver

working and fully

tested code

Scrum ConstraintsScrum project framework puts three major constraints on testing

Products are delivered on a fixed cadence and cannot be pushed out All features need to be working and meet the acceptance criteriaNo features are shipped to the customer if it is not tested, repaired, and retested

User stories/features by design are expected to evolveDetails and acceptance criteria in the backlog will evolve over timeMay be deferred until the maximum amount of information is availableDevelopment of the product itself may fill in the gaps

Customers may shift prioritiesThey are the customer after all!

13

Moving from the Waterfall to ScrumEvolve vs. cold turkey

A small co-located team can move faster

Train your team prior to starting!Scrum Master certified, Product Owner and team trainedMajor paradigm shift for everybody

14

My Plan

What Went Wrong… The Early Sprints

15

"A paradigm shifting without a clutch.“--Dilbert, 25 Aug 1995

Team Adoption

16

ScrumFalls

frAgile

Agile

1 5 10 15 20

Sprint

17

Transitioning to ScrumEffect on the Team

Role of managementJob titles and trustShifting to self-managed The role of validationPerception of micromanagement

Effect on the CustomerTimidLevel of involvementMisconceptions of the process

Fine Tuning the ProcessGetting “Done” defined correctlyInterfacing with non-scrum teamsCross-site/geo communications How Validation/QA should fit

Micromanagement??

18

Daily Standup

Focus of the Stand-up

Inform, Commit to Peers, Ask for Help

What Went Wrong...PO’s acting like administratorsAccountability EmbarrassmentScrum Master needing to be a stronger gate keeper

19

Effect on Testing Professionals

Stress, Anger, and FearTest strategy needed to change Working integrated Testing, debug, and retest fasterDealing with requirement changes

Source: Internal team survey

50%

33%

17%

Employee Burnout

Scrum Waterfall Ambivalent

Team was running too

fast!

What Went Right…

21

“Chaos in the world brings uneasiness, but it also allows the opportunity for creativity and

growth.”-- Tom Barrett

Roles and Responsibilities

22

Management

Self Managed

and Strong

customer orientation

High Focus on removing

obstacles and growing

people

Engaged and Reasonable

23

Getting to Know the Customer

24

EdwardApplication Engineer

“There is no ‘One Size Fits All’ with Independent Software Vendors (ISVs). Every ISV has a different environment, architecture and customer needs.”

Ed has been working in this role for 4 years. He was a SW Engineer before that for 10 years. His work focuses on the implementation of

AMT capabilities. Ed’s primary role is assisting ISV engineers in implementing specific features by customizing a solution for their

given environment. Much of his day is spent troubleshooting issues and writing new code to test. Usually, Ed travels to the ISV and

spends time face to face working with their implementation team. Given the economic climate, he has made changes to the way he interacts with his customers. Most of his interaction with ISVs is

done over the phone and via email.

Goals:• Simplify integration for ISV

partners• Solve issues fast• Demonstrate AMT value

Values:• Good customer relationships• Flexible architecture• Good documentation

Obstacles:• Each ISV requires custom solution• Troubleshooting issues remotely• Translating code to meet ISV needs

Design Implications:• Design should demonstrate how a feature

could be integrated--represent believable user experience

• Vanilla design = palatable for all potential customers

• Design should avoid appearing as a competing product

Accountability to the Customer

Simplified Metrics

27

Per customer survey

Source: Internal team survey

95%

5%

Team Collaboration

Scrum Waterfall Ambivalent

Source: Internal team survey 77%

15%

8%

Team Cooperation

Scrum Waterfall Ambivalent

Source: Internal team survey

90%

10%

Challenging Work Environment

Scrum Waterfall Ambivalent

Source: Internal team survey

65%

25%

10%

Overall Satisfaction

Scrum Waterfall Ambivalent

“…So I was driving home and contemplating 9 months of pregnancy, or 40 weeks. And I thought, well that

would only be 20 2 week sprints, or 10 4 week sprints!” – one of my employees

32

Last Thoughts

You need to iterate your processes just like your productDon’t fall into Wagile or let cowboy coding take overFocus your test effort on weeding out things that would create a bad user experience with your productDon’t stress on the change

33

References• Various Authors, Exploratory Testing, Wikipedia • Various Authors, Test Strategy, Wikipedia • Various Authors, Scrum (development), Wikipedia • Various Authors, Session-based testing, Wikipedia • The Scrum Alliance, http://www.Scrumalliance.org/ • Ray Arell, Change-Based Test Management, (ISBN: 0971786127) • James Bach, Heuristic Risk-Based Testing, STQE 11/99• James Bach, Risk and Requirements-Based Testing, Computer, June 1999• Ingrid Ottevanger, A Risk-Based Test Strategy, StarEast 2000• Bret Pettichord, The role of information in Risk Based testing, StarEast 2001• James Bach, Risk-Based Testing Troubleshooter, Paper Draft• Erik Petersen, Smarter Testing with the 80:20 Rule, StarWest 2002• Anne Campbell, Using Risk Analysis in Testing, StarEast 2000• Paul Gerrard and Neil Thompson, Risk-Based E-Business Testing• Gregory T Daich, Defining a Software Testing Strategy • Jim Highsmith, Agile Project Management• Ruku Tekchandani, Building a Effective Test Strategy• John Pruitt and Tamara Adlin, The Persona Lifecycle• Pettichord, Kaner, Bach, Lessons Learned in Software Testing, on-line • Jonathan Bach, Session-Based Test Management , http://www.satisfice.com/articles/sbtm.pdf• Daniel Pink, Drive the Surprising Truth about What Motivates Us, (eISBN: 97811001152140)