Planning, Estimating, and Monitoringgg Progress in Agile … · Planning, Estimating, and...
Transcript of Planning, Estimating, and Monitoringgg Progress in Agile … · Planning, Estimating, and...
Planning, Estimating, and Monitoring Progress in g g
Agile Systems Development Environments
Systems & Software Technology ConferenceSalt Lake CitySalt Lake City
April 2010
Dr. Suzette S. JohnsonNorthrop Grumman Corporation
Agile Engineering and Leadership
© Copyright 2010 Northrop Grumman
Report Documentation Page Form ApprovedOMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.
1. REPORT DATE APR 2010 2. REPORT TYPE
3. DATES COVERED 00-00-2010 to 00-00-2010
4. TITLE AND SUBTITLE Planning, Estimating, and Monitoring Progress in Agile SystemsDevelopment Environments
5a. CONTRACT NUMBER
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Northrop Grumman Corporation,2980 Fairview Park Drive,Falls Church,VA,22042
8. PERFORMING ORGANIZATIONREPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES Presented at the 22nd Systems and Software Technology Conference (SSTC), 26-29 April 2010, Salt LakeCity, UT. Sponsored in part by the USAF. U.S. Government or Federal Rights License
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as
Report (SAR)
18. NUMBEROF PAGES
31
19a. NAME OFRESPONSIBLE PERSON
a. REPORT unclassified
b. ABSTRACT unclassified
c. THIS PAGE unclassified
Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18
Discussion Outline
• Agile Systems Development Introduction
• Scenario– Levels of Planning – Estimatingg– Monitoring Progress
• Summary
• Recommended Readings and References
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman2
Agile Systems DevelopmentAgile Systems Development
Promote rapid delivery of value to customers
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman3
What is Agile Systems Development?
• Agility is a set of demonstrated industry best practices for developing software systemssoftware systems
• Agility focuses on:– Principles and Valuesp– Inspect and Adapt– Constant Commitment to Quality– Focus on the Value StreamFocus on the Value Stream– Team Ownership and
Empowerment
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman4
Agile Principles
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman5 5
Business People and Developers Must Work
Together Daily
The Best Architectures,
Requirements and Designs Emerge From Self-Organizing Teams
NORTHROP GRUMMAN
Continuous Attention to Technical Excellence
Regular Team Reflection on How to
Become More Effective
http:// a!Jilemanifesto.orgf Nt:JRJ'HRDP GRUMMAN
Agile Terminology
Term Definition
P d B kl R i /U S i b l dProduct Backlog Requirements/User Stories to be completed
Iteration Fixed time-box in which development occurs
User Story Similar to a requirement“As a user I want to action so that purpose”
Velocity The number of user story “points” delivered in a iteration
Capacity The hours the development team is available to work within an iteration
Product Burn Down Chart Progress for the release; Focuses on the remaining userProduct Burn Down Chart Progress for the release; Focuses on the remaining user story points
Iteration Burn Down Chart A development team’s progress; Focuses on the remaining hourshours
Product Owner Owns the product backlog, assigns priority to user stories
The Team Cross functional team
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman6
The Team Cross functional team
6
High Level Agile Stages
Planning Estimating Monitoring
Product Roadmap
Iteration Planning
Iteration Execution
VisionCustomer
Needs
Release Planning
Needs
Iteration Demo and
RetrospectiveDelivery
Retrospective
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman7
Agile Scrum Framework Commitment
The work to be done
The Daily Tasks
managed by
Identification of Impediments
Communication
Prioritized by Product Owners
g ythe team System
Development
Feature Demonstration and Retrospective
Inspect and AdaptAn example of an agile
management framework
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman8
Visibility and TransparencyImage from: Mike Cohn, MountainGoatSoftware.com
Reference: Ken Schwaber, www.controlchaos.org
ScenarioScenarioPlanning, Estimating, and Monitoring
© Copyright 2010 Northrop Grumman
Project Scenario: RestEZj
• Duration: One ReleaseP d t D l t f h t l b it f R tE• Product: Development of a hotel website for RestEz
• Size of Project: ~10 persons/team; 10 teams = 100 people• Release cycle is January 4 – March 31Release cycle is January 4 March 31• Six two-week iterations within the Release• Planning for the Release is started prior to January 4th
– Determining high level capabilities for the release– Story Writing workshop (Capability to User stories)
• Release Planning meeting is January 4the ease a g eet g s Ja ua y• Iteration 1 detailed planning is in the afternoon of January 4th
• January 5th is the first day of development
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman10
Agile Metrics During the Releaseg g
• PlanningVelocity– Velocity
– Capacity– Total story points planned for the release – Length of iteration and release– Planned work hours
• Monitoring Progress• Monitoring Progress– Product Burndown– Iteration Burndown– User Stories by State– Tracking Defects– Test Metrics– Actual worked hours
Scenario Walkthrough…
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman11
Planning and Estimatingg g
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman12
Levels of Planning
Vision, Roadmap, Release, Iteration, DailyVision, Roadmap, Release, Iteration, Daily
Iteration 1
Product Backlog(prioritized
requirements by Task
~2 - 6 months
~2 – 4 weeks fixed
Product
business value) Release 1 Story
Story
Task
Story
TaskTaskTaskGoals &User Stories
Daily Stand
Up
Vi i
Roadmap
Iteration 3
Iteration 2
VisionCustomer Needs
Iteration n…
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman13
Product Roadmap
Release 1Room
reservations and payment
User profiles forUser profiles for future visits Release 2
Conference offerings
Online chat support
Release 3Special discountsSpecial discountsLocal information
Release 4Google maps
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman
14
Release Planning: Use Case and User Stories g
Use Case
Capability/ScenarioWritten at the
capability level and can be completed in
one Release
Sum of all the test cases to fulfill the Release level test
criteriaone ReleaseEvery
Epic has validation criteria
Written at a level that the team can complete
in one iterationOwned by the Product
Unit Tests Component testing
Systems TestingDemo to
User Storiesy
OwnerDemo to
stakeholdersEvery story
has acceptance
criteria
Tasks
Tasks written by the Team<=8 hours of work
Based on a definition of “d ”
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman15
“done”Owned by the Team
User Stories
What is a User Story? Estimation
• Functional stories– often based off a scenario of a
use case
• User Story Points– Bigness of the task– Influenced by (a) how hard it is– On large projects a user can be
another system
• Non-functional stories
Influenced by (a) how hard it is (b) how much there is to do and (c) amount of uncertainty
– Estimated by the teamNon functional stories
• Definition of Done– Design, Write tests, code, unit
t t d t ti t
Estimated by the team– Relative values– Total story points for the Release
tests, documentation, etc.
• No credit for partial work –either done or not doneeither done or not done
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman16
Roadmap to Release PlanExample: Hotel Website
Release Plan (User Stories)
Example: Hotel Website
Release 1 Capability 1: Make Room Reservations
How much can we do?
PointsBusiness Value
80 As a vacationer I want to • Test with search on 1 12
TestUser Stories
How much can we do?
80 As a vacationer, I want to search room availability…
• Test with search on 1 room
• Test with search on ti it
12
executive suite….
75 As a vacationer, I want to save my request…
Test 8save my request…
Objective
70 As a vacationer, I want to pay with a credit card…
Test
Objective
16
Objective
Story n…
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman17
Release Plan, Iteration Plan, Daily PlanExample: Hotel Website
Release Plan (User Stories)
Example: Hotel Website
Capability 1: Make Room Reservations
Design Review 4Yesterday I started on the
Release Plan (User Stories) Iteration Plan (Tasks)Points
Hours
The Daily PlanBusiness
Value
80 As a Test with h
12
Test
Design Review 4
Install Baseline 4
ICD Updates 8
interface…. Today I plan to…
The one thing standing in my way…
vacationer, I want to
search room availability…
search on 1 room
Test with search on
Acquire Test Data 8
Code 24
Develop Tests 8
s a oexecutive suite….
75 As a Test 8 p
Run Tests 8vacationer, I want to save my request…
Objective
70 As a Test 1670 As a vacationer, I want to pay with a credit
card…
Test
Objective
16
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman18
Velocity (Based on history)
• Velocity is the amount of work a development team completes in an iteration (story points completed)( y p p )
• Velocity is a range; Look for the high, the low, and the mean.
35
40
45
Velocity for Team A
140
160
Project Velocity
10
15
20
25
30
35
40
60
80
100
120
Story Points
0
5
1 2 3 4 5 6 7
Iteration
T A V l it P j t V l it It ti
0
20
1 2 3 4 5 6 7
Project Velocity per Iteration
Team A Velocity
High: 45 story points
Low: 30 story points
Project Velocity per Iteration
High: 155 story points
Low: 120 story points
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman19
Mean: 37 story points Mean: 137 story points
Determining Team Capacity for an Iteration
• Capacity is the development team members’ available hours
Example for a two-week iterationteam members available hours to work per iteration
• Revisited each iteration
Team Member
Hours per day
Total hours per iteration
Bill 5 50
• Compare planned hours versus actual hours
Bill 5 50
Scott 5 50
Chris 8 80
• Compare team capacity hours to the hours in the iteration
Andy 7 70
Cindy 7 70
Mike 8 80
TEAM TOTAL 40 400
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman20
The Release Plan
• What is in the release plan?Capabilities identified– Capabilities identified
– User stories (functional and non-functional requirements)• Story points and prioritized• Project Teams average about 137 user story points per iteration; for a
release with 6 iterations this is about 900 story points. The scope is 720-930 user story points of work.
T t l b f t i l d (125)– Total number of user stories planned (125)– Total number user story points planned (~910 user story points)– Known or assumed velocity by development team and project team– Planned hours (WBS element)
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman21
Monitoring Progressg g
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman22
RestEZ: Product Burndown for the ReleaseProduct Burndown for the Release
1000
700
800
900
Focuses on the Points Remaining
400
500
600
Stor
y Po
ints
PlannedCompleted
100
200
300
S
0
100
Iteration1 Iteration2 Iteration 3 Iteration 4 Iteration 5 Iteration 6 Iteration 7
• 910 points planned • Initiates discussion• Project team or development team
perspective
• Progress made and work remaining
• Reviewed every iteration
• Can be a “Burnup” chart
• Reports user stories completed against
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman23
• Based on story points• Reports user stories completed against
the plan for the release
A Development Team’s Iteration Burndown
Week1Iteration 0 Tasks Owner Status Mon Tues Wed Thur Fri Mon Tues Wed ThurD i R i S tt C l t d 4 4 4 3 2 0 0 0 0
User Story: As a vacationer, I want to search room availability…
Design Review Scott Completed 4 4 4 3 2 0 0 0 0Install baseline Bill Completed 4 4 4 0 0 0 0 0 0ICD updates Scott Completed 8 8 6 3 3 2 0 0 0Acquire test data Bill Completed 8 5 0 0 0 0 0 0 0Code Scott Completed 24 20 16 14 10 10 3 0 0pDevelop tests Scott Completed 8 8 8 6 4 4 4 3 0Run Tests Scott Completed 8 8 8 8 5 3 4 2 0
1
• Emphasis on effort in hours
• Based on task hours for each story rseach story
• Hours remaining
• Updated daily
Hou
r
• Reviewed daily
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman24
The Iteration Burndown is The Iteration Burndown is for the Teamfor the Team
Tracking Stories by State
• How work is progressing during the iteration• State examples: Opened In Progress Verified Accepted and Completed• State examples: Opened, In Progress, Verified, Accepted, and Completed• Based on total number of stories for the release • May help identify bottlenecks• Verified means all tests have passed on the system• Completed means it is ready for release (i.e. potentially shippable)
Stories by StateCumulative Flow
80100
120140
160
f Sto
ries
CompletedAccepted V ifi d
020
4060
80
009
009
009
009
009
009
009
009
009
009
009
009
Num
ber o
VerifiedIn ProgressOpened
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman25
1/8/
2
1/15
/2
1/22
/2
1/29
/2
2/5/
2
2/12
/2
2/19
/2
2/26
/2
3/5/
2
3/12
/2
3/19
/2
3/26
/2
Reference and for additional information: http://leanandkanban.wordpress.com/2009/04/04/useful-agile-metrics-cumulative-flow/
Testing the User Stories
Testing User Stories (Features)
80100
120
140 Compare the two graphs.Graph 1 indicates a relatively
constant increase in
0
2040
60
80Number of passed tests “passed” user stories.
What about graph 2? 01 2 3 4 5 6
Testing User Stories (Features)
g p
Metrics can help teams identify bottlenecks and
Iteration
100120140
Testing User Stories (Features) yother challenges on the
project.
20406080
Number of passed tests What is the challenge in graph 2?
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman26
01 2 3 4 5 6
Iteration
Defect Metrics
• Defect metrics include:Number of open defects at the end of each iteration– Number of open defects at the end of each iteration
– Number of defects found during systems integration– Number of defects found in production– The type of defects occurring
• Useful in determining root cause
• May give indication as to what processes need to change• May give indication as to what processes need to change
• Agile teams strive for “zero” defects– Part of the Definition of Done is “no critical defects against the story” or the– Part of the Definition of Done is no critical defects against the story or the
story isn’t “done”
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman27
Summary
• Agile systems development environments:Emphasize ongoing iterative development with completed demonstrable– Emphasize ongoing iterative development with completed, demonstrable functionality at the end of every iteration
– Embrace practices that support changing requirements and mission needsE i lti l l l f l i– Engage in multiple levels of planning
• Valuable measures for estimating and monitoring progress:– Velocity (scope)– Velocity (scope)– User stories planned versus user stories completed (progress)– User stories added to a release; not part of the original plan
T k h b d (d il )– Task hour burndown (daily progress)– Tests passed (what’s working)– Hours planned and hours worked
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman28
Recommended Reading and References
Creating Adaptive BusinessesAd ti E t i St H k lAdaptive Enterprise Steven Haeckel
Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era
Marion, McKelvey, & Uhl-Bien. (2007). Leadership Quarterly, 18(4), 298-318.
Agile Development PracticesAgile Project Management with Scrum Ken Schwaber
Agile Software Development: Adaptive systems principles and best Meso and Jain g p p y p ppractices
Agile Software Development with Scrum Ken Schwaber and Mike Beedle
Agile Testing Lisa Crispin and Janet Gregory
Implementing Lean Software Development Poppendeick
MetricsAgile EVM – Earned Value Management in Scrum Projects Sulaiman, Barton, Blackburn
Agile Metrics (Agile 2009 Conference proceedings) Dan Rawsthorne, Danube
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman29
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman30
IVORTI-IROP GRU~~AIV
The Need for Change
• Shortened product life cycle and technological advancements
• Shortened development times
• Decreased time to market• Decreased time-to-market
• Complexity of systems
• Large program failures
• Desire for improved transparency of progressDesire for improved transparency of progress
• Reduce risk
© Copyright 2009 Northrop Grumman Copyright 2009 Northrop Grumman CorporationCopyright 2010 Northrop GrummanCopyright 2010 Northrop Grumman31