3 Estimation

SoberIT Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Previous lecture: Processes…



Process Choice

  There are several factors that affect which type of process you should use:   Product type   Business type   Project size   Project initial

uncertainty   Environmental

uncertainty   Organizational culture   …

Plan-driven

Risks ~ -  novelty of methods, tools, application domain - fuzziness of requirements Incremental

Iterative, prototyping

Product size/complexity



Dimensions Affecting Process Model Selection (Boehm & Turner 2003) Personnel

Size Culture

DynamismCriticality

(Percent thriving on chaos versus order)(Number of personnel)

(Loss due to impacton defects)

(Percent level 1B) (Percent level 2 and 3)40

30

20

10 30

0 35

25

20

15

(Percent requirementschange/month)

Agile Plan-driven

5030

105

1

90

70

50

30

10

3

10

30

100

300

Manylives

Singlelife

Essentialfunds

Discretionaryfunds

Comfort

Size (# of personnel)

Culture (% of thriving on chaos vs. order)



T-76.5612 Software Project Management Spring 2010

Tuomas Niinimäki

Department of Computer Science and Engineering Helsinki University of Technology

3: Software estimation



What is estimation?



What is estimation? In mathematics, use of a function or formula to derive a solution or

make a prediction.

Unlike approximation, it has precise connotations. In statistics, for example, it connotes the careful selection and testing of a function called an estimator. In calculus, it usually refers to an initial guess for a solution to an equation, which is gradually refined by a process that generates closer estimates. The difference between the estimate and the exact value is the error.

(Encyclopedia Britannica Concise)



Why would we want to estimate software?



The use of estimates (1/3) Phase / Activity Why The question answered

Strategic Planning Finding out potential application domains

Could we do this more efficiently than the others?

Prioritizing projects Should we emphasize this project over the others?

Feasibility study Cost-benefit analysis, making a bid

Is the project worth doing?

(Adapted from Boehm, 2000, Hughes & Cotterell, 2002)




Evaluation of suppliers’ proposals

Evaluating project cost How much should we pay for this project?

Evaluating supplier’s proposals

Is this bidding realistic?

Has the supplier understood the requirements?

Project Planning Budgeting How much is the project going to cost?

Resourcing and Scheduling

How long is the project going to take?

How does the resourcing decisions affect it?





System specification

Requirements analysis How long does it take to implement a feature/requirement?

Which features will take the most effort?

Project execution Iteration-level plans How much effort do the different tasks take?

Making trade-offs Should I leave this feature out to match the schedule wishes?




What to estimate in software project?




Size Schedule

Effort




Size Schedule

Effort

Time

Resources

Scope



What can be estimated in software?   Software size

  Lines of code?   Number of classes / methods?   Amount of database tables / queries / user views?   Functionality?

  Effort   Person-hours

  Schedule   Calendar-days or months



Derived estimates (examples)   Software size

  Potential / probable number of defects   Probable number of change requests   …

  Effort   Project cost   Team size

  Schedule   Suitable number of iterations / milestones



Project cost drivers (from COCOMO81)

… or even better,

Use the

relevant cost drivers

for

your project /organization



How to estimate software?   Estimation process in a nutshell:

  Provide the estimates in ranges   Periodically refine the ranges to provide increasing precision as

the project progresses (McConnell, Rapid Development, 1996)

Estimate the size of the product Estimate the effort Estimate the schedule

1 2 3



Problems in software estimation (1/6)

  Basic problem: Predicting the future by looking into the past

http://www.flickr.com/photos/wabbit42/3691518084/ CC BY-NC-SA 2.0



Problems in software estimation (2/6)   A lack of good historical information

  Does the historical information have such metrics that distinguish the previous projects from each others?

  Does the historical data contain enough cases to make a reliable estimation?

  Can these metrics describe the project you are estimating?   Do we have the right metrics to measure the project environment?



Problems in software estimation (3/6)   A lack of information on the project to be estimated

  Most influential decisions are made in the early phases of project, based on inadequate information

  Important information is not available   ... or there simply is not enough time to collect it



Problems in software estimation (4/6)   Estimates are done sloppily

  ”If they cannot be done perfectly, why pay attention to them?”

  There can be a order-of-magnitude difference between a careful and a sloppy estimate

  There are no “final estimates” - they can (and should!) be updated whenever new information becomes available



Initial product

definition

Approved product

definition

Requirements specification

Product design

specification

Detailed design

specification

Product complete

1.0×

0.25×

4×

2×

0.5×

1.5×

0.67×

1.25×

0.8× 1.0×

0.6×

1.6×

1.25×

0.8×

1.15×

0.85×

1.1×

0.9×

Project Cost (effort and size)

Project schedule



Problems in software estimation (5/6)   Estimates are not updated

  Estimating is done only in the beginning of the project, based on incomplete information

  New estimates are not done, but the old estimates are still followed

  As new information becomes available, estimates should be updated

  Estimates can be used in various stages of software project  e.g. size estimation for estimating the number of bugs



Problems in software estimation (6/6)   Estimates are not followed, respected or trusted

  An estimate should not be an opinion  An opinion can be overruled by your superior

  An estimate is not automatically a commitment  ... but may be considered as such

  Justify and criticize the estimates   Accept uncertainties   Visualize probabilities



Estimation in practice



Tools for estimation   Work breakdown structure   Product breakdown structure   Top-down estimation   Bottom-up estimation



Work breakdown structure   Work Breakdown Structure (WBS) is a graph (a tree) of activities

done during the project   Each activity is broken down into tasks, until desired level of

detail is reached   A task must contain work related only to their parent activity   The top-level activity (= project) must contain 100% of work

in the project  For each node, the sum of work shares in its sub-activities

must be 100%   A desired level of detail for tasks is usually the size of task

that can be allocated to a single person  Maximum size is usually 10-20 hours



Example: Work breakdown structure

Project

Specification Implementation Testing Delivery

Eliciting requirements

Analyzing requirements

Writing req. specification doc.

Creating architecture plan

Implementing module A

Implementing module B

Implementing module C

Integrating modules

Testing module A

Testing module B

Testing module C

Integration testing

Acceptance testing

Installing software

Training



Product breakdown structure   Product Breakdown Structure (PBS) is a similar construct as

WBS, but is based on the structure of product rather than the type of activities

  Lower levels in PBS describe the structure of item on level above   The top level of PBS is the product   Second level may consist of the major components of the

product   Lowest level should contain items that are “comprehensible”



Example: Product breakdown structure

Product

Documentation Software

Project plan

Requirements specification

Architecture specification

User manual

Module A

Module B

Module C

Main module

Class A_x

Class A_y

Class A_z



Top-down estimation   Idea: Allocate proportions of an effort estimate to different

activities of the project   Inputs: Main project activities (high level design) and

earlier project data on efforts spent for activities   Outputs: Rough work breakdown with corresponding effort

estimates   Advantages

  Takes into account integration, configuration management and documentation costs (e.g. algorithmic models seldom take)

  Disadvantages   Underestimating the difficult low-level technical problems



Example: Top-down estimation

Project



Analyzing requirements Writing req.

specification doc. Creating

architecture plan




Integrating modules

Testing module A Testing

module B Testing

module C Integration

testing Acceptance

testing

Installing software

Training

100 % 800 h




Project





architecture plan




Integrating modules


module B Testing


testing Acceptance

testing

Installing software

Training

100 % 800 h

20 % 160 h

40 % 320 h

30 % 240 h

10 % 80 h




Project





architecture plan




Integrating modules


module B Testing


testing Acceptance

testing

Installing software

Training

100 % 800 h

20 % 160 h

40 % 320 h

30 % 240 h

10 % 80 h

50 % 80 h

30 % 48 h

10 % 16 h

20 % 16 h



Bottom-up estimation   Idea: Calculate the total effort from the sum of effort

estimations of single tasks (by analogy estimation or expert judgement)

  Inputs: Detailed work breakdown (tasks 1-2 weeks for 1 person), descriptions of factors affecting production

  Outputs: Effort estimates on individual tasks, total effort   Advantages

  If the task division is accurate, the model may provide the most accurate estimates

  Disadvantages   Underestimates costs of system level (e.g. integration and

documentation)   Needs detailed breakdown of project tasks



Example: Bottom-up estimation

Project





architecture plan




Integrating modules


module B Testing


testing Acceptance

testing

Installing software

Training

60 h




Project





architecture plan




Integrating modules


module B Testing


testing Acceptance

testing

Installing software

Training

60 h

80 h

45 h

68 h




Project





architecture plan




Integrating modules


module B Testing


testing Acceptance

testing

Installing software

Training

60 h

80 h

45 h

68 h

252 h




Project





architecture plan




Integrating modules


module B Testing


testing Acceptance

testing

Installing software

Training

60 h

80 h

45 h

68 h

252 h

68 h

68 h

68 h

68 h

20 h

20 h

20 h

20 h

20 h

10 h

60 h

272 h 100 h 70 h

695 h



Estimation techniques




  Estimation by analogy   Expert judgment   Agile estimation   Algorithmic models

  Albrecht & MarkII function points   COCOMO 81 and COCOMO II



Estimation by analogy (1/2)

  Idea: Finding similar cases to the target project   Inputs: Earlier projects, each described with p features

 Outputs: Project size, effort estimate, schedule estimate, (estimation error)



Example: Estimation by analogy

  For new project:   Lines of code?   Complexity?   Platform experience?   Domain experience?

Proje

ct d

escr

iptio

n

Require

men

ts

Desig

n

Imple

men

tatio

n

Testin

g

Total

Calen

dar m

onths

Team

siz

e

Funct

ion p

oints

Lines

of c

ode

Comple

xity

Platfo

rm e

xper

ience

Domai

n exp

erie

nce

Project time tracking software 41 246 246 287 820 6.3 1 140 8300 Very low Nominal Very highCustomizable user register module 25 50 138 37 250 3.2 1 40 2300 Low High High

Mobile calendar and email application 525 1575 700 700 3500 12 3 310 22000 Nominal Very low NominalIntranet product 237 316 553 474 1580 11 1 260 18200 Low High NominalWeb survey tool 105 126 84 105 420 2 2 92 5200 Nominal High Low

Finding similar earlier projects

Deriving estimate (inter/extrapolation) from actual outcomes



Estimation by analogy (2/2)   Advantages

  If done systematically, the estimate can be justified to project stakeholders

  If a good analogy is found, the estimation can be fairly accurate   Tool support (e.g. ANGEL, http://dec.bournemouth.ac.uk/ESERG/ANGEL/)

  Disadvantages   The feature data on earlier projects is limited   There may not be an analogous project in the project data set   Choosing the right features for finding analogous project   The significance of many features only becomes visible afterwards

It might not be documented systematically even in earlier projects   By definition, no project is exactly equivalent



Expert judgment (1/4)   Idea: Ask the required estimate from persons that have previous

experience on some parts of the problem   Inputs: Descriptions of problem domain, technology or other

affecting factors   Outputs: Project size, effort estimate, schedule estimate (in all

varying metrics)

  Dominant technique for effort estimation (Jorgensen, 2004)



Expert judgment (2/4)   Who are the experts?

  Your own developers   Domain experts   Technology experts   Other project managers

  Non-technical people may be better at estimating than technical people   Skilled technical personnel tend to be overoptimistic

  Varying background (employment, education, domain experience) of estimators can improve the estimates   Different backgrounds yield different estimation strategies



Expert judgment (3/4)

  In practice expert judgment estimates are derived from analogies to earlier projects   Experience on similar projects is crucial   Require justification for estimates, so that they can be

reviewed



Expert judgment (4/4)   Advantages

  Cheap and fast   Easy to adapt to changing situations (a judgement can be made

based on whatever information is available)   If developers are making the estimates, they are more committed to

them   More accurate than algorithmic methods, if the experts have good

domain knowledge and the project is of low uncertainty   Disadvantages

  The analogies or evaluation criteria cannot be easily made visible How to distinguish good estimates from bad ones?

  Underestimation of large tasks and overestimating small   Can be strongly biased by information that is irrelevant for the

estimates (e.g. schedule constraints)



Agile estimation (Scrum) (1/5)

  Time-paced (heartbeat - sprint - release project - strategic)   Time is fixed, scope can change   User requirements come as “user stories”   “Things to do” (tasks) are stored in product & sprint backlog

Strategic Release Management

Release Project Cycle

Sprint 1 month

3 months



Agile estimation (Scrum) (2/5)   Each feature (a “user story”) is assigned a number of points

  This number represents how hard / time consuming it is to complete this feature

  What do these numbers mean?   Where do these numbers come from?

Story A 3 pt

Story B 5 pt

Story D 7 pt

Story C 4 pt



Agile estimation (Scrum) (3/5)   They do not mean much individually (they are relative)   They come out from team “consensus”

  Delphi method, planning poker

I guess A would be quite simple to do (A = 3)

I think B is much more work than A

(B = ~2 x A)

I agree, but I think D is the hardest one

(D ~ A + B)

Story A 3 pt

Story B 5 pt

Story D 7 pt

Story C 4 pt



Planning poker 1.  At the start of planning poker, each estimator is given a

deck of cards.   Each card has written on it one of the valid estimates (e.g.

1,2,3,5,8,13,…)

2.  For each user story or theme to be estimated, a moderator reads the description.   The product owner answers any questions that the

estimators have.

3.  After all questions are answered, each estimator privately selects a card representing his or her estimate. Cards are not shown until each estimator has made a selection   If estimates differ, the high and low estimators explain their

estimates.

4.  After the discussion, each estimator re-estimates by selecting a card.

5.  Continue the process as long as estimates are moving closer together http://www.flickr.com/photos/tomnatt/ CC BY-NC 2.0



Agile estimation (Scrum) (4/5)   After a few sprints

  The story points assigned to the user stories start to stabilize   You may start counting the story points implemented per

sprint  This measure is called velocity  Velocity tells how many story points can be completed on

average during a sprint  However, velocity is not a team performance metric

Story A 3 pt

Story B 5 pt

Story D 7 pt

Story C 4 pt

Sprint 1: planned scope = 8 pt



Agile estimation (Scrum) (5/5)   Velocity tells how many story points can be completed on

average during a sprint   Velocity can be used to estimate the amount of work for a

new sprint

Story A 3 pt

Story B 5 pt

Sprint 1: implemented => velocity = 8 pt Sprint 2: “maximum” story points = 8 pt

Story D 7 pt

Story C 4 pt

Backlog: sum of points = 11 pt > 8 pt



Agile estimation: Summary   Advantages

  Simple and comprehensible process   Easy to set up and maintain   Empowers developers (helps in team&project commitment)   Self-calibrating

  Disadvantages   Needs active participation, self-organization, motivation   It can be difficult to give reasoning to the story points   It usually takes a number of iterations to stabilize   Long-term estimation is difficult

  Stories are created   and story points assigned “just-in-time”



Algorithmic models   Based on an algorithm (procedure of calculation)

  Model based on (non-)linear regression  Research on software engineering

  Weights and factors derived from past project metrics

  Function point methods are used to estimate software size   Constructive cost model (COCOMO) is used to estimate effort

based on estimated software size   See course material (Estimation of Software) for more details

and examples



Function point methods (1/2)   Idea: Quantify the size of programs independently of programming

languages   Function point methods are based on historical data

  Mathematical model is similar as in (algorithmic) estimation by analogy, but the granularity is finer

  Function point methods should be calibrated to each organization   Even though they give one number, they are not necessarily more

(or less) accurate than other estimation techniques

  Inputs: ”External user types” (Albrecht) or ”transactions” (MarkII)   Outputs: Project size (in function points)



Function point methods (2/2)   Advantages

  Can be made in early stages of project (e.g. based on requirements or a detailed problem description)

  Independent of implementation details or the person estimating   Tool support

  Disadvantages   Difficult to apply for application domains which are hard to describe with

external user types or transactions   Do not take development process, organization or technology into account

(Apply corrective multipliers for deriving person-hours from function points, e.g. COCOMO or data on earlier projects)

  Need to be calibrated for each organization   May give false sense of accurate estimates



Albrecht function points (1/3)   Based on five major components - the external user types

  External input type: input transactions from user that update internal files

  External output type: transactions that output data to user   Logical internal file type: data storage used by the system   External interface file type: input/output between different computer

systems   External inquiry type: transactions initiated by users that do not

update the internal files



Albrecht function points (2/3)   Function points are calculated for each activity in the software,

including   functionality   reports   data storage

  The total function points for a system is the sum of function points for all activities



Albrecht function points (3/3)   Steps to calculate function points for an activity:

1. Identify the external user type for the activity 2. Identify the record types used in the activity

  Record types could be modeled as classes in OOP   Note that you might need to take a look at other activities to fully

determine the record types used in this activity 3. Identify the data types

 Data types could be modeled as attributes in OOP 4. Determine component complexity multiplier based on external user

type and the amount of record and data types involved

  Identifying record and data types requires training to do properly, and can be subjective in some cases



MarkII function points   MarkII function points are calculated for activities modeled as

transactions:   Input data is data provided by the user   The system then processes the input data, and (possibly)

manipulates some internal data (= entity types)   The system outputs data to user

  Calculating the total function points for the system is done by calculating the sum of function points of the transactions in the system



Constructive cost model (COCOMO)

  Idea: Quantify the effort required based on a size estimate   Inputs: ”Lines of code” (COCOMO 81 & II) or ”function

points” (COCOMO II), ”cost multipliers”, ”scale factors”   Outputs: Effort estimate (in person-months = 152 person-hours)

  COCOMO II dataset is updated continuously, software support is available   Possibly the biggest and most descriptive dataset on software

projects



Constructive cost model (COCOMO)   The basic model of COCOMO 81 is based on the equation

effort = c x sizek   effort = person-months   size = thousands of delivered source code instructions   c,k = constants (depend on the system type)

  System types are classified as follows:   Organic mode: relatively small team, highly familiar environment,

flexible interface requirements   Embedded mode: very tight constaints on the system   Semi-detached mode: characteristics between organic and embedded

mode



Constructive cost model (COCOMO)   The intermediate model of COCOMO 81 takes development

environment into account via 15 cost drivers, resulting in equation

effort’ = effort x dem = c x sizek x dem   effort’ = effort estimate adjusted by cost drivers   effort = original effort estimate from the basic model   dem = development effort multiplier

  The 15 cost driver coefficients are multiplied to get the development effort multiplier   Each cost driver is evaluated on scale (very low, low, nominal, high,

very high, extra high)   Values for each cost driver are determined from past projects



Constructive cost model (COCOMO)

  Advantages   Independent of the estimator (provided that the scale factors can be

determined objectively)   An effort estimate can be derived quickly if the needed information is

available

  Disadvantages   Underlying assumption that lines of code or function points are easier

to calculate than the effort needed   A company needs a huge dataset of its own projects to calibrate all

the scale factors   Risk: you get ”an exact number”



What techniques are actually used? Technique McAulay

1987 Heemstra

1989 Wydenbach

1995

Expert based

Expert Judgement 26% 86%

Intuition and experience 85% 62%

Estimation by Analogy 61% 65%

Model based

Algorithmic Models 13% 14% 26%

Other Price-to-win 8% 16%

Capacity related 21% 11%

Top-down 13%

Bottom-up 51%

Other 12% 9% 0%

(Moløkken et al.: A Survey on Software Estimation in the Norwegian Industry, 2004)



Effort estimation best practices (1/3)   Allow time for making the estimate, and plan it   Use documented data from previous projects   Use developer-based estimates   Estimate by walk-through   Estimate at a low level of detail   Don’t omit common tasks   Use software estimation tools   Use several different estimation techniques, and compare the

results   Monitor and adapt estimation practices as the project progresses

(McConnell, 1996)



Effort estimation best practices (2/3)   Reduce situational and human bias

  Find estimation experts with highly relevant domain background and good estimation records

  Ask the evaluators to justify and criticize their estimations

  Assess the uncertainty of the estimate   Use plus/minus qualifiers or value ranges to indicate the direction of

uncertainty or estimation range   Quantify risks – explicate the reasons of uncertainty

  Provide estimation feedback and training opportunities   Provide feedback on estimation accuracy and development task

relations   Provide estimation training opportunities

(Jørgensen, 2004)



Effort estimation best practices (3/3)

  Use several estimation techniques and compare them   If they converge, you are probably on the right track   Find out why the estimates are different

  Delphi method - Combine several expert opinions   Ask each expert to make an estimate independently   Then let each of them present their estimate and reasoning   Based on this discussion, let them adjust their estimation   Iterate until the estimates converge

  Ask several different estimates – optimistic, probable and pessimistic – and compare them



Exercise 1 best practices   Start early!   Read the exercise carefully!   Read the additional material (especially Estimation of Software)   Estimation is sometimes subjective

  Don’t get frustrated!   Sometimes you just have to assume   Remember to give reasoning and make your process visible

  Each question is graded separately   Don’t give up!

  Keep track of the time you are spending and give feedback!



Thank you.

Questions?

Tuomas Niinimäki

[email protected]



Examples



Example: Albrecht function points   Actual requirement from exercise 1: List category products: This screen lists the product, its name, price, short description,

thumbnail image,variations (size, colors and materials) and their stock availability. When clicking a product name or image, a detailed product data screen is activated (see FR3). The user is also able to make an order through this screen (see FR4).

  Step 1: Identify the external user type   User type is external inquiry, as no data structures are updated.

  Step 2: Identify record types   Category, Product, Customer, Discount   Note that you may have to look into other requirements to identify

all records that are present - in this case, the Discount type is stated in a different requirement (not shown here).

  Step 3: Identify data types   categoryName, productName, price, shortDescription, smallImage,

size, color, material, availability



Example: Albrecht function points   Step 4: Determine component complexity multiplier based on

external user type and the amount of record and data types involved   External inquiry with 4 record types, 9 data types   External inquiries use whichever complexity is higher,

external input or external output complexity   In this case, both give a high complexity   For our requirement, the multiplier for high external

inquiry complexity is 6; therefore, we get 6 adjusted function points for this requirement!



Example: MarkII function points   Actual requirement from exercise 1:

List category products: This screen lists the product, its name, price, short description, thumbnail image,variations (size, colors and materials) and their stock availability. When clicking a product name or image, a detailed product data screen is activated (see FR3). The user is also able to make an order through this screen (see FR4).

  Step 1: Identify input types, entity types and output types   Input types: categoryName   Entity types: Category, Product, Customer, Discount   Output types: productName, price, shortDescription, smallImage,

size, color, material, availability   Step 2: Calculate function point amount using weights

  0.58*1 + 1.66*4 + 0.26*8 = 9.30 function points.



Example: Constructive cost model   Using COCOMO 81 to estimate effort:

1. Calculate function points for the system   Example: FP = 19

2. Calculate source lines of code for the system based on the function points   Example: Development in Java, thus SLOC = 60 x 19 = 1140

3. Apply the basic model for nominal effort estimate   Example: Organic model (c=2.4,k=1.05) => effort = c x sizek =

2.4 x (1.140)1.05 = 2.75 person-months 4. Apply the development effort multiplier

  Example: Other cost drivers at nominal level, but programmer capability (PCAP) and operating system experience (VEXP) is high: dem = 0.80 x 0.90 = 0.72 => effort’ = 0.72 x effort = 1.98 person-months

3 Estimation

Documents

Transcript of 3 Estimation