Software Test Estimation
-
Upload
ananya-reddy -
Category
Documents
-
view
225 -
download
0
Transcript of Software Test Estimation
-
8/3/2019 Software Test Estimation
1/26
Nga Huynh
What is the best approach tosoftware test estimation?
-
8/3/2019 Software Test Estimation
2/26
Agenda
Introduction
Why good estimates are important and reasons for poorestimates?
Test Estimation vs Estimation (of any other activities)
Confidence rating
Estimation Methods ways to estimate test effort
Considerations
Questions?
-
8/3/2019 Software Test Estimation
3/26
Introduction
Successful test estimation is a challenge
Difficulty to estimate the software projectdevelopment accurately even harder to estimate fortesting effort
Lack of detailed information about, example Detailed requirements
Organisation's experience with the similar projects in the past
Understanding of what to should be included in a testing effort
-
8/3/2019 Software Test Estimation
4/26
Why good estimates are important and reasonsfor poor estimates?
Why good estimates are important?
Testing is often blamed for late delivery
Testing time is squeezed
It promotes early risk assessment
Reasons for poor estimates? Incomplete/ambiguous requirements
New technology
Skills (Testers & developers)
Project delays
Poor environments
-
8/3/2019 Software Test Estimation
5/26
Estimation vs Test Estimation
Any estimation involves the following: Identify/estimate/assign tasks
Start and finish for each task
Resources and skills
Dependencies (task precedencies)
Test Estimation
No. of iterations/cycles
Availability of the resources (test environment, people etc)
Quality (Software, test environment, test cases etc)
Delivery of the software to be tested Reviews of test artefacts
-
8/3/2019 Software Test Estimation
6/26
Confidence rating
System under test Test Effort Confidence Rating
System A 400 hrs 20 %
System A 350 hrs 80%
Start of the project
More detailed info
-
8/3/2019 Software Test Estimation
7/26
Estimation Methods ways to estimate testeffort
What is the best approach to software test estimation?
Ways to estimate test effort
Guessing
Formula Based
Parkinsons Law versus Pricing to Win
work expands to fill the time allocated for it
Minimum amount that we can get away with Consensus Of Experts
Past Project Knowledge
Work Breakdown Structure
MITs Model (MITs = Most Important Tests)
-
8/3/2019 Software Test Estimation
8/26
What is the best approach to software testestimation?
Highly dependent on Organisation
Project
Experience of the resources involved
Criticality of the system Life-critical equipment software
Low-cost computer game Complexity and size of the system
Allocate time for estimation and re-estimation
~ 5% - 7.5% of the testing budget on test estimation(Ross Collard,)
Overhead
More than one method to check each other
Involve the team
-
8/3/2019 Software Test Estimation
9/26
Estimation Methods
Guessing - Finger In Air (FIA) Pure guess/gut feel
Based on experience
past project experiences
expertise of the estimator
Bad method to base our final estimate on
Easily questioned and challenged
Try to delay
Past Project Knowledge
Historical estimation
Estimation based on previous test efforts
e.g. 40 hours, 80 tests (0,5 hrs/test ) 100 tests, 50 hours
Best predictor only if the historical data is available applicable andaccurate
-
8/3/2019 Software Test Estimation
10/26
Estimation Methods
Formula Based 40% of Development Effort
Quick method
Dependent of accurate development effort estimates
System and acceptance test
Function Point Analysis (FPA) Formal technique to estimate size of the system under test
Based on five elements plus adjustment factors (inputs,outputs, queries, files, interfaces etc)
Test Point Analysis (TPA)
Test estimation technique based on FPA
Described in TMAP
TPA takes risk into account
-
8/3/2019 Software Test Estimation
11/26
Consensus of Experts
Useful when unsure ofsystem, e.g. new technologyor application
3-4 knowledgeable peoplefor independent judgement
If the estimates are similarthen take the average
If the estimates are differentthen a consensus must beproduced
-
8/3/2019 Software Test Estimation
12/26
Work Breakdown Structure(micro-estimating)
Identify tasks & activities
Identify dependencies
What tasks must be start/finish before others
Effort & resources, start & end times
Who should do what & when? Schedule
What is the critical path?
Involve the team
-
8/3/2019 Software Test Estimation
13/26
Work Breakdown Structure(micro-estimating)
Phase Historical Value % of Project Preliminary Estimate Adjusted Estimate
1 Project Startup 140 2.6 179 179
2 Early Project Support (requirementsanalysis, etc.)
120 2.2 152 152
3 Decision to Automate Testing 90 1.7 117 -
4 Test Tool Selection and Evaluation 160 3 207 -
5 Test Tool Introduction 260 5 345 345
6 Test Planning 530 10 690 690
7 Test Design 540 10 690 690
8 Test Development 1,980 37 2,553 2,553
9 Test Execution 870 17 1,173 1,173
10 Test Management and Support 470 9 621 621
11 Test Process Improvement 140 2.5 173 -
PROJECT TOTAL 5,300 100% 6,900 6,403
-
8/3/2019 Software Test Estimation
14/26
The Most Important Tests (MITs) Method
Sizing test efforts based on the risk of failure in the
system
Criticality
Complexity
Objective Weighting
Planned Test Effort
-
8/3/2019 Software Test Estimation
15/26
The Most Important Tests (MITs) Method
Function Complexity Criticality Objective
Weighting
Planned Test
Effort
Function A 50 1.75 40% 35
Function B 150 2.25 5% 17
Function C 60 2.00 20% 24
Total Units 76
Complexity determined by countinge.g.
Business decisions
Number of screens involved
Number of data values entered
etc
-
8/3/2019 Software Test Estimation
16/26
The Most Important Tests (MITs) Method
Function Complexity Criticality Objective
Weighting
Planned Test
Effort
Function A 50 1.75 40% 35
Function B 150 2.25 5% 17
Function C 60 2.00 20% 24
Total Units 76
Criticality consider the following:
e.g.
Customer Affected
User Affected
Business Affected
Frequency
-
8/3/2019 Software Test Estimation
17/26
Determining Criticality
Customer
Affected
User Affected Business
Affected
Frequency
High (3) Directly Cannot work Loss of business
Everyday
Medium (2) Could be Work around Loss of money Occasional
Low (1) Not at all User doesntknow
No effect onbusiness
Once only
Total: 7 Average: 7/4 = 1.75
-
8/3/2019 Software Test Estimation
18/26
2420%2.0060Function C
76Total Units
175%2.25150Function B
3540%1.7550Function A
PlannedTest Effort
ObjectiveWeighting
CriticalityComplexityFunction
-
8/3/2019 Software Test Estimation
19/26
Objective Weighting
Purpose:
Allows testers to specify an appropriate amount oftest effort.
Example of Objective Weighting5% - 25% minimal regression testing
30% - 50% thorough regression testing
55% - 75% minimal testing
80% - 100%thorough testing
> 100% rigorous testing
-
8/3/2019 Software Test Estimation
20/26
Planned Test Effort
Planned Test Effort (units) = Complexity x Criticalityx Objective Weighting
Planned Test Effort figures give the proportion oftest effort that will spent in each function/area
Specify what a unit means Hour
Test case
-
8/3/2019 Software Test Estimation
21/26
Determining Criticality Other Criteria
Visibility Number of people who will see a failure
Reliability
Number of faults that have occurred in the past
Tolerability Rating how forgiving end users are to failure
-
8/3/2019 Software Test Estimation
22/26
Considerations... 1(2)
The testing target
Level of testing
Type of test
The quality of the system under test
Test environment setups (test tools)
Test maturity of the organisation
Scope of test requirement
Test engineer skill level
-
8/3/2019 Software Test Estimation
23/26
Considerations... 2(2)
Domain knowledge
Involve the team
Test iterations/Cycles (typical 3-4 iterations)
Defects found in previous testing/to be found
Time to report defects the more time spent onreporting the less testing you will actually do!
-
8/3/2019 Software Test Estimation
24/26
Finally...
Follow up your estimations
A combination of methods to get amore comprehensive and accurate
estimate
-
8/3/2019 Software Test Estimation
25/26
Questions?
-
8/3/2019 Software Test Estimation
26/26
Thank You!