T19 performance testing effort - estimation or guesstimation revised
-
Upload
eurostar-software-testing-conference -
Category
Documents
-
view
58 -
download
0
Transcript of T19 performance testing effort - estimation or guesstimation revised
© 2006, Cognizant Technology Solutions. All Rights Reserved.The information contained herein is subject to change without notice.
By
Narendra Ponnuswamy
Performance Testing
Effort :
Estimation or
Guesstimation ?
Agenda
Performance Testing & Estimation
Pain Areas in Estimation
Divide & Conquer Strategy
Estimation Tips
Approach
Summing it up
2
Performance Testing & Estimation
Performance Testing is the process of determining the speed of an application under heavy user load.
It is an automated testing normally done using Industry standard tools.
“Well begun is half done”
A good estimate is a solid foundation for the project.
But some inevitable questions arise during any
Investment.
3
Triangle Model
Optimal resources to
achieve timely
delivery
Quality should not be
compromised
Cost Benefit and Maximum
Value with least investment
Cost
4
Time Quality
Pain Areas in Estimation
NFR’s (Non Functional Requirements) not properly
defined
Less time planned for Performance Testing
Unplanned additional runs due to performance issues
identified
Frequent code changes causing rework
Building a test environment similar to production
5
“There is nothing so easy to learn as experience
and nothing so hard to apply”
- Josh Billings
Divide & Conquer Strategy
“Divide and Conquer “ Strategy was successfully used
by British and Roman empires to rule the world.
Requirements
Analysis
Performance Testing
Effort
Execution
Delivery
Design and
Development
6
Non
Functional
Requirement
(NFRs)
Analysis
Estimate
Development
Tool
Evaluation
Requirements
Signoff
Requirements
Analysis
Design and
Development
Test
Execution
Delivery
•Requirements
Analysis
•Coordination
with other
teams
•Proof of
Concept if any
•Test Strategy
•Test Plan
•Test Design
•Test Scripts
•Smoke Test
Execution
•Test Execution
•Coordination
with other
teams
•Monitoring and
tuning
•Test
Summary
Report
•Executive
Summary
Report
•Review and
Sign Off
Critical Efforts in Various Stages
7
8
Approach
Level 0
• High level Estimate
• Roles defined
• Includes detailed
Requirement Analysis
and Planning effort
• Includes POC effort
• Tool investment if any
• Refined Hour estimate
• Detailed split up of
estimate for various
stages
• Includes detailed
scripting and test
execution effort
• Refined Hour
estimate if
requirements
change
• Includes additional
scripting and test
execution effort
due to additional
complexities
Benefits
Snapshot
Benefit Snapshot
•High level
Estimate
• Proof Of
Concept
effort
•Tool
Investment
if any
• Refined Hour estimate
• Detailed split up of
estimate for various
stages
• Includes detailed
scripting and test
execution effort
•Helps Project
Manager plan for
Performance
Testing effort
•Requirement
Analysis and
Planning effort
well defined
• Informs
Stakeholders with
detailed effort split
up
•Opportunity to
reduce scope (if
there are any
budget
constraints )
•Change in
requirements
is reflected
•Helps avoid
last minute
surprises
8
Level 1 Level 2
Estimation Tips
10
Which requirements should I consider from the chart below ?
Never be “Penny wise and Pound foolish”
Estimation Tips
11
• Determine application complexity
• Determine the number of Non Functional Requirements
• Identify the number of test cases (Each test case becomes a
script)
• Finalize the number of test runs
• Determine servers that need to be monitored
• Determine Data Setup and Environment Setup effort
Commonly identified risks:
• Frequent application changes impacting initially planned effort
• Unstable environment causing additional effort
I80/20 Rule
12
Tasks that consume more time :
Scripting Activities
Test Execution
Data Setup
Sample display of distribution :
Other Activities
Test Development
Test Execution
Data Setup
13
Performance Testing estimation should be given early in
the project
Listing assumptions in the estimate clearly
communicates the constraints involved.
Bottom Line:
Estimation is not an exact science but it is not an
approximation either.
Lack of a proper technique could make it a
Guesstimation.
Summing it up