COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.
-
date post
22-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.
![Page 1: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/1.jpg)
COMP8130 and 4130 Adrian Marshall
8130 and 4130
Test Management
Adrian Marshall
![Page 2: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/2.jpg)
COMP8130 and 4130 Adrian Marshall
Overview
• Principal Testing Activities• Testing Problems• Scoping, Scheduling and Estimating• Planning• Documenting• Controlling• Test Teams (and independence)
![Page 3: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/3.jpg)
COMP8130 and 4130 Adrian Marshall
Principal Testing Activities
Planning• scope, approach, resources, budget, documentation set, roles &
responsibilities, schedule
Preparation• test documents, tools, environments, training
Execution• developers, systems integrators, users, external specialists, Independent
V&V agents
Reporting• informal, formal, automated
• defects, progress, resource usage, issues & risks….
![Page 4: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/4.jpg)
COMP8130 and 4130 Adrian Marshall
Testing Problems
Modern software systems present the following potential problems to testing:
•Large, complex and expensive distributed processing / multi-tier systems are common•Lack of a single end-to-end solution development owner•Variability of component maturity and quality•Variability of supplier capability and workmanship•Significant component interface and integration problems
![Page 5: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/5.jpg)
COMP8130 and 4130 Adrian Marshall
Testing Problems
Modern software systems present the following potential problems to testing:
•Large user populations, and large numbers of concurrent users•Unpredictable user loads which are often driven by uncontrollable circumstances•Enterprise wide implementation of software systems creates critical business dependency on systems availability ….
![Page 6: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/6.jpg)
COMP8130 and 4130 Adrian Marshall
Testing Problems
Modern software systems present the following potential problems to testing:
•Significant component interface and integration problems•Large user populations, and large numbers of concurrent users•Unpredictable user loads which are often driven by uncontrollable circumstances•Enterprise wide implementation of software systems creates critical business dependency on systems availability ….
![Page 7: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/7.jpg)
COMP8130 and 4130 Adrian Marshall
Testing Problems
Problems can occur in:•Quality focus;•Test scope;•Test priorities;•Test budget;•Test schedule;•Test skills and experience;•Test documentation;•Communications.
![Page 8: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/8.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
How do we scope the testing?• Understand the system to be tested
(requirements)
• Understand the risks
• Understand the development process
• Understand the technology
• Understand the business objectives
![Page 9: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/9.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Where do we get our requirements from?
Operational requirements documents• User feedback from prototypes, field trials, site
visits• Existing systems and their documentation
Customers:• System specification(s)• Statement of work, • Contract• MIL, ISO or AS Standards• Customer Furnished Information
![Page 10: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/10.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Where do we get our requirements from?
Specifications:
• Interface Specifications
• Product Requirements Specifications,
• Marketing Specification
• Drawings
• Technical Specifications and standards
![Page 11: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/11.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Where do we get our requirements from?
Corporate:
• The bid
• Marketing Specification
• Company and project policies,
• standards and procedures
• Existing design
![Page 12: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/12.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Analysing the risks
• Hazard identification• Business risks
• System risks
• Technology risks
• Management risks
• Risk sizing
• Risk management / mitigation
![Page 13: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/13.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Understand the development process (and influence it?)
• Incremental
• Iterative
• Use Case definitions
• Integration methods
• Phasing of builds
![Page 14: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/14.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Understand the technology
(examples):
• PKI
• Embedded systems
• Windows / Unix
• Smart-card
• EFTPOS
• jpeg
![Page 15: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/15.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Understand the business objectives
(examples):
• Time to market critical
• Phased delivery
• Low price, high volume sales
• Security, confidentiality, IP protection
![Page 16: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/16.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Scoping.
Finally…..
…….determine the required tasks to fulfill the scope…..
Inspections, unit, integration, system, acceptance, performance, reporting
![Page 17: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/17.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Scoping exercise:Consider requirements, risks,
development process, technology and business objectives for the development of:
a. R&D program for biometric authentication;
b. Game based on the latest movie;c. Fuel management system for the
A380
![Page 18: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/18.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Estimating
• Understand the scope
• Understand the size of the system (may use COTS products) and the size of the development
• Understand the complexity of the system
• Historic data for similar testing scope
• Gut feel
• Rule of thumb
![Page 19: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/19.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Estimating - Useful numbers:
Overall testing effort (all activities) = 30% of the Test planning = 15% of total test effort
Designing and documenting tests cases = 10% of total test effort
Test procedures and test execution = 70% of total test effort
Test reporting = 5% of total test effort
![Page 20: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/20.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Estimating
• Evaluate estimates• Record actual effort (test metrics)
• Don’t be too granular (testers aren’t accountants or lawyers)
• Refine estimates
• Refine schedule
![Page 21: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/21.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Scheduling• Identify the sequence of events
• Identify the critical path
• Communicate the draft schedule Development team Project manager
• Refine the schedule (communicate and changes to the risk profile)
![Page 22: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/22.jpg)
COMP8130 and 4130 Adrian Marshall
Scoping, Scheduling and Estimating
Scheduling and estimating exercise (small groups):• Three month schedule from start of
development to release
• First build due at the end of the second month
• 12 developers working fulltime
• Estimate the effort, size of the test team, and present a simple schedule
![Page 23: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/23.jpg)
COMP8130 and 4130 Adrian Marshall
Planning
Building the test plan:
• Scope, Schedule and Estimate
• Explain the test execution strategy
• Allocate roles and responsibilities
• Outline the testing schedule
• Identify test resource requirements
• List assumptions, issues and risks
Approving the test plan
• Have appropriate stakeholders authorise / sign the plan….
![Page 24: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/24.jpg)
COMP8130 and 4130 Adrian Marshall
Planning
• Planning should be done very early - be part of the project bid / project initiation team
• Obtain clear solution success / acceptance / quality criteria to enable identification of test focus areas
• Identify & evaluate risks and create a testing strategy which helps to manage them
• Produce a professional and defendable Test WBS
• Be appropriately independent
• Act as a support service not an authority figure
• Keep testing team and ‘customers’ informed
• Be involved across the project lifecycle….
![Page 25: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/25.jpg)
COMP8130 and 4130 Adrian Marshall
Documenting
Why do we document?
What do we document?
How do we document?
Master Test PlanEntry and exit criteriaUnit level testing documentationDocumentation for subsequent phasesPlans are not static… they must be constantly evaluatedTest CasesTest ProceduresTest Reports
![Page 26: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/26.jpg)
COMP8130 and 4130 Adrian Marshall
Controlling
What is needed to control the activities?
Metrics• Rate of test design
• Rate of execution
• Product quality
• Product stability
Reporting
![Page 27: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/27.jpg)
COMP8130 and 4130 Adrian Marshall
Test resources
Test Teams
Test Environments
Test Tools
![Page 28: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/28.jpg)
COMP8130 and 4130 Adrian Marshall
Test resources
Test Teams
• Independence
• Identify skills / experience
• Hiring / contracting
• Motivating / retaining
![Page 29: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/29.jpg)
COMP8130 and 4130 Adrian Marshall
Test resources
Test Environments
• Procure
• Develop
• Control/manage
![Page 30: COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d805503460f94a64990/html5/thumbnails/30.jpg)
COMP8130 and 4130 Adrian Marshall
Test resources
Test Tools
• Evaluate
• Procure
• Develop
• Validate/verify
• Training