Testing & Software Metrics CM602 Effective Systems Development.
-
Upload
morgan-gray -
Category
Documents
-
view
231 -
download
5
Transcript of Testing & Software Metrics CM602 Effective Systems Development.
![Page 1: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/1.jpg)
Testing& Software Metrics
CM602 Effective Systems Development
![Page 2: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/2.jpg)
Objectives
In this lecture we will consider Validation, verification and testing in
UML Objectives of Software Metrics Planning a Metrics Programme
What are metricsGQM and other techniques“Dos and Don’ts”
![Page 3: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/3.jpg)
Validation and Verification
Validation refers to a set of activities that ensure that the software that has been built is traceable to customer requirements “Are we building the right product ?”
Verification refers to the set of activities that ensure that software correctly implements a specific function. “Are we building the product right ?”
(Pressman, 2000)
![Page 4: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/4.jpg)
Validation in UMLUse the design walkthrough technique to check: classes (including interface objects) and class
diagrams interaction sequence diagrams collaboration diagrams state diagrams data dictionary (possibly)
Check for consistency between models
Check for completeness and redundancy
Check for correctness of all OO models
![Page 5: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/5.jpg)
Difficulties in ValidationIt is difficult to know exactly what is being looked for in validation
Looking for anything that might make the final system less useful for the customer / user than the system should be.
To do validation effectively need to involve the customer
![Page 6: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/6.jpg)
Software ValidationDemonstrate conformity with requirements.Check for: all functional requirements satisfied all behavioural characteristics are achieved all performance requirements are attained documentation is correct
Two possible conditions after testing conformance to requirements non-conformance: create a defect list
![Page 7: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/7.jpg)
Verification in UMLTesting of one explicit thing (the product) against another (the specification)Verify that the use cases described in the UML model satisfy the requirements described in the requirements specificationVerify that the classes are capable of providing the use casesVerify that the code corresponds to the classes in the design
![Page 8: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/8.jpg)
OO testing strategiesUnit testing – Class (or Component) Level Focuses on the operations and the state
behaviour of the class.
Integration testing – Subsystem Level Test that combinations of components work as
expected
Validation testing – System testing focus on user interactions and user visible actions
of the system
![Page 9: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/9.jpg)
Unit Test CasesTest cases should focus on the class (or component).
Test cases primarily derived from sequence and class diagrams.
Need to consider the implications of inheritance in test cases
White box test cases – written with knowledge of the internal workings of the test item.
![Page 10: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/10.jpg)
System Testing – Done by Developers
Exercise the whole system using use cases as the basis for many of the test scripts.Also includes: Recovery testing Security testing
Stress testing quantity, frequency, volume, growth
Performance testing run time of software, response times
![Page 11: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/11.jpg)
Acceptance Testing – Done by Users
Exercise the whole system using use cases as the basis for many of the test scripts.Also includes: Recovery testing Security testing
Stress testing quantity, frequency, volume, growth
Performance testing run time of software, response times
![Page 12: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/12.jpg)
Metrics: Your starter for ten.
Two programmers are working for the same company, though not on the same project. In one week of work, Fred produces half as much again as Jim, measured in lines of source code.What do you conclude from this?
![Page 13: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/13.jpg)
Software Metrics
Measurement is defined as a process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to describe them according to clearly defined rules. (Fenton N 1994)
![Page 14: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/14.jpg)
Direct and Indirect
We measure by direct (most accurate) and indirect measurements.Many quality factors are measured indirectly.
Direct or indirect?
Speed = Distance*time
![Page 15: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/15.jpg)
Internal/External Attributes
Typical internal attributes:- Time, effort, and number of events of a
particular type which occurred during the process, eg number of defect reports raised
Others include size, modularity
Typical external attributes Cost, controllability and stability,
reliability of the code, readability of the documents
![Page 16: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/16.jpg)
Problems with Measurement
Costs incurred The result of any business activity
should be related to the business needs (ie reduce cost). No point measuring for the sake of measuring.
Disruptions caused Developing and using the right method
(consistency in results & repeatability).
![Page 17: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/17.jpg)
Basics
We need to assert whether we are measuring to predict or to assess.All metrics must be validated
Next question:-When do we measure what?
![Page 18: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/18.jpg)
Requirement Definition
Structural Design
Detailed Design Coding Unit Testing
Integration testing
Document metrics
Size of product
System structure
Defect detection and removal effort
Effectiveness of defect correction process
Requirements faults and changes
Structural design faults and changes
Detailed design faults and changes
Changes in number of modules
Configuration management system
Test planning
Test effectiveness
Effectiveness of test planningLife-cycle stages and metricsManns T & Coleman M (1996)
![Page 19: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/19.jpg)
Software Metrics Considerations
Cost Oriented labour cost (normal time) £/hour labour cost (overtime) £/hour
Size Oriented lines of source code 000s project elapsed time months number of project staff people project effort person hours system documentation pages
Function Oriented number of tasks number task complexity scale number of system interfaces number
Productivity Oriented output LOC/person-month documentation produce pages/person-month
![Page 20: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/20.jpg)
Function Point Analysis
algorithmic modelling techniqueproposed by Albrechtproductivity measurement based on function pointsmethod based on functionality rather than sizeuseful analogies: building a house: each type of house has a configuration of windows, rooms, fittings ... etc. all with a norm for construction upon which the surveyor bases his cost estimates. FPA uses the software design as a blueprint and the design components as its configuration
standard costs in Accounting: everything in a business has a cost
![Page 21: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/21.jpg)
Function Point Analysis
Function points are computed by counting the software characteristics:
number of user inputs number of user outputs number of user inquiries number of files number of external interfaces
a complexity value is associated with each of the countsa Function Point Count is computed using the formula:
FP = count-total x [0.65 + 0.01 x SUM(Fii)]
once calculated function points are used in a way analogous to LOC
![Page 22: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/22.jpg)
GQM
1. Set goals specific to needs in terms of purpose, perspective and environment.
2. Refine the goals into quantifiable questions that are tractable.
3. Deduce the metrics and data to be collected (and the means for collecting them) to answer the questions
![Page 23: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/23.jpg)
![Page 24: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/24.jpg)
GQM phases
Interpretation
Data Collection
Definition
Measurement resultsAnswers to Questions
Evaluation of goal attainment
Collected Data
Goals, questions, metrics, hypotheses
PlanningProject selected
Project plan
![Page 25: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/25.jpg)
Example
Goal – Reduce helpdesk maximum turnaround time for basic queries below 48hrs.Questions: What proportion of queries take longer than
48hrs? What types of queries take longer than 48
hrs? What throughput of queries per person leads
to unacceptable turnaround time?
![Page 26: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/26.jpg)
Example metrics
Metrics Average turnaround time for the last year. Number of queries dealt with. Number of queries taking > 48 hrs. Correlation of staff on duty with turnaround
time. Correlation of query type with turnaround
time. Etc.
![Page 27: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/27.jpg)
What can goals relate to?
Products which are output from processes Artefacts, deliverables or documents which
arise from the processes
Processes Are any software related activities which take
place over time
Resources which are inputs to processes E.g. person-hours, processing time, office
space.
![Page 28: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/28.jpg)
Checklist for using GQM
Define human resources are in placeAll necessary resources are availableGoals are identifiedProject team support the goalsProject plans are availableCommunication strategy is in place
Metrics are defined and are consistent with goals and questionsGQM/Measurement/ Analysis plan are available and agreedManagement are fully involvedTrial measurement period is held
![Page 29: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/29.jpg)
Making the Decision to use metrics
Triggers: General quality concerns Concerns about customer complaints Customers’ requirements for increased information Knowledge of competitors activities in the metrics area Process improvement programmes Cost Cutting Management pressure for information A consultancy recommendation New awareness of possibility of measurement, perhaps
as a result of someone attending a conference Forrester Research Inc. (Barnett et al., 2005)
![Page 30: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/30.jpg)
Developing a Measurement Culture
Plan your measurement activities Before you start the work
Fear of data being used negatively Confidentiality of the data
Educate the team Provide appropriate training, explain
why
Share the data within the team
![Page 31: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/31.jpg)
Developing the habit
Measurements do not have to be time consuming Use commercial toolsTracking forms/Spreadsheets/ChartsStart small/explain why/share resultsUse the information extracted to improve development work
![Page 32: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/32.jpg)
Starting points (1)
Individual developers Work Effort distribution Estimation vs task durations and
efforts Code covered by unit testing No of defects found per unit testing Code and design complexity
![Page 33: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/33.jpg)
Starting points (2)
Project Teams Product size Work effort distribution Requirements stats, (how many,
implemented, verified) Estimation vs actual Defect counts found when integrating
systems Defect counts found by inspections Requirement stability Number of tasks planned and completed
![Page 34: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/34.jpg)
What to avoid
Lack of management commitmentMeasuring too much, too soonMeasuring too little, too lateMeasuring the wrong thingsImprecise metrics definitions
![Page 35: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/35.jpg)
What to avoid
Using metrics data to evaluate individualsUsing metrics to motivateCollecting data that is not usedLack of communication & trainingMisinterpreting metrics data
![Page 36: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/36.jpg)
Summary
Overview of V&V, testing.What software metrics are concerned with, and what we can measure.Approaches, especially GQM.Good and bad practice in using metrics.
![Page 37: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/37.jpg)
ReadingPressman (2000)
Chapter 4 on project and process metrics; sections on estimating & function points in chapter 5.
(Also some good material on testing.) Equivalents in other editions will do.
Good article on GQM: “Software Acquisition GOLD PRACTICETM Goal-
Question-Metric (GQM) Approach” at http://www.goldpractices.com/practices/gqm/index.php
Bennett, McRobb & Farmer (2005), Chapter 19, on implementation (testing) Or equivalent in earlier versions
![Page 38: Testing & Software Metrics CM602 Effective Systems Development.](https://reader033.fdocuments.net/reader033/viewer/2022051214/56649ede5503460f94bee7d8/html5/thumbnails/38.jpg)
References
Bennett, S., McRobb, S. & Farmer, R. (2005), Object-Oriented Analysis and Design using UML, 3rd edn. McGraw-Hill.Pressman, R.S. (2000), Software Engineering, A Practitioner’s Approach, 5th edn (European), McGraw-Hill.Barnett, L., Visitacion, M., Gilpin, M. & Symons, C. (2005), “Metrics For Application Development: Selecting A Balanced Set Is No Easy Task” available at http://www.forrester.com/Research/ [viewed 1/12/05].