Software Project Management Matrics

Software Project Management Matrics

Complied by

Heng Sovannarith

[email protected]

Introduction

• Hardware is declining while software is increasing.

• Software Crisis:

– Schedule and cost estimates -> Inaccurate

– Software poor quality

– A productivity rate that is increasing more slowly than the demand for software.

Why we need Software Metrics

• The software crisis must be addressed and, to extent possible, resolved.

• Accurate schedule and cost estimates, better quality products, and higher productivity.

• Use software matrices can help better management result, both short time and long term,

Software Metrics

• The terms measure, measurement, and metrics are often used interchangeable.

• We CANNOT Control what we CANNOT Measure -- Tom DeMarco,

American Software Engineer.

Software Metrics (cont.)

• A software metric is a technique or method that applies software measurements to a class of software engineering objects to achieve a predefined goal

• The goal is obtaining objective, reproducible and quantifiable measurements, which may have numerous valuable applications in – schedule and budget planning, – cost estimation, – quality assurance testing, – software debugging, – software performance optimization, – and optimal personnel task assignments.

Software Metrics (cont.)

• More than 300 metrics have been defined.

• How can you judge the two images? What will you to judge them?

Types of Metrics ( Types of Measured data)

• Nominal - Categories -- Database programming,

operating system...

• Ordinal - Ranking -- programmer experience level maybe measured as low, medium, or high.

• Interval - Differences. For example... complexity is 4 unit more complex than a program with complexity f 2.

• Ratio scale - example. a program which have 2000 line of code is twice as large as a program of 1,000 line...

Characteristics of a Metric

• Object of measurement

– Products, processes and projects

• Purpose of measurement (Measurable

– Characterization, assessment, evaluation, prediction

• Source of measurement

• Measured property

• Context of measurement

Who benefit from Measurement?

• Managers – What does each process cost? – How productivity is the staff? – How good is the code being developed? – Will the user be satisfied with the product? – How can we improve?

• Engineers – Are the requirement testable? – Have we found all the failures? – Have we met our product or process goals? – What can we predict about our software product in the

future?

Software Metrics Categories

• Product metrics – Used to measure aspects of artifacts delivered to

the customer

– Information flow metrics, function points, complexity

• Process metrics – Used to measure aspects of the development and

maintenance process

– example: time to correct defect, number of components changed per correction

Software Metrics Categories (cont.)

• Design metrics :

– Design metrics enable you to decide how much you have deviated from the requirements of the project.

• Maintenance metrics

– These measure the progress of a maintenance project.

Product Metrics

• Analysis done using the product metrics can provide you an insight into the quality of work done by the development teams.

• Product metrics trace and measure the quality of a deliverable in different phases.

• A set of metrics measures the quality and timeliness of the deliverable.

Product Metrics (cont.)

• To measure the metric of source code, you can use the format displayed


• Effectiveness of a deliverable is measured in terms of the quantity of the deliverable completed and the number of quality goals achieved.

• You can use the following formula to calculate effectiveness of a deliverable:


• For example, 80 percent of source code is completed and 90 percent of the quality goals are achieved in developing that source code.

• Effectiveness = 80 * 90/100

Effectiveness = 72

Design Metric

• Complexity is a characteristic measured during this phase.

• Complexity can be measured for the structure, data components, and the interface design of a software program.

• Complexity can affect the size, testability, and effort spent on developing and testing the modules of a project.

Design Metric (cont.)

• The measure of the complexity of a software design is called a design metric.

• Design metrics are seldom used in organizations because organizations find it time-consuming to analyze their results.

• Moreover, design metrics are difficult to implement.

Design Metric (cont.)

• There can be many varieties of design metric.

• One variety is architecture design metrics – It address the three types of complexities of a

software program. • Structural complexity

– Eg. a module invoked by a module parent -> increase the complexity

• Data complexity – Input and output variable

• System complexity – Combine structural complexity and data complexity

Project Metrics

• Project metrics are specific to the actual execution of a project.

• They enable you to execute a project with a systematic and technical approach.

• Project metrics can help you avoid project delays.

• You can also use them to avoid and mitigate project risks.

Project Metrics (cont.)

• Project metrics are also used to assess the quality of a finished product on a periodic basis.

• Examples of project metrics are the metrics that measure the size, complexity, or the time and effort spent on a project.

Project Metrics (cont.)

• Six important project metrics are commonly used in an organization. – Effort – the amount of effort required to complete a project – Productivity – Measure human effort according to the number of

hours spend on work. – Cost -- measures the planned versus actual expense incurred on a

project – Size (important) -- calculated for an entire project and not for any

particular phase of a project. – Defects –

• Defects are errors that occur in a work product • The number of defects defines the quality of a project.

– Testing • measure the number of test cases required to test software. • Test case is a specification that needs to be executed to test a particular

module in a software program.

Effort Metrics

• The effort metric enable you to determine the amount of effort to complete a project.

• It is calculated in person-month (man-months).

• A person-month is a mount of effort required to complete work in a month.

• Person-month is a measure of work effort. Other similar measures are person-day and person-year.

Effort Metrics

Metrics Analysis Design Coding Testing Total

Effort Planned effort (person-months)

5 12 25 10 52

Planned effort (person-months)

8 14 35 8 65

Planned effort (person-months)

60 16.5 40 -20 25

Effort Metrics (cont.)

• If a project will take 2 months to finish with 3 people working full time on it, the project requires 2*3 = 6 person-month effort.

• If an employee worked 20 days on a project, his

contribution to the project is 1*20/30 = 0.6666 person-month. (Note that month is considered 30 days in such calculations.) Effort percentage question isn't clear but if it means the percentage effort put in by an employee towards a certain project, just calculate his person-months and divide that by total person-months for that project.

Productivity Metric

• What is Productivity? – Amount (or value) of output per unit of input (e.g., cars per

work hours)

• Why do we care? – To make the right resource allocations & have the right policies

• Obviously measuring personal productivity is important, and the reasons why are obvious.

• If you do not know how your staff is doing, then how can you truly know the inner workings of your own company?

• Staff discontent can put your company in serious jeopardy, while on the other hand, high staff productivity can be your best company asset.

Productivity Metric (cont.)

• The definition of productivity is the output-input ratio within a time period with due consideration for quality. – Productivity = outputs/inputs (within a time period,

quality considered)

• The formula indicates that productivity can be improved by (1) by increasing outputs with the same inputs, (2) by decreasing inputs but maintaining the same outputs, or (3) by increasing outputs and decreasing inputs change the ratio favorably. – Software Productivity = Function Points / Inputs

Productivity Metric

• The effort in performing an FP (Function Point) of work in an hour is called productivity.

• To calculate productivity, you first determine the total amount of FP for the SW project.

• Measurements that help determine how to make more products with fewer resources, such as producing the most parts with the fewest numbers of employees.

Productivity Metric

• Function Points measure software size by quantifying the functionality provided to the user based solely on logical design and functional specifications

• Has gained acceptance in terms of productivity (for example FP/year) and quality (defects/FP)


Measurement parameter Weighting factor

Simple Average Complex

Number of user inputs 3 4 6

Number of user outputs 4 5 7

Number of user inquiries 3 4 6

Number of files 7 10 15

Number of external interfaces 5 7 10


Metric Analysis Design Coding Testing Total

Productivity in hours/FP

Planned 1.2 1.6 2.8 2.4 8.0

Actual 1.0 1.4 3.0 2.5 7.9

Difference (+/-) 0.2 0.2 -0.2 -0.1 0.1

• To calculate accurate productivity for a particular phase, you need type of data: • Actual effort expressed in person hours. • Actual size of the project in FP

Cost metrics

• A cost metric measure the planned vs actual expense incurred on a project.

• An important component of the cost metric is the cost of resources (human resource and material resources)

Cost Metrics (cont.)

Metrics Analysis Design Coding Testing Total

Cost in USD

Resources Planned 40,000 100,000 200,000 100,000 440,000

Actual 55,000 130,000 300,000 250,000 735,000

Communication

Planned 10,000 7,000 1,000 1,000 19,000

Actual 20,000 4,500 700 500 25,700

Total Actual 75,000 134,500 300,700 250,500 760,700

Total deviation in percentage (actual planned/planned * 100)

50 25 50 148 66

Size Metrics

• Size metric is a calculated for an entire project and not for any particular phase of a project.

• The size of a project may vary with respect to the changes required by the customer.

• The size metric directly affects effort, testing, and productivity metrics of a project.

• There are two main components of the size metric: – Change in the required effort due to change in project

size – Percentage growth in project size

Size Metrics (cont.)

• Line of Code: It is one of the earliest and simpler metrics for calculating the size of computer program. It is generally used in calculating and comparing the productivity of programmers.

• Function Count: – The size of a large software product can be estimated in

better way through a larger unit called module. A module can be defined as segment of code which may be compiled independently.

– For example, let a software product require n modules. It is generally agreed that the size of module should be about 50-60 line of code. Therefore size estimate of this Software product is about n x 60 line of code.

Size Metric (cont.)

• Token Count: – Operands and operator of any computer language is

called tokens. Any symbol or keyword in a program that specifies an algorithmic action is considered an operator,

– while a symbol used to represent data is considered an operand.

– In terms of the total tokens used, the size of the program can be expressed as N=N1+N2

Here, • N1: count of total occurrence of operators • N2: count of total occurrence of operands

Size Metrics (cont.)

Metric Total

Size in FP

Size Planned 100

Size Actual 120

Change in size +20

Growth of size in % 20

Defects Metric

• The number of defects defines the quality of a project.

• If the number of defects is large, the quality of the product is inferior.

• Defects can occur during any phase of the SDLC.

• Defects not defected in a particular phase continue to be present during the subsequent phases in the SDLC.

Defects Metric (cont.)

• Very useful as measure of organizational capability to produce defect-free outputs – Can be compared with other organizations in the same domain

• Outlier information useful to spot problem projects and problem components

• Can be used in-process, if comparison is with defect densities of other projects in same phase – If much lower, may indicate defects not being found

– If much higher, may indicate poor quality of work

– (In other words, need to go behind the numbers to find out what is really happening. Metrics can only provide triggers)


• Defect evaluation is driven by the need for Defect prevention. Understanding defects and metrics surrounding them will help eliminate defect injection and rework.

• Most metrics won’t mean anything at first until we can establish a baseline to compare to.

• This is digging deeper into the data available and the resources involved finding out why a defect has entered the system. – Examples could be Developer Error, Missing Tests/Requirements,

Unclear Requirements, or Environmental (Client/Server) or any of the reasons. It’s important to identify it correctly to be able to look for patterns or repeats of specific reason types.


• This will help to identify problem areas and propose solutions to resolve.


• Errors – Defects in the human thought process made while trying

to understand given information, to solve problems, or to use methods and tools

• Faults – Concrete manifestations of errors within the software – One error may cause several faults – Various errors may cause identical faults

• Failures – Departures of the operational software system behavior

from user expected requirements – A particular failure may be caused by several faults


• Severe defects critically affect the functionality of a product.

• Major defects logically affect the functionality of a product.

• A minor defect is a slight defect that may act as an irritant for users but does not disturb the functionality of the application.


• Defect density – Number of defects per size of the software project – Size of project measured in LOC or Function Points – Useful metric in determining if a software release

is ready for delivery to the customer

– Defect density in released code (“defect density at release”) is a good measure of organizational capability

– Defects found after release / size of released software

– Can only compare across similar projects


• Pair Programming – Two software developers work together at one

computer

– One developer is coding, the other developer is reviewing

– Potential to find defects earlier because the code is under constant review

– On average, PP leaves 15% fewer defects than conventional methods

Defect Metrics

Metric Analysis Design Coding Testing Total

Defects per KLOC

Severe Planned (less than equal to) 0 0 0 0 0

Actual 5 3 15 8 31

Major Planned (less than equal to) 5 2 5 10 22

Actual 3 6 12 18 39

Minor Planned (less than equal to) 20 15 15 20 70

Actual 12 10 12 25 59

Defect Metrics (cont.)

• Prevention Methods – Introduced by Michael Fagan in the 1970s at IBM

– Involves reviewing a software work product by a group of trained individuals

– Reviewers review the software code changes and relay comments back to the author for clarification or further work.

– Initial costs are in training individuals on the process.

– Software developers are reviewing instead of coding.


• Figure is a cost analysis of defect prevention • Clearly, defects found earlier saves money • Team can be set up dedicated to the defect

prevention activities


• Personal Work Experience – Software development experience of 2 years – Software test experience of 3 years – Use peer reviews to minimize software defects

• Run unit tests and integration tests with test team witness

– Been a part of 2 major software releases and several patch releases • Tough to anticipate all behaviors of system

Testing Metrics (cont.)

• Test metrics provides the visibility into the readiness of the product, and gives clear measurement of the quality and completeness of the product.

• Testing metric is used to track actual testing progress against plan and therefore to be able to be proactive upon early indications that testing activity is falling behind.

• Indicators that show the quality of the software under test and the effectiveness of the testing activities.


• Test metrics can give us the information about the software quality, and help the PM to make decision.

• Metrics are the numerical data which will help us to measure the test effectiveness.


• Test Metrics is a process of measuring the Software Testing Process with respect to the Actual Test Plan in the organizations. Based on the feedback, the organizations will improve the testing process that they are following.

• Test metrics are essentially data which provide you insight into the status of either – 1) your testing itself or

– 2) the product under test.


• The first is a metric which provides such insight as how far through your testing are you (how many test cases exist, how many have you executed, how many remain, what's your execution rate, what is your projected end date).

• The second is a metric providing insight into the state of the product under test. – How many open bugs are there? – How many resolved bugs? – How many bugs are being opened per day? Per test pass? Per

test case? – Is the rate of bugs being resolved higher than the rate of bugs

being opened? – How many bugs are 'stale' (older than a certain number of

days?), etc.


• Test Metrics is measuring the Testing Activities. • Test Metrics are varying from company. Every

company following there own metrics. • In some companies, they following metrics are:

– Number of Test Cases Executed – Number of Test Cases Passed – Number of Test Cases Failed – Total Number of Defects Raised. – Number of Defects Raised in SIT (System Integrated

Test) – Number of Defects Raised in UAT (User Accept Test)


• The testing metric is also used to measure the number of test case required to test SW.

• Test case is a specification that needs to be executed to test a particular module in a SW program.

• There are a separate test cases for integration testing and unit testing: – Integration testing refers to the overall black box testing of

the entire SW project. – In contrast, unit testing refers to testing individual modules

of a SW project..

• Every testing project should be measured for its schedule and the quality requirement for its release.

Why do we need the test metric?


Metrics Coding

Testing Test cases per FP

Planned 5

Actual 4

Testing Metrics

• Test case Defect Density The number of errors found in test cases v/s test cases developed and

executed – ( Defective Test cases / Total Test cases ) * 100

Example : Total no of test cases developed is 1360, total test cases executed

is 1280, total no of test cases passed is 1065, total no of test scripts failed is 215

So Test case Defect Density is : 215 X 100 ------------------------------- = 16.8 % 1280 The 16.8 % value can also be called as Test Case Efficiency % which

depends upon the total number of Test cases which found defects

Process Metrics

• Process metrics are collected across all projects and over long periods of time

• They are used for making strategic decisions • The intent is to provide a set of process indicators

that lead to long-term software process improvement

• The only way to know how/where to improve any process is to – Measure specific attributes of the process – Develop a set of meaningful metrics based on these

attributes – Use the metrics to provide indicators that will lead to a

strategy for improvement

Process Metrics

• We measure the effectiveness of a process by deriving a set of metrics based on outcomes of the process such as – Errors uncovered before release of the software – Defects delivered to and reported by the end

users – Work products delivered – Human effort expended – Calendar time expended – Conformance to the schedule – Time and effort to complete each generic activity

Maintenance Metrics

• Software Maintenance is the general process of changing a system after it has been delivered.

• The changes made to the software may be

– simple changes to correct the code errors, more extensive changes to correct design errors, or significant enhancements to correct specification errors or accommodate new requirement.

A Typical Maintenance Flow

Proposed M. R.’s

Approved M. R.’s

Modified source & documentation

Current source & documentation

Change control board Maintenance engineer

Written MR’s

Customer

Help desk

nominal path

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Types of Maintenance • Corrective

– defect identification and removal

• Adaptive

– changes resulting from operating system, hardware or DBMS changes

• Perfective

– changes resulting from user requests

• Preventative

– changes made to the software to make it more maintainable

Fixing

Enhancing

Maintenance Metrics (cont.)

• When development of a software product is complete and it is released to the market, it enters the maintenance phase of its life cycle.

• During this phase the defect arrivals by time interval and customer problem calls (which may or may not be defects) by time interval are the de facto metrics.

• However, the number of defect or problem arrivals is largely determined by the development process before the maintenance phase.

• Not much can be done to alter the quality of the product during this phase.

Maintenance Metrics (cont.)

• What can be done during the maintenance phase is to fix the defects as soon as possible and with excellent fix quality.

• Such actions, although still not able to improve the defect rate of the product, can improve customer satisfaction to a large extent.

• The following metrics are therefore very important: – Fix backlog and backlog management index

– Fix response time and fix responsiveness

– Percent delinquent fixes

– Fix quality

Fix Backlog and Backlog Management Index

• It is related to both the rate of defect arrivals and the rate at which fixes for reported problems become available.

• It is a simple count of reported problems that remain at the end of each month or each week.

• Using it in the format of a trend chart, this metric can provide meaningful information for managing the maintenance process.

Fix Backlog and Backlog Management Index (cont.)

• Another metric to manage the backlog of open, unresolved, problems is the backlog management index (BMI).

• As a ratio of number of closed, or solved, problems to number of problem arrivals during the month, if BMI is larger than 100, it means the backlog is reduced.

• If BMI is less than 100, then the backlog increased. • More investigation and analysis should be triggered when

the value of BMI exceeds the control limits. Of course, the goal is always to strive for a BMI larger than 100.

Fix response time and fix responsiveness

• Most organisations have established guidelines on the time limit within the fixes should be available for the reported defects

• Severe problems are usually being as soon as possible, less severe has more relaxed time

• The fix response time metric is usually calculated for all problems as follows:

Mean time of all problems from open to closed

Fix response time and fix responsiveness (cont.)

• Sometimes there are less severe problems that customers just ignore, so the problem remains open for a long time

->distortion of mean time

->in these cases, medians should be used instead of mean time values

• Idea: short time in fix process leads to customer satisfaction and determines how good the process is in this field

Percent Delinquent Fixes

• The mean (or median) response time metric is a central tendency measure. A more sensitive metric is the percentage of delinquent fixes.

• For each fix, if the turnaround time greatly exceeds the required response time, then it is classified as delinquent:

Fix Quality

• Fix quality or the number of defective fixes is another important quality metric for the maintenance phase.

• From the customer's perspective, it is bad enough to encounter functional defects when running a business on the software.

• It is even worse if the fixes turn out to be defective.

• A fix is defective if it did not fix the reported problem, or if it fixed the original problem but injected a new defect.

Fix Quality (cont.)

• The metric of percent defective fixes is simply the percentage of all fixes in a time interval (e.g., 1 month) that are defective.

• This metric, therefore, should be a straight count of the number of defective fixes.

• The quality goal for the maintenance process, of course, is zero defective fixes without delinquency.

Steps To Create Metrics

• Define the goal of a metric

• Identify the requirement of a metric

• (Human, Data Collection Technique, Methodology)

• Determine the baseline figure of the metric

• Review the metric created

References

• Pok, Leakmony. Software Engineering, Phnom Penh, RUPP, 2012.

• Kan, Stephen H., Metrics and Models in Software Quality Engineering (2nd Edition), Boston: Addison-Wesley, 2003.

• Sommerville, I. Software Engineering (9th edition), Pearson, 2010,

• Other resources…

Software Project Management Matrics

Documents

Transcript of Software Project Management Matrics