MTAT.03.243 Software Engineering Managementset of activities) related to software development and/or...

MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013

MTAT.03.243

Software Engineering Management

Lecture 07:

SPI & Measurement –

Part A

Dietmar Pfahl

email: [email protected] Spring 2013


Structure of Lecture 7

• Motivation and Definitions (Measure, Measurement)

• Example Measures (Process, Product, Resource)


Project Planning and Control

All activities related to match the current course of the project with the planned course of the project


Types of Process Models

Processes

Engineering Processes Non-Engineering Processes

Product-Engineering

Processes Process-Engineering

Processes Business

Processes Social

Processes

Development

Processes

Maintenance

Processes

Project Mgmt

Processes

Quality Mgmt

Processes

Conf Mgmt

Processes Product Line

Processes

Improvement

Processes

Process Modeling

Processes

Measurement

Processes

Process Taxonomy

Software

Knowledge

Process

Models

Product

Models

Quality

Models

Life Cycle

Models

Engineering Process Models

Business Process Models

Social Process Models

Technical Process Models

Managerial Process Models

Process Engineering

Proc. Models

. . .

. . .

PROFES


SPI Planning and Control

All activities related to match the current course of the SPI project with the planned course of the SPI project

SPIP Start

SPIP Planning SPIP Control

SPIP Steering

SPIP Enactment SPIP End

SPIP = SPI Project


Measurement in PROFES


Definitions:

Measurement and Measure

Measurement:

• Measurement is the process through which values (e.g., numbers) are assigned to attributes of entities of the real world.

Measure:

• A measure is the result of the measurement process, so it is the assignment of a value to an entity with the goal of characterizing a specified attribute.

Source: Sandro Morasca, “Software Measurement”, in “Handbook of Software Engineering and Knowledge Engineering - Volume 1: Fundamentals” (refereed book), pp. 239 - 276, Knowledge Systems Institute, Skokie, IL, USA, 2001, ISBN: 981-02- 4973-X.

4 e *

3 d *

2 c *

1 b *

0 a *

A B

Scale & Unit

LOC (lines of code)

Size Measure

Entity: Program Attribute: Size


Software Measurement Challenge

• Measuring physical properties (attributes):

entity attribute unit* scale (type) value range*

Human Height cm ratio 178 (1, 300)

Human Temperature C° interval 37 (30, 45)

• Measuring non-physical properties (attributes):

entity attribute unit* scale (type) value range*

Human Intelligence/IQ index ordinal 135 [0, 200]

Program Modifiability ? ? ? ?

• Software properties are usually non-physical:

size, complexity, functionality, reliability, maturity, portability, flexibility, understandability, maintainability, correctness, testability, coupling, coherence, interoperability, …

‘unit’ and ‘range’ are sometimes used synonymously with ‘scale’


Measurement Scale Types


Measurement Scale Types – cont’d

The classification of scales has an important impact on their practical use, in particular on the statistical techniques and indices that can be used. Example: Indicator of central tendency of a distribution of values (“Location”).

Mode = most frequent value of distribution Median = the value such that not more than 50% of the values of the distribution are less than the median and not more than 50% of the values of the distribution are greater than the median


Scale types and meaningful measurement

• Scales are defined through their admissible transformations

• Scales (and their admissible transformations) help us decide

– whether a statement involving measures is meaningful

– what type of statistical analyses we can apply

• Definition of Meaningfulness:

A statement S with measurement values (i.e., measures m1,

…, mn) is meaningful iff its truth of falsity value is invariant

under admissible transformations Tr.

iff: “if and only if”

Tr(S[m1, …, mn]) is true iff S[Tr(m1), …, Tr(mn)] is true


Meaningfulness of Measurement-Based Statements

Definition:

A statement involving measures is

meaningful, if its truth value remains

unchanged under any admissible

transformation

Example:

“In Madrid, during summer, it’s on average

twice as hot as in Tartu” (measured on the

Celsius scale: e.g., 40 C vs. 20 C)

-> Meaningful?


Meaningfulness of Measurement-Based Statements

Procedure to check for meaningfulness:

1. Apply the admissible transformation to

measures in a statement S and obtain a

transformed statement S’.

2. If S’ can be shown to be equivalent to S,

then the statement S is meaningful for the

scale associated with the admissible

transformation.


Meaningfulness – Example 1

• Is statement (1) on the right meaningful, if X is measured on a ratio scale?

• Apply any admissible transformation M’=aM (a>0) for ratio scales:

• By arithmetic manipulation, (2) can always be made equivalent to (1). Therefore, the first statement is meaningful for a ratio scale.

(1)

(2)

mxx

2

21

Ratio Scale

maxaxa

2

21



• Is statement (1) on the right meaningful, if X is measured on an interval scale?

• Apply any admissible transformation M’=aM+b (a>0) for interval scales:

• By arithmetic manipulation, (2) can always be made equivalent to (1). Therefore, the first statement is meaningful for an interval scale.

(1)

(2)

mxx

2

21

Interval Scale

bmabxabxa

2

21



• Is statement (1) on the right meaningful, if X is measured on an ordinal scale?

• Apply an admissible transformation for ordinal scales, e.g., x’=x3:

• For any pair of measurements x1 and x2, there exists always one admissible transformation such that statement (2) is false when (1) is true. Therefore, statement (1) is not meaningful for an ordinal scale.

(1)

(2)

mxx

2

21

Ordinal Scale

3

213

3

2

3

1

22

xxm

xx


Meaningfulness – Geometric Mean

• The geometric mean of a data set [a1,

a2, ..., an] is given by

• On which scale type is the geometric

mean meaningful?

Scale Type ?


Structure of Lecture 7

• Motivation and Definitions (Measure, Measurement)

• Example Measures (Process, Product, Resource)


Measurable entities in a SW Process Model

• An entity can represent any of the

following:

–Process/Activity: any activity (or

set of activities) related to software

development and/or maintenance

(e.g., requirements analysis,

design, testing) – these can be

defined at different levels of

granularity

–Product/Artifact: any artifact

produced or changed during

software development and/or

maintenance (e.g., source code,

software design documents)

–Resources: people, time, money,

hardware or software needed to

perform the processes


Examples of Software Product Attributes

• Size

–Length, Complexity,

Functionality

• Modularity

• Cohesion

• Coupling

• Quality

• Value (Price)

• ...

• Quality (-> ISO 9126)

–Functionality

–Reliability

–Usability

–Efficiency

–Maintainability

–Portability


Product Measure – Ex. 1: ’Code Size’

• Entity

• Attribute

• Unit

• Scale Type

• Range

• Who collects/reports the data?

• When (how often) is the data collected?

• How is the data collected?

• Who is responsible for data validity?

• Code module

• Size (or better: Length)

• Netto Lines of Code (NLOC)

• Ratio

• (0, ∞)

• Developer

• Once, at end of week

• Using tool ’CoMeas’

• Project Manager


Product Measure – Ex. 2: ’Code Quality 1’

• Entity

• Attribute

• Unit

• Scale Type

• Range





• Code module (class file)

• Quality (or better: Correctness)

• Defects (Def)

• Ratio

• [0, ∞)

• Developer

• Continuously during unit testing

• Using defect reporting tool ’TRep’

• Project Manager


Product Measure – Ex. 3: ’Code Quality 2’

• Entity

• Attribute

• Unit

• Scale Type

• Range





• Code module (file)

• Quality (or better: Defect Density)

• Def / NLOC

• Ratio

• [0, ∞)

• Developer

• Continuously during unit testing

• Using tools ’TRep’ and ’CoMeas’

• Project Manager


Direct vs. Indirect (Derived) Measures

• Direct measure: a measure that directly characterizes an empirical

property and does not require the prior measurement of some other

property

• Indirect measure: uses one or more (direct or indirect) measures of one

or more attributes in order to measure, indirectly, another supposedly

related attribute.

– Requires first the measurement of two or more attributes,

– then it combines them using a mathematical model.

speed = distance / time [km/h] accuracy = ( |actual – estimate| / estimate ) * 100% [Percentage]

Is ’estimate’ a measure?


Types and Uses of Measures

• Types of Measures

– Direct vs. Indirect

– Subjective vs.

Objective

– Qualitative vs.

Quantitative

• Uses of Measures

– Assessment vs.

Prediction

NB: Measurement for

prediction requires a

prediction model


Indirect Measures

• Examples:

– Defect Density (DD)

– Reliability (Rel)

– Productivity (Prod)

• Scale type of an indirect measure M will generally be the

weakest of the scale types of the direct measures M1, …,

Mn


Indirect Measures

• Examples:

– DD = Quality 1 / Size [Unit: #Def/NLOC]

– Reliability = Quality 1 / Time [#Def/hour]

– Productivity 1 = Size / Time [NLOC/hour]

– Productivity 2 = Size / Effort [NLOC/person-hour]

– ...


Examples of Software Process and Resource

Attributes

• Process Efficiency:

How fast, how much effort, how much quantity/quality per time or

effort unit?

• Process Effectiveness:

Do we get the quantity/quality we want?

• Process Maturity:

CMMI level

• People/Organisation-related:

Skills, knowledge, learning, motivation

• Method/Technique/Tool-related:

Effectiveness, Efficiency, Learnability, Cost


Process Measure – Ex. 1: ’Acceptance Test Time’

• Entity

• Attribute

• Unit

• Scale Type

• Range





• Acceptance Test

• Time (or ’Duration’)

• Calendar Day

• Interval

• [0, ∞)

• Customer XYZ

• At end of every test day

• Using reporting template ’RT’

• Product Owner


Process Measure – Ex. 2: ’Coding Effort’

• Entity

• Attribute

• Unit

• Scale Type

• Range





• Coding

• Effort

• Person-hour

• Ratio

• [0, ∞)

• Developer

• At end of every work day

• Using reporting template ’RE’

• Project Manager


Effort vs. Time Trade-Off

Person Effort = 4 person-days (pd)

1 1 1 1

Person Effort = 4 pd

1 1 1 1


1

1 1

1

Day

Day

Day

What does it mean when I

say:

• ”This task takes 4 days”

• ”This task needs 4 person-

days”


Effort vs. Time Trade-Off

Person Effort = 4 person-days (pd)

1 1 1 1


1 1 1 1

?


1

1 1

1

?

COCOMO

Source: B. Boehm: Software Engineering Economics, Prentice-Hall, 1981.

Day

Day

Day ?

COCOMO Intermediate model: Effort Adjustment Factor -> Schedule (SCED)

b d

devE aS EAF T cE


Agile Measurement:

Sprint Burndown Chart – Example

Sprint

Backlog

(Task List)


Agile Measurement: Velocity (1)



Solid agile teams have consistent velocity (+/- 20%) Fluctuations? -> Look to stabilize team / environment Velocity trending up/down? -> Look at technical debt handling (rework) and team processes ...



Pitfalls to avoid

• Velocity measures team history of delivered software (in terms

of story points or similar units)

Remarks

• Do not use not-done work in velocity calculations

• Velocity between teams is not comparative or additive

• Velocity does not show relative performance or productivity of

teams


Resource Measure – Ex. 1: ’Programming

Skill’

• Entity

• Attribute

• Unit

• Scale Type

• Range





• Team Member

• Programming Skill

• Programming Test Score (PTS)

• Ordinal

• [0, 1, 2, 3, 4, 5] – NB: Each

number needs explanation!

• Skill Test Agency

• Whenever a test is conducted

• Programming Test

• Certification Agency


Resource Measure – Ex. 2: ’Personality’

• Entity

• Attribute

• Unit

• Scale Type

• Range





• Team Member

• Personality

• Myer Briggs Type Indicator (MBTI)

• Nominal

• Set of 16 Types: ISTJ, ISFJ, INFJ,

..., ENTJ

• Test Agency

• Whenever a test is conducted

• MBTI Instrument

• Certification Agency


Next Lecture

• Topic: SPI & Measurement – Part B

• For you to do:

– Have a look at the PROFES Quick Reference and

Manual -> What does it say about measurement?

– Have a look at some of the industry case examples in

the book “The Goal/Question/Metric Method: a practical

guide for quality improvement of software development”

(it’s in the reading materials)

MTAT.03.243 Software Engineering Managementset of activities) related to software development and/or...

Documents

Transcript of MTAT.03.243 Software Engineering Managementset of activities) related to software development and/or...