04/19/23 1

The Importance of the Requirements Phase

Shuly Cooper

04/19/23 2

• Product Quality – Definition• The issues and their root cause• Cost of Quality (or lack of it)• A Tool –

– Quality Function Deployment (QFD) • Briefly –

– More quality tools ► Inspections

► Iterative/incremental dev. (small deliverables) – The deliverables of the Requirements Phase

To Be Discussed

04/19/23 304/19/23 3

19451980s

“Higher Accuracy & Reliability”

The Era of Statistical Process Control (SPC)

Dr. W. Edward Deming

Dr. Peter F. Drucker

Product Quality - Definition

Act Plan

Check Do

The Deming Cycle:

04/19/23 4

Studies by the Standish Group

• 2001 data• 44% of software projects finish on time• 56% of projects completed at –

– 222% of the duration originally planned– 189% of the original budgeted cost– 70% of projects reduced their planned scope

significantly– 30% were cancelled before completion

• Ref: http://www.pqa.net/ProdServices/ccpm/W05002001.html

The Challenges (1 of 2)

04/19/23 5

Capers Jones, Software Productivity Research LLC

• 1995 - 2004 data

• An analysis of ~250 large software projects • Size > 10,000 function points (~ 1,250,000 statements in

C)• Success vs. failure were analyzed at opposite ends of the

spectrum– ~ 25 (10%) projects were successful– ~ 50 (20%) projects had delays or overruns below 35% – ~ 175 (70%) projects experienced major delays, budget

overruns, reduced scope, or were terminated

The Challenges (2 of 2)

04/19/23 6

• Poor requirements 27%• Unanticipated technical difficulties 25%• Training 20%• Poor Project Management 17%• Lack of management support 6%• Other 5%

Source: John Carter - Product Development Consulting, Inc - Spin 1997 -

Root Cause for Development Schedule Slip

04/19/23 7

Engineers’ Awareness at the Start of a Project

• Priority Decision Criteria 20% 36%• Product Positioning 22% 41%• Strategic Alignment 18% 41 %• Organization Support 20% 43%• Risk Assessment 26% 44%• Competitive Analysis 21% 47%• Understanding User Needs 21% 48%• Regulation Compliance 60% 73%

Others Best

Source: John Carter - Product Development Consulting, Inc Berkeley Software Forum - 1996

The Current State of the Requirements Phase

04/19/23 8

Capers Jones

• The two most common causes of software project failures are :– Risk analysis– Vague a/o changing customer’s requirements

• Ref: http://www.isixsigma.com/library/content/c010618b.asp

04/19/23 10

The Cost of Fixes

Resources’ bleeding

04/19/23 11

Cost of QualityIBM -- late 70’s

100K

10K

1K

100

10

0

Log (Cost to Find&Fixper Defect) ($)

Reqs. Design Coding Sys.Test Release

Phase

04/19/23 12

Cost to Fix a DefectSemantech -- 90’s

Requirements $195

Design $489

Coding $997

System Testing $7,136

Deployment Much more

SEMATECH Technology Transfer #92111389A-TRG, March 5, 1993

Source: Chuck Hoffman - Virtual Consulting Inc. - Spin 2001

04/19/23 1304/19/23 13

Log [Cost of NOT Finding & Fixing a req. defect($/Defect)

Cost of Quality

100K

10K

1K

100

10

0

Log [Cost to Find&Fixper Defect] ($)

Reqs. Design Code Sys.Test Release

Phase Found

=> The most expensive faults are seeded during the Requirements Phase.

When faults migrate from phase to phase, they multiply 10-12 times

A Quality Gates

IBM

04/19/23 15

Defects Classified by Time of Introduction (1970s)

Jones* Thayer** Boehm*** Bell Labs

Requirements 10% 8-10% 15%

Functional Design 15% 55% 15-20%

Logical Design 20% 25-35%

Coding 30% 35% 25% 35%

Documentation, etc. 35% 17-20%

* “Measuring Programming Quality and Productivity,” Jones, IBM Systems Journal, 1978

** Software Reliability: A Study of Large Project Reality, Thayer, Lipow & Nelson, North-Holland, 1978

*** “Developing Small Scale Application Software Projects: Some Experimental Results,” Boehm, Proceedings, IFIP 8th World Computer Congress,1980

Source: Chuck Hoffman - Spin 2001

45%45%

04/19/23 16

Hewlett Packard’s Experience (1990s)

28%

30%

7%

35%

Specification

DesignCoding

Other

Source: “Implementing and Sustaining a Software Inspection Program in an R&D Environment,” MacLeod, Hewlett-Packard Journal, June 1993

Source: Chuck Hoffman - Spin 2001

04/19/23 17

Distribution of Effort to Fix Defects

Code7% Other

10%Design

27%

Requirements56%

Code1%

Other4%

Design13%Requirements

82%

Why Manage Requirements (2000)

Distribution of Defects

Source: Larry Boldt - Technology Builders, Inc. - Spin 2001

04/19/23 18

Log Cost to Find & Fixper Defect ($)

Find & Fix Defects as Early as Possible

100K

10K

1K

100

10

0Req. Design Code Sys. Test Release

Phase

10%

20%

30%

40%

50%

% of DefectsIntroduced

Quality Gates

O

O

O

O

04/19/23 20

A New Definition of Quality

04/19/23 21

1980

Exceeding Customer expectations

Prof. Yoji Akao -- U. of Tokyo

Deals with the total customer experience

Part of “Hoshin Kanri” (Japan)

Ref – http://www.qfdi.org/

Product Quality – Definition

04/19/23 2204/19/23 22

QFD Maps “What” to “How”

WHAT

HOWMRKT.

DEV.

HMLH

M

L

1234

5

6

A B C D E

H

H

H

HM

M

M

M

L

L

L

L

H

H

TOTAL

COMPETITOR

TOTAL

IMPORTANCE F

L=1M=3H=5

NEEDS

FEATURES

04/19/23 2304/19/23 23

QFD for this Lecture

MRKT.

DEV.

HHMM

M

MH

HH

H

HL

L

M

H

L

TOTAL

COMPETITOR

TOTAL

IMPORTANCE

Clear/EasyCorrect (no errors)

Promote QA/Req.

Low cost

Long shelf life

Research

Provide D

ata

Colors

Sections

Font size

Exam

ples

Fun/Aesthetic

H H

L=1, “Bells & Whistles”M=3H=7, “A Show Stopper”

91

39

110 66 66 42 39 30 353

M

M

M M

L L

91108 0

24M

HOW

WHAT

04/19/23 24

A Sample of Customers’ Needs (QFD – first column) (1 of 2)

• Feature set/Capabilities• Reliability (MTBF)• Safety/Security• Recoverability (zero loss of data)• Scalability (different platforms, apps., tunable,

modular)

04/19/23 25

A Sample of Customers’ Needs (QFD – first column) (2 of 2)

• Performance (higher speed)• Usability (easy to set-up, use, maintain)• Friendliness (easy to learn)• Supportability (service, 24x365)• Release date/Availability • Aesthetics

04/19/23 26

QFD – Benefits – (1 of 2)

A TEAM communication tool

• A “semi-mathematical” tool• Drives consensus across organizational

boundaries• Prevents “politics”, “lobbing” and “power struggle”• Stabilizes requirements

04/19/23 27

QFD – Benefits – (2 of 2)

• Helps organizations to focus on customers’

satisfaction Maps customers’ needs into high level features

• Cost saving –“Gets it right the first time” • Reduces risk Identifies competitive features

and strategies

04/19/23 28

QFD – Cost

• Planning• Depends on project size, 4-10 marketing and

development engineers (all groups involved)• About 1 hr/10X10 matrix• Analyzing, storing, and reporting results

04/19/23 29

Companies that are using QFD

IBM MitsubishiDigital ToyotaAT&T FordHP GMSony Daimler ChryslerGE BoeingRaytheon Lockheed Martin

Sybase - ClusteringMicro Focus - Year 2000 Solution

04/19/23 30

Entry Criteria for the Requirements Phase

There is – • A vision and an idea• A sponsor for the first phase of the project

04/19/23 31

Requirements Phase Deliverables

• Market Requirements Document• Product Specifications • High Level Project Plan• Management support for the next phase

• Process Documents and Infrastructure• Maintenance Documents and Infrastructure• Black Box and Acceptance Test Plans

• Initial contracts with potential customers and vendors

04/19/23 32

Exit Criteria for the Requirements Phase

All documents – • Are written, inspected and approved by all

stakeholders • All related issues/risks are resolved or managed• Objectives are measurable, optimized, prioritized

and approved

04/19/23 3304/19/23 33

Quality Gates – following the Requirements/Design Phases

Documents’ Inspections/Peer Reviews

• 1976 – Introduced by M. E. Fagan - IBM• A formal process• Bell Labs experience:

– Cost of inspecting all deliverables 5-20% of project development cost*– ROI of inspecting all deliverables 200-1200%*

* Bell Labs data

04/19/23 3404/19/23 34

HP – Inspection Data

Grady, R. B. and Van Slack, T., “Key Lessons in Achieving Widespread Inspection Use”, IEEE Software, V. 11, N. 4, Month, 1994, pp. 46-57

Phase Est. Starting Point

Est. Adoption 1993

% Cost Saved

% Rework

Est. $ Savings Per Year

Spec. 1% 29.5% 28.5% 17% $10,175,000

Design 1% 34.8% 33.8% 11% $7,808,000

Code 5% 42.3% 37.3% 4% $3,133,000

Test Plan 1% 17.1% 16.1% 1% $338,000

TOTAL 33% $21,454,000

04/19/23 36

The Incremental Model – Web

Requirements H.L.Design

L.L.Design

Coding

TestingTesting

Integration

Sys.Test

Delivery

L.L.Design

Coding

TestingTesting

Integration

Sys.Test

Delivery

L.L.Design

Coding

TestingTesting

Integration

Sys.Test

Delivery

04/19/23 37

The Incremental Model – Med. Device

Requirements H.L.Design

L.L.Design

Coding

TestingTesting

Integration

Sub-Sys.Test

L.L.Design

Coding

TestingTesting

Integration

Sub-Sys.Test

L.L.Design

Coding

TestingTesting

Integration

Sub-Sys.Test

Integration+Test

A “big-bang” delivery Delivery

04/19/23 38

The Iterative Model – Web Reqs+Arch Reqs+Arch. Reqs+Arch.

Design

Coding

TestingTesting

Integration

Sys.Test

Delivery

Design

Coding

TestingTesting

Integration

Sys.Test

Delivery

Design

Coding

TestingTesting

Integration

Sys.Test

Delivery

04/19/23 39

• It is a strategic phase• If this phase is not performed well the project has a

higher probability for failure• Invest about 15-20% of resources • Many organizations skip the phase or implement it

poorly• More than 15-20% change in requirements means a

new project and a new round of planning (!)

The Requirements Phase– Lessons Learnt