SPI-Software Process Improvement

7/28/2019 SPI-Software Process Improvement

1/24

Results-Oriented

Software Process Improvement

Jessee Ring

Software Quality First

40119 San Carlos Place

Fremont, CA 94539510-915-2353 Fax: 510-573-7464

Email: [email protected] Web: www.sqa1st.com


2/24

2003 Software Quality First [email protected] www.sqa1st.com 2

Results-Oriented SPI: Overview

Software Process Improvement(SPI):An effective means forimproving performance insoftware developmentorganizations.

Results-Oriented SPI: Faster,cheaper appraisals that facilitatemore timely and effectiveimprovement action. An overall

focus on results-oriented,measurable software processimprovement action.


3/24


Certification vs. Improvement

Use the process model as a

guide to makingimprovements.

All improvement projects

should be linked to business

results. Dont do anythingjust because the process

model says so.

Using the model as a criteria

for certification often

leads to irrational behavior.

Be results oriented. Metrics.

Time

Performanc

e


4/24


Getting Started on SPI

Usually Begins With An

Appraisal Appraisal have tended to

be:* Very formal;

* Done by a team;* Results presented in

verbiage only;* Identify deficiencies, not

what to do about them;* Approached from the

perspective ofcertification vs.improvement.

SOFTWARE DEVELOPMENT

AND QA ACTIVITIES

PROCESS MODEL

Questionnaire

Interviews

Doc. Review


5/24


Results-Oriented SPI: The Appraisal

) Done by one person

)Must have extensiveexperience inmultiple areas:

Systems and software

development Testing

SQA

SPI

As an individual

contributor &

manager


6/24


Results-Oriented SPI: The Appraisal

Process

Plan

Kick-off

Collect

Data

ProcessData

Evaluate

Data

WriteReport

Present

Results

Make

JudgmentsTake Action

Reference: CMM Appraisal Framework SEI publication dated Feb., 1995


7/24


Results-Oriented SPI

Time Requirements For The Appraisal

One Calendar Week

40 50 hours by the appraiser

Kick-off Meeting: 1 hour Participants: 2 3 hours during

the week

Results presentation: 1 Hour


8/24


Results-Oriented SPI: Appraisal Data

Collection

Questionnaire

Based upon the processmodel being used.

Small group discussionsGroups can be by jobfunction or cross-functional.

Document review

Selected documents, such as

requirements specs, designdocs, defect reports, CMrecords, project plan &schedule, etc.


9/24



Appraisal Questionnaire Data Analysis

2.414.30.00.00.00.00.0% of Don't Know

100000Number of Don't Know (4)

2.40.014.30.00.00.00.0% of N.A.

010000Number of N.A. (3)31.057.157.114.342.90.014.3% of No

441301Number of No (2)

64.328.628.685.757.1100.085.7% Yes

226476Number of Yes (1)

431212Person 7222211Person 6

221211Person 5

221111Person 4

221111Person 3

111111Person 2111111Person 1

RM-allRM-6RM-5RM-4RM-3RM-2RM-1Name


10/24


Achievement Ratings

A model for others.9 - 10

Fully implemented with ahistory of success.

8

Some experience andbetter results.

6 7

Making progress, gettingsome results.

4 - 5

Just starting2 -3

Nothing in place.1


11/24



Process Model Based Ratings

7

Goal 3:Affected groups and individuals agree totheir commitments related to the software project.

5

Goal 2: Software project activities andcommitments are planned and documented.

5.03

Goal 1: Software estimates are documented foruse in planning and tracking the software project.

Software Project Planning

10

Goal 2: Software plans, products and activities

are kept consistent with the system requirementsallocated to software.

10.010

Goal 1: System requirements allocated tosoftware are controlled to establish a baselinefor software engineering and management use.

Average

Per

KPARatingRequirements Management


12/24


Results-Oriented SPI: Process Model Based

Ratings Graphical Presentation

0123456789

10

RM SPP

SPTO

SQA

SCM

OPF

OPD

ISM

SPE IC PR TP

Software XYZ - Year 1 Software XYZ - Year 2

Scaleof

110with10

BeingHighes

t

KPA Ratings


13/24


Results-Oriented SPI: Appraisal

Results

Summary of Results

Overall Strengths Major Issues

Consequences

RecommendationsDetailed Results

Per KPA

Strengths Issues

Recommendations


14/24



Results - Consequences

Consequences" of the

issues that were found aredescribed.

Makes the issues of more

immediate interest. Answers the question: So

what?.

Justifies the allocation ofresources.


15/24



Results - Recommendations

Recommendations are a critical element of theResults-Oriented SPI approach. By includingthem in the Appraisal Report, people become focusedon Improvement action. It also speeds up the wholeprocess since the organization doesnt have to waitto find out what to do.

?


16/24


Example Of Results Reporting

Appraisal Results

Issue: Peer reviews are unstructured; no data orrecords are kept.

Consequences:Peer review effectiveness is

unknown but is probably low.

Recommendation: Train participants on realinspections vs. walkthroughs (with roles, requiredpreparation, record keeping). Track all items foundin inspections to closure. Use data to evaluateinspection effectiveness and make improvements.


17/24


Results-Oriented SPI: Metrics Plan

Two types of metrics: product & process

Product Metrics Number of open defects by

severity

Defect density

Defect arrival rate Total size of the software

Number of lines of code

added, deleted, modified

Number of new featuresand/or fixes contained in a

new version

Reliability growth

Complexity of the software

Process Metrics

Test effectiveness (number of

escaped bugs)

Percent of bugs found by test and

inspections.

Amount of time to find a bug by

test and by inspections.

Cycle time of regression testing.

Test coverage

Inspection coverage

Process maturity ratings over

time


18/24


Results-Oriented SPI Sample Metrics

020406080

100120140160

Version1.1 Version1.2 Version1.3 Version1.4 Version1.5 Version1.6 Version1.7Crash, lock-up, etc. Functionality - major Functionality - mediumFunctionality - minor Cosmetic

Open Defects at Time of Release

Number

ofDefects


19/24


Results-Oriented SPI: Sample Metrics

020406080

100120140160180200

Over 30 Days Over 60 Days Over 90 DaysCrash, lock-up, etc. Functionality - major Functionality - mediumFunctionality - minor Cosmetic

Defect Aging

Numbe

rofDefects


20/24


Results-Oriented SPI: Sample Metrics

0100200300400500600700800900

1000

Jan. Feb. Mar. Apr. May June July Aug. Sep. Oct. Nov. Dec.

Mean time between failure

Software Reliability Growth

Ho

urs


21/24


The Need For Software Process

Improvement

Mars Climate Orbiter

Lost because of failure to convert from English to metric units in thenavigational (orbiting) program.

Mars Polar Lander

Lost because of a missing line of code that caused the braking

engines to shut down prematurely.

Result

$360 million out-of -pocket cost

Indefinitely postpone all future launches to Mars

Address shortcomings at JPL in areas ranging from systems analysisand testing to staffing and communications.


22/24


Results-Oriented SPI: Summary

Use one very experienced person to perform asoftware process appraisal.

Give numeric ratings to achievement levels ofprocess model requirements.

Use the process model as a guide; not a mandate.

Link all process improvements to business results.

Use process maturity ratings to guide improvementactions, not as a final goal. Dont publish them.

In the process appraisal report, identifyconsequences of the issues that were found, and

recommendations for addressing them. Use metrics to manage the improvement actions.

KISS


23/24


Results-Oriented SPI: Benefits

Faster, less expensive appraisal.

A sense of urgency. Can start on improvement immediately after appraisal (dont

have to wait on another team to formulate a plan).

Can track process maturity levels over time graphically.

Improvement actions are focused on the issues that willimpact the bottom line.

Participant see it as being worthwhile.

Greater likelihood of upper management support for SPI. All of which leads to a successful improvement program.


24/24



SPI-Software Process Improvement

Documents

Transcript of SPI-Software Process Improvement