SPC Training Material Ver 4.0

8/10/2019 SPC Training Material Ver 4.0

1/56

TCS Confidential Presentation1

Statistical Process Control Version 4.0July 2001

Statement of ConfidentialityThis Training material contains information that is proprietary and confidential to

TATA Consultancy Services.


2/56


Course CoverageThis course on SPC covers the following topics:

Overview of basic conceptsExploration on Process and Product QualityQuantitative Process Management/Software QualityManagement overview, Quality GoalsShewart/Deming Measurement Process and SiteImplementationComputation Formulas for Individuals XmR and u-chartsControl Chart Pattern AnalysisControl Limit Drawing Guidelines


3/56


What is a Process? Process applies to any sequence of stepsperformed for a given purpose [IEEE -STD-610]whether they are the software development

process or a subset within the softwaredevelopment processProcess implies the way we do things. In thesoftware context, it implies the way we do

software. Processes produce two outputs:Product of serviceData


4/56


QPM and SQMWhile QPM focuses on quantitativelymanaging the process quality, Software

Quality Management (SQM) focuses onmanaging the product quality.

QPM Process Quality SQM Product Quality


5/56


Process and Product QualityProcess quality refers to the quantification of processcharacteristics whose values indicate a degree of quality ofthe underlying process

For example, we can use Review Effectiveness to quantifythe efficiency of the review process , or the ScheduleSlippage to quantify the efficiency of our process by whichwe deliver our products in timeProduct quality refers to the quantification of product

characteristics whose values indicate a degree of quality ofthe productFor example, we can use Defect Density to quantify thequality of our software deliverables ( how many defects arewe delivering?)


6/56

TCS Confidential Presentation

6

Why Quantitative Management?

Quantitative management is about makingdecisions based on sound informationQuantitative management is aboutmanagement by fact Facts are derived out of information Information is the interpretation of data Data comes from processes


7/56


8/56TCS Confidential Presentation

8

Quantitative Process ManagementQuantitative Process Management involves:1. Establishing goals for the project s defined software process 2. Taking measurements of process performance (project data)3. Analyzing these measurements4. Making adjustments to maintain process performance

(project s UCL and LCL) within acceptable limits

The organisation collects data from the software projects anduses these data to come up with the PCB of the OSSP. PCBdescribes the range of expected results from following asoftware process. These PCB data are used by the projectsto analyse the performance of the projects defined software

process



9

Software Quality ManagementThe purpose of Software Quality Management is todevelop a quantitative understanding of the quality

of the project s software products and achievequality goals.SQM involves:1. Defining quality goals for the software products

2. Establishing plans to achieve these goals3. Monitoring and adjusting software plans, work

products, activities and the quality goals to satisfy theneeds and desires of the customer and end user



10

What is a Quality Goal?

Quality Goals apply to the SQM Key Process

Area1. A quality factor is defined for the softwareproduct

2. A quantitative measure for the factor is

established3. A target or desired value of the factor is

established - this constitutes the Quality Goal



11

Scenario 1: A system must be designed such that its backend response time is not more than 5

milliseconds.Here:

Quality factor = Performance(Responsiveness)Quality goal = < 5 milliseconds

What is a Quality Goal?



12

Quality Goal - ExampleScenario 2:

A module that is to be delivered should nothave more than 5 defects/KLOC at the timeof delivery

Here:Quality factor = Reliability

Quality goal = < 5 defects/KLOC



13

QPM and SQM - Integrated ViewQuantitative Process Management

Focuses on Process Quality

Process performance ismeasured

Measurements on processmetrics result in changes to thesoftware process (OPD)

Software Quality Management

Focuses on Product Quality

Product quality is measured

Measurements on qualitygoals result in changes tosoftware quality plan, workproducts, activities



14

QPM and SQM - Implementation An integrated implementation of the key practices ofQPM and SQM is followed.

Process MetricsReview Effectiveness % Rework Effort% Schedule slippage % Review Effort% Effort slippage %Size Deviation

Product MetricsDefect density% Bad FixesTDCE



15

QPM and SQM - Implementation

SQM and QPM activities are outlined in the Metrics Section ofthe Software Quality Assurance Plan or UPP.

Branch level Targets are used as performance limits for theprocess metrics and quality goals for the product metricsOther product quality goals other than the ones described maybe defined for projects:

Example:Productivity



16

Shewart/Deming Cycle, Measurement Process and Site Implementation

Shewart/Deming Cycle Activity in the software measurementprocess Purpose

Plan Identify scope, define procedures Establish a purpose, plan, andprocedure for measurement tied toorganizational goals

Do Collect data Execute the plan to collect the data

Check Analyze the data Analyze the data to determine theprogress towards measurementgoals

Act Evolve process Make decisions based on theprogress towards goals andrecommend improvements on themeasurement process being used



17

What is Statistical Process Control?

Statistical Process Control (SPC) is a method todefine operational limits for acceptable variation inprocess data, and to examine the processperformance in absolute statistical terms

SPC and its associated control charts weredeveloped by Walter A Shewart in the 1920s togain control of production costs and quality



18

What is new in SPC? Statistical Process Control is NOT a new concept

SPC is a widely used technique in other domains ofengineering - such as manufacturing.

What is new is the application of statistical processcontrol techniques to Software Engineering andSoftware Project Management



19

Process Capability

3s3s

Projects

LCL

Projects

UCL

BranchsUCL

BranchsLCL

ProcessCapabilityBaseline

Stretched Targets

LST UST

All points within projects

fixed UCL/LCL impliesprocess is stable

All points within branchs

fixed UCL/LCL (PCB)implies process is capable



20

Basic Definitions UCL : Upper Control Limit

LCL : Lower Control Limit

USL : Upper Specification Limit

LSL : Lower Specification Limit

UST : Upper Stretched TargetLST : Lower Stretched Target

PCB: Process Capability Baseline

When PCB isestablished,the Spec Limits of the

centre are usually set asthe control limits

derived as per the PCB

Set by Centre Head as aLong term Goal

Not derived from data

Derived from data



21

What is an Outlier?

In simple terms, it is a data point that fallsoutside the Control Limits. It indicatesdisturbance or abnormality in the process.


22/56


22

Basic Definitions A process is said to be stable if there are no outliers wrt UCL

and LCL of the process. If points lie outside the control

limits, the process is unstable and is said to be out of

control.

A process is said to be capable if there are no outliers wrt

Specification Limits.

SEPG establishes Process Capability Baseline(PCB) when the

process is stable and Capable .This is verified at the end of

every three months


23/56


23

How do you carry out the Process

Measurement and Analysis Activities in theprojects?


24/56


24

SPC Analysis at project Level

Analyze and Correct Process andthen Prevent it from Recurring

Now the Process is said to bestable at the project level

Analyze and Correct outliers wrtSpec Limits

No Outliers wrt Spec Limits,Process is Stable and Capable

Now Fix the UCL and LCL at the project level as Process

Performance Baseline(Fixed UCLand LCL)Continue for the next month

Look for any outlier wrt ProjectsControl limits

Collect Process Performance data


25/56


25

How are Process Measurement and Analysis

Activities carried out at the organization

level?


26/56


26

SPC Analysis at Organisation Level SEPG collects data from projects

on a monthly basis

SEPG consolidates data

SEPG looks for any Outliers wrtcurrent CLs and Spec Limits of

the Organisation

SEPG along with PLs analyzesand corrects these outliers

SEPG fixes the Control Limitsand they form Spec Limits forthe later months

Projects in turn use the SpecLimits as their targets (USL and

LSL )

Spec Limitsare

Quality Goals


27/56


27

Process Improvement and Distribution Spread

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

-150 -100 -50 0 50 100 150 200 250

Metricvalue

P r o

b a

b i l i t y

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

-150 -100 -50 0 50 100 150 200 250

Metr icv alu e

P r o b a b i l i t

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

-150 -100 -50 0 50 100 150 200 250

Metricvalue

P r o b a b i l i t y

Process Improvements

More Process Improvements

The bell-curve becomes:Taller Thinner with more process improvements


28/56


28

Types of SPC charts

Run ChartsControl charts

X-mR chart Attribute charts p-chart np-chart

c-chart u-chart z-chart

Not all SPC techniques may be equally applicable

for software processes


29/56


29

Computation formulae for theIndividuals X-mR chart

Characteristic:Each individual measurement is used andplottedMoving range values are absolute values of

the present measurement minus the previousmeasurement


30/56


30

Computation formulas for theIndividuals X-mR chart

X-bar = X-avg = X/nR-bar = R-avg = R/(n-1)

UCL [X] = X-bar + 3*Sigma= X-bar + 3*R-Bar/d2 where d2 = 1.128= X-bar + 2.66 * R-bar

LCL [X] = X-bar 2.66 * R-barUCL [mR] = 3.267 * R-barLCL [mR] = 0 * R-bar = 0


31/56


31

Some control charts - Organisation

LevelTCS, Bangalore -

sensitive and confidentialdata

X Chart Schedule Slippage

-20

-15

-10

-5

0

5

10

15

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Components

% S

c h S l i p p a g e

Slippage/component Xbar Current UCL of the ProjectCurrent LCL of the Project UST (Upper Stretch Target) Project's Prev (Fixed) UCLProject's Prev (Fixed) LCL Upper Spec Limit Low er Spec LimitLST (Low er Stretch Target)


32/56


32

Causes of Variation Random or Common

Normally, we cannot do anything to change thisContinuously active and part of the processProcess performance noise

Special or AssignableWe can do something to change this

Not always active or part of the process Are extraordinary events Process non performance signals


33/56


33

System Causes of Variation

When variation is due only to system causes andthere is a sufficient volume of data, the variationswill tend to take a normal (bell shaped) distributioncentered around a middle measureIf special causes are present, the curve will bedistorted and lose its normal distribution


34/56


34

Special Causes of Variation

identifying variations in performance that are outside the boundsof process capability; i. e., the normal range of processperformanceidentifying patterns in system variations above or below theaverage

Both require attention and potentially corrective action

average


35/56

12/29/2014 TCS Confidential Presentation

35

How to Interpret?

Special causesIs it explainable ? (PL, GL to investigate)Can we prevent recurrence of it?

Common causesPareto, FishboneCorrective actions at Project Level and/or Branch Level


36/56

12/29/2014 TCS Confidential Presentation

36

Examples

Special causesShifts in the quality of raw materialsInadequately trained peopleChanges to work environmentstool failures, altered methods, failures to follow theprocess, etc.,

Common causesProblem in the processProcess is not clearly defined


37/56


37

Control & Specification Limits

OK Not OK

stable and capable stable but not capable


38/56


38

Using control charts to correctthe process

Correcting the process involves control and/or:removal of assignable causes

use of the chart as a dynamic picture of quality over timepsychological effect on the personnel

The whole process of using control charts is to:gain finer and finer control

identify and correct assignable causesperfect procedures and attitudes that ensure the maximumutilization of the process capabilities


39/56


39

Control chart pattern analysisSeveral distinct patterns can be recognized incontrol charts which can be potential indicators orsignals of instability.

The reasons of the selection of patterns is basedon sound mathematical principles behind controlchart theory and probability theory.

Once a distinct unstable pattern is identified, thesearch for an assignable cause must be initiatedand corrected to improve the process.


40/56


40

Basic PathologiesThe following are the basic pathologies for processinstabilities:Points outside the 3 control limits

7 point patterns7 points in a row above the mean7 points in a row below the mean7 points in a row alternating up and below the mean7 points in a row going up7 points in a row going down

Points between + 1 5 points in a row between + 1 from the mean6 of 7 points in a row between + 1 from the mean


41/56


41

Basic Pathologies

Points between + 1 and + 2 3 points in a row between + 1 and + 2 4 of 5 points in a row between + 1 and + 2

Points between + 2 and + 3 2 points in a row between + 2 and + 3

10 of 11 patterns10 of 11 points in a row above the mean10 of 11 points in a row below the mean10 of 11 points in a row alternating up and below the mean10 of 11 points in a row going up10 of 11 points in a row going down


42/56


42

Basic Pathologies12 of 14 patterns

12 of 14 points in a row above the mean

12 of 14 points in a row below the mean12 of 14 points in a row alternating up and below the mean12 of 14 points in a row going up12 of 14 points in a row going down


43/56


43

Control Limits RecalculationThe Upper and Lower Control Limits are theboundaries of process stability.

A useful control chart can be drawn only with aminimum of 20 points for a software process.

The general guidelines for control limit revision aredescribed in the following charts.


44/56


44

When to draw a control chartIf you have less than 20 points, DO NOT DRAW a controlchart.

Draw a run chartUse the branch s UCL and LCL to see whether your process iscapable or not

Once you have 20 or more pointsDraw a control chart and calculate the control limits. Treat theselimits as fixed for the further data pointsUse the control limits to analyze process stabilityUse the branch s UCL and LCL to analyze process capability

Whenever a new data point is available Add it to the control chart dataRedraw the control chart BUT DO NOT RECALCULATE THECONTROL LIMITS - USE THE OLD (FIXED) CONTROL LIMITSINSTEAD


45/56


45

When to draw a control chartContinue using the old (fixed) control limits until youbring the process under control (with respect to theold (fixed) control limits) or a prescribed period of

time (say 3 months)When the process is found to be within control (orafter the prescribed period of time), recalculate thecontrol limits and fix them as the new thresholds for

future data pointsContinue this FIX -DRAW process to analyzeprocess stability and capability


46/56


46

Why should you fix the control limits?

You should not recalculate the control limitswhenever a new data point is added because, the

single data point may change the control limits andalter your perspective of process stability.

This becomes important if the new data point

indicates an out-of-control process and may benoise in the process. If you revise the controllimits, you will not get a good picture of processstability.


47/56


47

You initially have 8 data points ...

Run Chart (defecs/PD)

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6 7 8

Instance

D e

f e c

t / P D

Defect density (defects/pd) Branch's LCL Branch's UCL

You have less than 20 points - so draw a runchart, use Branchs UCL and LCL


48/56


48

12 more points are now available... You now have 20 data points - so draw a control chart,calculate the control limits (process shown to be in control)UCL

Calculated


49/56


49

2 more data points are now available... You now have 22 datapoints - so draw acontrol chart, but DONOT RECALCULATETHE CONTROLLIMITS, use the OLDONES INSTEAD -(process shown to bein control using oldlimits)

UCLnot calculated,carried over


50/56


50

If you had recalculatedthe control limits...

Process is out of control becausecontrol limits were revised with the

addition of two data pointsUCLrecalculated withthe addition of 2more points

Process is now out of control!


51/56


51

An example of recalculating control limitsat each point

Example of control limits recalculated at each data point

0

2

4

6

8

10

12

14

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Instance

D e f e c t D e n s i t y ( D e f e c t s / P D )

Upper controllimit recalculatedat each point


52/56


52

U Charts

U stands for per unit Attribute Control Chart i.e. when sample reflects qualitativecharacteristics such as defective/not defective, go/no goSample size variesData distribution follows Poisson distribution (discrete values)Control Chart for non-conformities or defectsVarying UCL and LCL


53/56


53

Calculating U Charts

Ubar = d i / n i UCL U = U bar + 3 sqrt (U bar / n i)LCL U = U bar - 3 sqrt (U bar / n i) or 0 for less than 0

where n i = Size of i th sampled i = Non-conformities in i th sample

Using U chart for Defects/pd:

n i = Planned effort for the work item id i = Defects detected in the work item i


54/56


54

Example of U Chart

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Time

D e f e c t s / p d

UCL

LCL


55/56


55

Metrics under SPC

Project Level Metric Chart type

Schedule Slippage - Run/ XmREffort Slippage - Run/ XmRDefects/pd - Review + UT - U Chart

Organisation Level Schedule Slippage - XmR ChartEffort Slippage - XmR Chart

Defects/pd for Code & Document - U ChartReview Effectiveness - Xmr ChartTotal Defect Containment Effectiveness - XmR ChartRework Effort - XmR ChartReview Effort - XmR Chart


56/56

56

Thank you !

SPC Training Material Ver 4.0

Documents

Transcript of SPC Training Material Ver 4.0