A Comparison ofShewhart and CUSUM Methodsfor Diagnosis in a Vendor Certification Study

Erwin M. SanigaDept. of Bus. Admin.

University of DelawareNewark, DE 19716

302-831-2555sanigae@be.udel.edu

James M. LucasJ.M. Lucas and Associates

302-368-12145120 New Kent RoadWilmington, DE 19808

Jamesm.lucas@worldnet.att.net

Darwin J. DavisDept. of Bus. Admin.

University of DelawareNewark, DE 19716

302-831-2555davisd@be.udel.ed

Presenter: Erwin Saniga

Purpose: To examine the performance of alternative methods to study processes where quality is measured by counts and counts are low.

Example Large Wilmington (DE) area credit card bank Processes credit card applications Four vendors process these applications Wish to implement a vendor certification and

quality improvement program

BankVendorA

VendorD

VendorB

VendorC

We will show in this presentation:

When exploring this type of data for performance evaluation or process capability analysis, four different types of plots can reveal different things about the process A traditional sequence plot Adding a Shewhart UCL and a

method to detect improvements to the traditional sequence plot

A CUSUM plot Adding a “V-mask” to the

CUSUM plot

Actual results for the four vendors

Each point represents the number of defectives resulting from an inspection of 50 random credit card applications that were processed during the day. These were taken at the end of the day.

Each credit card application can be processed correctly or incorrectly.

Vendor Analysis

Average vendor performance Vendor A average p = 0.0246 Vendor B average p = 0.0510 Vendor C average p = 0.0247 Vendor D average p = 0.0294

Analyst questions: What caused the spikes at various points in time for

each vendor? What caused the sequence of “good” (zero

defective) samples for various vendors? Why is Vendor B doing poorly when compared to

Vendors A, C, and D? Are these substantive differences?

If the data were available in real time and we could plan the data collection we might:

Investigate the cause of a spike or run of good points immediately

Keep a diary or log of variables identified during a focus group meeting of employees, managers, etc.

CUSUM SEQUENCE (DIAGNOSTIC) PLOTS

Let Xj = the actual number of defectives observed in the jth sample

The ith CUSUM is then:

where k is the reference value. We use a reference value of

k=1.25 which is the average count of defectives in a sample of 50 for the three “good” vendors.

i

1jji kXC

Process Averages for CUSUM Sequence Plot

The CUSUM sequence plot can identify “good”, “average” or “bad” regimes.

Regime average is determined by the slope (in this case, slope = 0 implies 1.25 defects).

The average count from periods L to M is given by:

Vendor A ExampleRegime 1 – Days 46 to 122Ave. count = 1.25 + [-27.5 – (-0.25)]/[122 – 46 + 1]

= 0.896Regime 2 – Days 176 to 199Ave. count = 1.25 + (6.25 – (-21.75)/(199 – 176 + 1)

= 2.417

1LM

CC25.1CountAverage 1LM

CUSUM Sequence Plot Summary

There are various “regimes’ noted by the CUSUM sequence plots that are not immediately recognizable from the traditional sequence plot of the counts.

Some of these regimes indicate notable “good” or “bad” performance.

CUSUM sequence plot questions: What are the reasons for the good

performance in certain regimes? What are the reasons for the bad

performance in certain regimes? What happened on the particular days a

change point was observed?

If the data were available in real time we might keep a diary or log of variables possibly associated with performance and investigate these.

Vendor A Comparison

A comparison of the CUSUM sequence plot with the traditional sequence plot

Traditional sequence plot: spikes at 35, 128, 157, 169, 193, and 197

zero counts from 200 to 207 consecutively

CUSUM sequence plot:“Average” process on days 1-14“Good” process on days 15 to 32“Bad” process on days 33 to 45“Good” process on days 46 to 122“Average” process on days 123 to 175“Bad” process on days 176 to 199“Good” process on days 200 to 207

Summary of plots

Traditional sequence plots are good for detecting shocks to the system and rare events

CUSUM sequence plots are good for detecting regimes (periods of good, bad, and average behavior)

Other Questions

For traditional sequence plots: Are the spikes unusual when

compared to what might happen under pure chance?

Are the zero count sequences unusual in the sense that they indicate the process has improved?

For CUSUM sequence plots: Are the regimes we observed unusual

when compared to what might happen under pure chance?

Traditional Sequence Plot

Shewhart chart provides a simple and effective way to “signal” spikes as being significantly unusual.

Control Limits for the Shewhart chart:

where p is the average proportion defective.

)p1(np3npLCL,UCL

The Upper Control Limit

For our data with an average count of 1.25 (p=0.025, n=50) we have

UCL =4.56 (the Shewhart 3 sigma limit)

In our examples we use UCL = 5 (signal at 6) to ensure the ARL in control is sufficiently large.

For UCL= 4 (signal at 5) ARL in control = 123For UCL= 5 (signal at 6) ARL in control = 662

(Generally, for low count data adding 1 to the UCL yields a more desirable in control ARL)

The Lower Control Limit

LCL = -2.06 (no lower control limit)Solutions for the lower side (detecting

improvement) Ryan(1989) and Schwertman and Ryan(1997)

suggest equal tail probability methods. These do not work well when P(0) is high:

In control ARL is low or, equivalently, false alarm probabilities are high.

Acosta-Mejia (1999) suggests counting successive results below a modified centerline Dominated by CUSUM methods in terms of

ARL. Does not work well for P(0) large. CUSUM methods (e.g. Reynolds and

Stoumbos (1999, 2000) Optimal for a particular shift in terms of

ARL yields Harder to design and use than special

CUSUMs we will provide.

Count the Zeros: Special CUSUMs

The simplest low side CUSUMs

Method 1: Signal if k in a row samples have zero defectives

Method 2: Signal if 2 in t samples have zero defectives

Properties: Optimal for large shifts Easy to use Easy to design Easy to understand

Low Side Schemes

Comparison of our special CUSUM (k in a row) and the CUSUM.

Recommended Policies and ARL Formulas for Low Side Attribute Schemes with

Low Side In-Control ARL 500.

P(0) Recommended Policy

for Low Side Attribute Scheme

Low Side ARL Formula

P(0) 0.046 k in a row

(k 2)

1k

ojk

j

)0(P

)0(PARL

0.002 < P(0) 0.046 2 in t (t 2) ])]0(P1[1[*)0(P

)]0(P1[2ARL

1t

1t

P(0) 0.002 Shewhart Lower Control Limit

1

1ARL

The Computational CUSUM

yi = the observed count

For the high side Si = Max {0, Si-1 - kH + yi }

If Si > hH in a signal is given

For the low side Si = Max {0, Si-1 + kL - yi }

If Si > hL in a signal is given

For our example we use:kH = 2 hH = 4

kL = 0.75 hL = 5.25

Asymmetric V-Mask

hL

hH

Slope = kH – reference value

Slope = kL – reference value

LastCusum Value

Comparison of Signals

Vendor A

CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side

35 S 35 S 205 R 45 R 157 S 157 S 197 S

180-197R(2 signals) 207 R

Vendor B

CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side

37-72 R (9 signals) 37 S 91-101 R (3 signal) 46 S

52 S 55 S 58 S 63 S 72 S

106-123 R (13 signals)

106-123 R (8 signals)

Comparison of Signals

VENDOR C

CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side

72 S 12 R 55 S 12-19 R 89 R 103 R 72 S 101-104 R

106-107 R 89 S 105-107 R

(3 signals)

VENDOR D

CUSUM SIGNALS SHEWHART SIGNALS High Side Low Side High Side Low Side

30-44 R (5 signals) 30 S 55, 56 R 116-120 (2 signals) 38-423 R 147 R 170-173 (2 signals) 58 S

116 S 120 S 170 S

We have shown…

Four different types of plots can reveal different things about a process A traditional sequence plot Adding a Shewhart UCL and a

method to detect improvements to the traditional sequence plot

A CUSUM plot Adding a “V-mask” to the

CUSUM plot