Lecture 2.11 © 2016 Michael Stuart Design and Analysis of Experiments Lecture 2.1 1.Review...

Lecture 2.1 1© 2016 Michael Stuart

Design and Analysis of ExperimentsLecture 2.1

1. Review– Minute tests 1.2– Homework– Experimental factors with several levels

2. Analysis of Variance3. Randomised blocks design:

illustrations4. Randomised blocks design and analysis:

case study

Postgraduate Certificate in Statistics Design and Analysis of Experiments


Minute Test: How Much



Minute Test: How Fast



Exercise 1.2.1Process Development Study

Process: pellet makingRequirement: specification limits for pellet sizeProblem: proportion meeting specification

too lowProposal: change machine speed from A to B




Replication

Naive experiment:run process once at speed A,run process once at speed B,calculate response difference

Q: is response difference due tochange

orchance?



Replication

Improved experiment:run process twice at speed A,run process twice at speed B,calculate mean response at each speed,

difference in mean responsesmeasures change effect

calculate response difference at each speed,mean of response differences measures chance effect


Process Development Study


Mean 78.75 75.25

Difference 5.1 3.5

Speed B Speed A

76.2 73.581.3 77.0


Exercise 1.2.1

Formal test:

Numerator measures change effect,

Denominator measures chance effect.

Carry out the test using the results from the first two runs at each speed. Compare with test using complete data

n/s2yyt

2AB



Process Development Study

Variable N Mean StDev

Speed B 2 78.75 3.61Speed A 2 75.25 2.47

Speed B Speed A

76.2 73.581.3 77.0



Replication

Two measurements per sample provides a valid test– but not a powerful test

More measurements per sample provides – more precision in estimating within-sample variation,

i.e., estimating sand, therefore,

– more power in testing between-sample variation.Recall the discussion in Base Module Chapter 4:

– 11 replications needed to detect a 5% improvement







case study



A multi-level experimental factorFilter membrane improvement project

Four membrane types:

A: current standardB: newly developed alternativeC: OEM 1D: OEM 2

Criterion: failure pressure level (kPa)

Objectives: (i) is Type B better than Type A?

(ii) are OEM membranes better?



Filter membrane improvement project

Procedure: from each of 10 production batchesof each membrane type,sample 5 membranes,

for each sample of 5, run the filtering process using each membrane, increasing pressure until membrane failure,

calculate sample mean failure pressure reading.



Filter membrane improvement projectthe response

the experimental factor

the factor levels

the treatments

an experimental unit

an observational unit

unit structure

treatment assignment

replication

burst strength

membrane type

A, B, C, D

A, B, C, D

5 process test runs

process test run

simple

no information

10Postgraduate Certificate in Statistics Design and Analysis of Experiments


ResultsMean burst strengths (failure pressure level, kPa) of

10 samples from each of 4 filter membrane types

Classwork 1.2.3 Make dotplots of the breaking strengths

Membrane A Membrane B Membrane C Membrane D 95.5 90.5 86.3 89.5 103.2 98.1 84.0 93.4 93.1 97.8 86.2 87.5 89.3 97.0 80.2 89.4 90.4 98.0 83.7 87.9 92.1 95.2 93.4 86.2 93.1 95.3 77.1 89.9 91.9 97.1 86.8 89.5 95.3 90.5 83.7 90.0 84.5 101.3 84.9 95.6

Ref: Membrane strength.xls



Initial data analysisBurst strength (kPa) of 10 samples

of each of four filter membrane types

Variable Membrane N Mean StDev Minimum Maximum RangeStrength A 10 93 4.8 85 103 19 B 10 96 3.4 91 101 11 C 10 85 4.3 77 93 16 D 10 90 2.8 86 96 9

1051009590858075

A

BC

D

Strength

Membran

e







case study



One-way ANOVA: Strength versus Membrane

Source DF SS MS F PMembrane 3 709.2 236.4 15.54 0.000Error 36 547.8 15.2Total 39 1257.0

S = 3.901

F3,36;0.05 ≈ 2.85

Conclusion:

Differences between means are highly statistically significant:

process variation has standard deviation of almost 4.

Comparing several means





S = 3.901

F3,36;0.05 ≈ 2.85

Conclusion:

Differences between means are highly statistically significant:

process variation has standard deviation of almost 4.

Comparing several means



Analysis of Variance Explained

Decomposing Total Variation

Expected Mean Squares

Ref: Base Module Chapter 5, §5.1



Decomposing Total VariationElementary decomposition:

Analysis of Variance decomposition:

SSTO = SSM + SSE DFTO = DFM + DFE

)yy()yy(yy iijiij

totaldeviation

membranedeviation

errordeviation= +

dataall

2iij

dataall

2i

dataall

2ij )yy()yy()yy(




Classwork 1.2.4

Confirm the degrees of freedom (DF) and sum of squares (SS) decompostion and confirm the calculation of the mean squares and the F-ratio in the membrane analysis of variance table.






All mi = m ↔ E(MSM) = E(MSE)

All mi ≠ m ↔ E(MSM) > E(MSE)

Hence, MSM ≈ MSE suggests all mi ≈ m, and

MSM >> MSE suggests all mi ≠ m.

F = measures by how much MSM exceeds MSE

1I)(

J)MSM(E2

i2

mms

2)MSE(E s

MSEMSM



Multiple comparisons



Multiple comparisons

Confidence interval for difference between means:

If 0 is not within the interval,then

0 is more than 2SE fromso

is more than 2SE from 0,that is

means are statististically significantly different


SE2YX

YX

YX


Interpreting multiple comparisons

• Membrane B mean is significantly stronger than Membranes C and D means and close to significantly stronger than Membrane A mean.

• Membrane C mean is significantly less strong than the other three means.

• Membranes A and D means are not significantly different.

Membrane Type

Mean Strength

B 96.08 A 92.84 D 89.89 C 84.63



Multiple comparisons explained

• Simultaneous confidence intervals slightly wider than individual confidence intervals.– level of confidence in

• several intervals simultaneouslyversus– level of confidence in

• a single interval; – more opportunities for being wrong

• Widening intervals increases confidence.– extent of widening chosen to compensate for

reduction in confidence involved.Postgraduate Certificate in Statistics Design and Analysis of Experiments


Diagnostic analysis

95.092.590.087.585.0

10

5

0

-5

-10

Fitted Value

Res

idua

l

10

5

0

-5

-10210-1-2

Res

idua

l

Score

N 40AD 0.736

P-Value 0.051

Versus Fits(response is Strength)

Normal Probability Plot(response is Strength)



Report


Reminder of objectives of the experiment:(i) is the company's newly developed membrane Type

B better than the standard Type A?(ii) is there any advantage in introducing other

companies' membranes?

Answer (ii): NOAnswer (i): "Some evidence" that B is better than A,

but other factors may be more important.Possibly make further comparisons.


Brief management report

Membrane Type C can be eliminated from our inquiries.

Membrane Type D shows no sign of being an improvement on the existing Membrane Type A and so need not be considered further.

Membrane Type B shows some improvement on Membrane Type A but not enough to recommend a change.

It may be worth while carrying out further comparisons between Membranes Types A and B.



Fisher on Analysis of Variance Table

"a convenient method of arranging the arithmetic"

(so don't show it in management reports!)







case study



Part 3Randomized complete blocks design

Example 1: treating crops with fertiliser to improve yield.

Four fertilisers being tested:

divide a single field into four plots (experimental units) to form one block,

assign treatments at random to the four plots,

repeat with several other fields to form several blocks,

choose blocks in varying locations, for generalising.



Blocks of experimental plots at Rothamstead

© Rothamsted ResearchPostgraduate Certificate in Statistics Design and Analysis of Experiments


Randomized blocks design

Example 2: treating long spools of rubber to improve abrasion resistance.

Four treatments being tested:

cut a single piece into four experimental units to form one block,

assign treatments at random to the four units,

repeat with several other pieces to form several blocks.




Piece 1

B

Piece 2

A

Piece 3

C

Piece 4

D

Block 1Postgraduate Certificate in Statistics Design and Analysis of Experiments



Piece 1

A

Piece 2

D

Piece 3

C

Piece 4

B

Block 2Postgraduate Certificate in Statistics Design and Analysis of Experiments



Block 3

Piece 1

B

Piece 2

A

Piece 3

D

Piece 4

C




Block 1 Block 2 Block 3 Block 4 etc.

Blocking accounts for anticipated variation patterns along the length of the spool of rubber

Randomization allows for unanticipated sources of variation within blocks,

e.g., side to side, diagonal, any other

B A C A A D D B

D C B D C B C A




Example 3: assessing process changes.

Five versions of the process being assessed:

assess the five versions on five successive days in a working week,

Randomize the time order in which the versions are used,

repeat over several weeks to form several blocks.

NB: Pairing = Blocking with two units per blockPostgraduate Certificate in Statistics Design and Analysis of Experiments


Randomized block design

Where replication entails increased variation, replicate the full experiment in several blocks so that• non-experimental variation within blocks is as

small as possible,

– comparison of experimental effects subject to minimal chance variation,

• variation between blocks may be substantial,

– comparison of experimental effects not affected



Illustrations of blocking variables

Agriculture:

fertility levels in a field or farm,

moisture levels in a field or farm,

genetic similarity in animals, litters as blocks,

etc.




Clinical trials (stratification)

age,

sex,

height, weight,

social class,

medical history

etc.




Clinical trials

body parts as blocks,

hands, feet, eyes, ears,

different treatments applied to the same individual at different times,

cross-over, carry-over, correlation,

etc.




Industrial trials

multiple machines,

multiple test laboratories,

time based blocks,

time of day, day of week, shift

etc.







case study



Case StudyReducing yield loss in a chemical process• Process: chemicals blended, filtered and dried• Problem: yield loss at filtration stage• Proposal: adjust initial blend to reduce yield

loss• Plan:

– prepare five different blends– use each blend in successive process runs, in

random order– repeat at later times (blocks)



Classwork 1.2.5: What were the

response:experimental factor(s):factor levels:treatments:experimental units:observational units:unit structure:treatment allocation:replication:



Classwork 1.2.5: What were the

response:experimental factor(s):factor levels:treatments:experimental units:observational units:unit structure:treatment allocation:

replication:

yield lossBlendA, B, C, D, EA, B, C, D, Eprocess runsprocess runs3 blocks of 5 unitsrandom order of blends within blocks3



Unit Structure

Block 1 Block 2 Block 3


Unit 1Unit 2Unit 3Unit 4Unit 5




Unit Structure

Block 1 Block 2 Block 3


Unit 1_1Unit 1_2Unit 1_3Unit 1_4Unit 1_5



Blocks

Units

Units nested in Blocks


Randomization procedure1. enter numbers 1 to 5 in Column A of a spreadsheet,

headed Run,

2. enter letters A-E in Column B, headed Blend,

3. generate 5 random numbers into Column C, headed Random

4. sort Blend by Random,

5. allocate Treatments as sorted to Runs in Block I,

6. repeat Steps 3 - 5 for Blocks II and III.

Go to ExcelPostgraduate Certificate in Statistics Design and Analysis of Experiments


Part 2 Randomised blocks analysis

• Exploratory analysis• Analysis of Variance• Block or not?• Diagnostic analysis

– deleted residuals• Analysis of variance explained



Results

Ref: BlendLoss.xls

Block Run Blend Loss, per cent I 1 B 18.2 2 A 16.9 3 C 17.0 4 E 18.3 5 D 15.1 II 6 A 16.5 7 E 18.3 8 B 19.2 9 C 18.1 10 D 16.0 III 11 B 17.1 12 D 17.8 13 C 17.3 14 E 19.8 15 A 17.5



Initial data analysis

• Little variation between blocks• More variation between blends• Disturbing interaction pattern; see laterPostgraduate Certificate in Statistics Design and Analysis of Experiments


Formal Analysis

Analysis of Variance: Loss vs Block, Blend

Source DF SS MS F PBlock 2 1.648 0.824 0.94 Blend 4 11.556 2.889 3.31 0.071Error 8 6.992 0.874Total 14 20.196

Classwork 1.2.6: Confirm the calculation of• Total DF, • Total SS, • MS(Block), MS(Blend), MS(Error) • F(Block), F(Blend)



Formal Analysis



Classwork 1.2.6: Confirm the calculation of• Total DF, • Total SS, • MS(Block), MS(Blend), MS(Error) • F(Block), F(Blend)



Assessing variation between blendsF(Blends) = 3.3F4,8;0.1 = 2.8

F4,8;0.05 = 3.8

p = 0.07F(Blends) is "almost statistically significant"

Multiple comparisons:All intervals cover 0;Blends B and E difference "almost significant"

Ref: Lecture Note 1.2, p. 20.Postgraduate Certificate in Statistics Design and Analysis of Experiments


Assessing variation between blocks

F(Blocks) = 0.94 < 1; MS(Blocks) < (MS(Error)

differences between blocks consistent with chance variation;

Source DF SS MS F PBlock 2 1.648 0.824 0.94 0.429Blend 4 11.556 2.889 3.31 0.071Error 8 6.992 0.874Total 14 20.196

Source DF SS MS F PBlend 4 11.556 2.889 3.34 0.055Error 10 8.640 0.864Total 14 20.196Postgraduate Certificate in Statistics Design and Analysis of Experiments


Was the blocking effective?

Source DF SS MS F PBlock 2 1.648 0.824 0.94Blend 4 11.556 2.889 3.31 0.071Error 8 6.992 0.874Total 14 20.196

S = 0.9349

Source DF SS MS F PBlend 4 11.556 2.889 3.34 0.055Error 10 8.640 0.864Total 14 20.196

S = 0.9295



Was the blocking effective?• F(Blocks) < 1• Blocks MS smaller than Error MS

• When blocks deleted from analysis– Residual standard deviation almost unchanged

and– F(Blends) almost unchanged

• Blocking NOT effective.



Block or not?





Block or not?

Omitting blocks

increases DF(Error),therefore

increases precision of estimate of s,and

increases power of F(Blends)

F4,8:0.10 = 2.8; F4,8:0.05 = 3.8

F4,10:0.10 = 2.6 F4,10:0.05 = 3.5

Smaller critical value easier to exceed, more power.Postgraduate Certificate in Statistics Design and Analysis of Experiments


Block or not?

Statistical theory suggests no blocking.

Practical knowledge may suggest otherwise.

Quote from Davies et al (1956):

"Although the apparent variation among the blocks is not confirmed (i.e. it might well be ascribed to experimental error), future experiments should still be carried out in the same way.

There is no clear evidence of a trend in this set of trials, but it might well appear in another set, and no complication in experimental arrangement is involved".



Diagnostic plots

• The diagnostic plot, residuals vs fitted values– checking the homogeneity of chance variation

• The Normal residual plot,– checking the Normality of chance variation



Diagnostic analysis


• One exceptional case– likely to be related to interaction pattern.

see Slide 55− resist deletion and refitting!


Initial data analysis

• Little variation between blocks• More variation between blends• Disturbing interaction pattern; see laterPostgraduate Certificate in Statistics Design and Analysis of Experiments


Analysis of Variance Explained








SS(TO) = SS(Block) + SS(Blend) + SS(Error)



Model for analysis

Yield loss includes

– a contribution from each blend

plus

– a contribution from each block

plus

– a contribution due to chance variation.



Model for analysis

Y = m + a + b + ewhere

m is the overall mean,a is the blend effect, above or below the mean,

depending on which blend is used,b is the block effect, above or below the mean,

depending on which block is involvede represents chance variation



Estimating the model




statistical residual format

mathematically simplified format

)]yy()yy()yy[()yy()yy(

yy

jiij

j

i

ij

)yyyy( jiij

SSTO = SS(Blocks) + SS(Blends) + SS(Error)

dataall

2jiij

dataall

2j

dataall

2i

dataall

2ij )yyyy()yy()yy()yy(




F(Blends) = tests equality of blend means

F(Blocks) = assesses effectiveness of blocking

1I

)(J= )EMS(Blends i

2i

2

mms

1J

)(I)Blocks(EMS j

2j

2

mm

s

2)Error(EMS s

)Error(MS)Blends(MS

)Error(MS)Blocks(MS



Minute test

– How much did you get out of today's class?– How did you find the pace of today's class?– What single point caused you the most

difficulty?– What single change by the lecturer would have

most improved this class?



Reading

EM Ch. 4, §7.2

Base Module, §2.2, §2.4, §§4.1- 4.3, §5.1

MGM §2.1, §§3.1,3,2

DCM §2-4.1 to §2-4.3, §2.5, §§3.1 to 3.4, §3.5.7, §4.1

DV §3.5, §§4.2.1-4.2.3, §4.3.2, §4.4.1, §§4.4.4-4.4.6, §10.3, §10.4 (with back references)


Lecture 2.11 © 2016 Michael Stuart Design and Analysis of Experiments Lecture 2.1 1.Review...

Documents

Transcript of Lecture 2.11 © 2016 Michael Stuart Design and Analysis of Experiments Lecture 2.1 1.Review...