Chap 6- Anova

8/3/2019 Chap 6- Anova

1/54

Chapter 6

Analysis of Variance


2/54

Chapter Overview

Analysis of Variance (ANOVA)

F-test

Tukey-Kramer

test

One-WayANOVA Two-WayANOVA

InteractionEffects

RandomizedBlock Design

MultipleComparisons


3/54

General ANOVA Setting

Investigator controls one or more independentvariables

factors (the characteristic which differentiates the

treatment or population from one another) Each factor contains two or more levels (the different

treatments or population)

Observe effects on the dependent variable

Response to levels of independent variable

Experimental design: the plan used to collectthe data


4/54

Completely Randomized Design

Experimental units (subjects) are assignedrandomly to treatments

Subjects are assumed homogeneous Only one factor or independent variable

With two or more treatment levels

Analyzed by one-way analysis of variance(ANOVA)


5/54

Examples

An experiment to study the effects of five(levels) different brands of gasoline (factors) onautomobile engine operating efficiency

An experiment to asses the effects of amounts(factors) of a particular drug on manual dexterity

Level: different setting of the factor


6/54

One-Way Analysis of Variance

Evaluate the difference among the means of threeor more groups

Examples: Accident rates for 1st, 2nd, and 3rd shift

Expected mileage for five brands of tires

Assumptions

Populations are normally distributed Populations have equal variances

Samples are randomly and independently drawn


7/54

Hypotheses of One-Way ANOVA

All population means are equal

i.e., no treatment effect (no variation in means among

groups)

At least one population mean is different

i.e., there is a treatment effect

Does not mean that all population means are different

(some pairs may be the same)

c3210 :H

samethearemeanspopulationtheofallNot:H1


8/54

Partitioning the Variation

Total variation can be split into two parts:

SST = total of the squared deviations ofscores from the overall mean

(Total variation)SSA = Sum of Squares Among Groups

(Among-group variation)SSW = Sum of Squares Within Groups

(Within-group variation)

SST = SSA + SSW


9/54


Total Variation = the aggregate dispersion of the individualdata values across the various factor levels (SST)

Within-Group Variation = dispersion that exists amongthe data values within a particular factor level (SSW)

Among-Group Variation = dispersion between the factorsample means (SSA)

SST = SSA + SSW

(continued)


10/54

Partition of Total Variation

Variation Due toFactor (SSA)

Variation Due to RandomSampling (SSW)

Total Variation (SST)

Commonly referred to as: Sum of Squares Within

Sum of Squares Error

Sum of Squares Unexplained

Within-Group Variation

Commonly referred to as: Sum of Squares Between

Sum of Squares Among

Sum of Squares Explained

Among Groups Variation

= +

d.f. = n 1

d.f. = c 1 d.f. = n c


11/54

Total Sum of Squares

c

1j

n

1i

2

ij

j

)XX(SST

Where:

SST = Total sum of squares

c = number of groups (levels or treatments)

nj = number of observations in group j

Xij = ith observation from group j

X = grand mean (mean of all data values)

SST = SSA + SSW


12/54

Total Variation

Group 1 Group 2 Group 3

Response, X

X

2cn

212

211 )XX(...)XX()XX(SST c

(continued)


13/54

Among-Group Variation

Where:

SSA = Sum of squares among groups

c = number of groups

nj = sample size from group j

Xj = sample mean from group j


2j

c

1j

j )XX(nSSA

SST = SSA + SSW


14/54


Variation Due toDifferences Among Groups

i j

2j

c

1j

j )XX(nSSA

1cSSAMSA

Mean Square Among =

SSA/degrees of freedom

(continued)


15/54



Response, X

X1

X 2X

3X

22

22

2

11)xx(n...)xx(n)xx(nSSA cc

(continued)


16/54


Where:

SSW = Sum of squares within groups

c = number of groupsnj = sample size from group j

Xj = sample mean from group j

Xij

= ith observation in group j

2jij

n

1i

c

1j

)XX(SSWj

SST = SSA + SSW


17/54


Summing the variationwithin each group and thenadding over all groups cn

SSWMSW

Mean Square Within =

SSW/degrees of freedom

2jij

n

1i

c

1j

)XX(SSWj

(continued)

j


18/54



Response, X

1X

2X

3X

2ccn

2212

2111 )XX(...)XX()Xx(SSW c

(continued)


19/54

Obtaining the Mean Squares

cn

SSWMSW

1c

SSAMSA

1n

SSTMST


20/54

One-Way ANOVA Table

Source ofVariation

dfSS MS(Variance)

AmongGroups SSA MSA =

WithinGroups

n - cSSW MSW =

Total n - 1SST =SSA+SSW

c - 1MSA

MSW

F ratio


n = sum of the sample sizes from all groups

df = degrees of freedom

SSA

c - 1

SSW

n - c

F =


21/54

One-Way ANOVAF Test Statistic

Test statistic

MSA is mean squares among groups

MSWis mean squares within groups

Degrees of freedom

df1 = c 1 (c = number of groups)

df2 = n c (n = sum of sample sizes from all populations)

MSWMSAF

H0: 1= 2= = c

H1: At least two population means are different


22/54

Interpreting One-Way ANOVAF Statistic

The F statistic is the ratio of the amongestimate of variance and the within estimateof variance

The ratio must always be positive df1 = c-1 will typically be small

df2 = n- c will typically be large

Decision Rule:

Reject H0 if F > FU,otherwise do notreject H0

0

= .05

Reject H0Do notreject H0

FU


23/54

One-Way ANOVAF Test Example

You want to see if threedifferent golf clubs yielddifferent distances. You

randomly select fivemeasurements from trials onan automated drivingmachine for each club. At the

0.05 significance level, isthere a difference in meandistance?

Club 1 Club 2 Club 3254 234 200263 218 222

241 235 197237 227 206251 216 204


24/54

One-Way ANOVA Example:Scatter Diagram

270

260

250

240

230

220

210200

190

Distance

1X

2X

3X

X

227.0x

205.8x226.0x249.2x 321

Club 1 Club 2 Club 3254 234 200263 218 222

241 235 197237 227 206251 216 204

Club1 2 3


25/54


26/54

F= 25.275

One-Way ANOVA ExampleSolution

H0: 1 = 2 = 3

H1: j not all equal

= 0.05

df1= 2 df2 = 12

Test Statistic:

Decision:

Conclusion:

Reject H0 at = 0.05

There is evidence thatat least one j differs

from the rest

0

= .05

FU

= 3.89Reject H0Do not

reject H0

25.275

93.3

2358.2

MSW

MSAF

CriticalValue:

FU

= 3.89


27/54

SUMMARY

Groups Count Sum Average Variance

Club 1 5 1246 249.2 108.2

Club 2 5 1130 226 77.5Club 3 5 1029 205.8 94.2

ANOVA

Source ofVariation

SS df MS F P-value F crit

BetweenGroups

4716.4 2 2358.2 25.275 4.99E-05 3.89

WithinGroups

1119.6 12 93.3

Total 5836.0 14

One-Way ANOVAExcel Output

EXCEL: tools | data analysis | ANOVA: single factor


28/54

The Tukey-Kramer Procedure

Tells which population means are significantlydifferent e.g.: 1 = 23

Done after rejection of equal means in ANOVA

Allows pair-wise comparisons

Compare absolute mean differences with criticalrange

x1 = 2 3


29/54

Tukey-Kramer Critical Range

where:

QU = Value from Studentized Range Distributionwith c and n - c degrees of freedom forthe desired level of

MSW = Mean Square Within

nj and nj= Sample sizes from groups j and j

j'j

U

n

1

n

1

2

MSWQRangeCritical


30/54

The Tukey-Kramer Procedure:Example

1. Compute absolute meandifferences:Club 1 Club 2 Club 3

254 234 200263 218 222241 235 197237 227 206251 216 204 20.2205.8226.0xx

43.4205.8249.2xx

23.2226.0249.2xx

32

31

21

2. Find the QU value from the table with

c = 3 and (n c) = (15 3) = 12 degrees of freedomfor the desired level of ( = 0.05 used here):

3.77QU


31/54

The Tukey-Kramer Procedure:Example

5. All of the absolute mean differencesare greater than critical range.Therefore there is a significant

difference between each pair ofmeans at 5% level of significance.Thus, with 95% confidence we can concludethat the mean distance for club 1 is greaterthan club 2 and 3, and club 2 is greater than

club 3.

16.2855

1

5

1

2

93.33.77

n

1

n

1

2

MSWQRangeCritical

j'j

U

3. Compute Critical Range:

20.2xx

43.4xx

23.2xx

32

31

21

4. Compare:

(continued)


32/54

The Randomized Block Design

Like One-Way ANOVA, we test for equalpopulation means (for different factor levels, forexample)...

...but we want to control for possible variationfrom a second factor (with two or more levels)

Levels of the secondary factor are called blocks


33/54


Total variation can now be split into three parts:

SST = Total variationSSA = Among-Group variationSSBL = Among-Block variationSSE = Random variation

SST = SSA + SSBL + SSE


34/54

Sum of Squares for Blocking

Where:


r = number of blocks

Xi. = mean of all values in block i


r

1i

2

i. )XX(cSSBL



35/54


Total variation can now be split into three parts:

SST and SSA are

computed as they werein One-Way ANOVA


SSE = SST (SSA + SSBL)


36/54

Mean Squares

1c

SSAgroupsamongsquareMeanMSA

1r

SSBLblockingsquareMeanMSBL

)1)(1(

cr

SSEMSE errorsquareMean


37/54

Randomized Block ANOVA Table

Source ofVariation

dfSS MS

AmongBlocks

SSBL MSBL

Error

(r1)(c-1)SSE MSE

Total rc - 1SST

r - 1 MSBL

MSE

F ratio

c = number of populations rc = sum of the sample sizes from all populations

r = number of blocks df = degrees of freedom

AmongTreatments SSA c - 1 MSA

MSA

MSE


38/54

Blocking Test

Blocking test: df1 = r 1

df2 = (r 1)(c 1)

MSBL

MSE

...:H 3.2.1.0

equalaremeansblockallNot:H1

F =

Reject H0 if F > FU


39/54

Main Factor test: df1 = c 1

df2 = (r 1)(c 1)

MSA

MSE

c..3.2.10...:H

equalaremeanspopulationallNot:H1

F =

Reject H0 if F > FU

Main Factor Test


40/54

The Tukey Procedure

To test which population means are significantlydifferent e.g.: 1 = 23 Done after rejection of equal means in randomized

block ANOVA design

Allows pair-wise comparisons Compare absolute mean differences with critical

range

x= 1 2 3


41/54

etc...

xx

xx

xx

.3.2

.3.1

.2.1

The Tukey Procedure(continued)

r

MSERangeCritical uQ

If the absolute mean differenceis greater than the critical rangethen there is a significantdifference between that pair ofmeans at the chosen level ofsignificance.

Compare:?RangeCriticalxxIs .j'.j


42/54

Factorial Design:Two-Way ANOVA

Examines the effect of

Two factors of interest on the dependentvariable

e.g., Percent carbonation and line speed on soft drinkbottling process

Interaction between the different levels of thesetwo factors

e.g., Does the effect of one particular carbonationlevel depend on which level the line speed is set?


43/54

Two-Way ANOVA

Assumptions

Populations are normally distributed

Populations have equal variances

Independent random samples are

drawn

(continued)

T W ANOVA


44/54

Two-Way ANOVASources of Variation

Two Factors of interest: A and B

r = number of levels of factor A

c = number of levels of factor B

n = number of replications for each cell

n = total number of observations in all cells

(n = rcn)

Xijk = value of the kth observation of level i of

factor A and level j of factor B

T W ANOVA


45/54

Two-Way ANOVASources of Variation

SSTTotal Variation

SSAFactor A Variation

SSBFactor B Variation

SSAB

Variation due to interactionbetween A and B

SSERandom variation (Error)

Degrees ofFreedom:

r 1

c 1

(r 1)(c 1)

rc(n 1)

n - 1

SST = SSA + SSB + SSAB + SSE

(continued)


46/54

Two Factor ANOVA Equations

r

i

c

j

n

k

ijk XXSST1 1 1

2)(

2r

1i

..i )XX(ncSSA

2c

1j

.j. )XX(nrSSB

Total Variation:

Factor A Variation:

Factor B Variation:


47/54


2

r

1i

c

1j

.j.i..ij. )XXXX(nSSAB

r

1i

c

1j

n

1k

2.ijijk )XX(SSE

Interaction Variation:

Sum of Squares Error:

(continued)


48/54


where:MeanGrand

nrc

X

X

r

1i

c

1j

n

1k

ijk

r)...,2,1,(iAfactorofleveliofMeannc

X

X th

c

1j

n

1k

ijk

..i

c)...,2,1,(jBfactorofleveljofMeannr

X

X

th

r

1i

n

1k

ijk

.j.

ijcellofMeann

XX

n

1k

ijk.ij

r = number of levels of factor A

c = number of levels of factor B

n = number of replications in each cell

(continued)


49/54

Mean Square Calculations

1r

SSAAfactorsquareMeanMSA

1c

SSBBfactorsquareMeanMSB

)1c)(1r(

SSABninteractiosquareMeanMSAB

)1'n(rc

SSEerrorsquareMeanMSE

T W ANOVA


50/54

Two-Way ANOVA:The F Test Statistic

F Test for Factor B Effect

F Test for Interaction Effect

H0: 1.. = 2.. = 3..=

H1: Not all i.. are equal

H0: the interaction of A and B isequal to zero

H1: interaction of A and B is notzero

F Test for Factor A Effect

H0: .1. = .2. = .3.=

H1: Not all .j. are equal

Reject H0

if F > FUMSE

MSAF

MSE

MSBF

MSE

MSABF

Reject H0

if F > FU

Reject H0

if F > FU

T W ANOVA


51/54

Two-Way ANOVASummary Table

Source ofVariation

Sum ofSquares

Degrees ofFreedom

MeanSquares

FStatistic

Factor A SSA r 1MSA

= SSA/(r 1)

MSA

MSE

Factor B SSB c 1MSB

= SSB/(c 1)

MSB

MSE

AB

(Interaction)SSAB (r 1)(c 1)

MSAB

= SSAB/ (r 1)(c 1)

MSAB

MSE

Error SSE rc(n 1)MSE =

SSE/rc(n 1)

Total SST n 1

F t f T W ANOVA


52/54

Features of Two-Way ANOVAFTest

Degrees of freedom always add up

n-1 = rc(n-1) + (r-1) + (c-1) + (r-1)(c-1)

Total = error + factor A + factor B + interaction

The denominator of the FTest is always the

same but the numerator is different

The sums of squares always add up

SST = SSE + SSA + SSB + SSAB

Total = error + factor A + factor B + interaction


53/54

Example

In a study on automobile traffic and air polution , air sample taken at fourdifferent times and at five different location were analyzed to obtain the

amount of particulate matter present in the air (mg/m3).

Is there any difference in a true average amount of particulate matter

present in the air due to either different sampling times or different location

E l


54/54

Examples:Interaction vs. No Interaction

No interaction:

Factor B Level 1

Factor B Level 3

Factor B Level 2

Factor A Levels

Factor B Level 1

Factor B Level 3

Factor B Level 2

Factor A Levels

MeanResponse

Me

anResponse

Interaction ispresent:

Chap 6- Anova

Documents

Transcript of Chap 6- Anova