Chapter 8 실험계획 및 분산분석(ANalysis Of VAariance, ANOVA)

Updated 2018/4/30

분산분석 (analysis of variance):전체변동을 몇 개의 성분으로 분할하는 기법(Divide total variation into several components)전체변동에 대해 각각의 변동요인의 기여 규모를파악 (contribution of particular components)

목적 (Aims):모분산의 추정과 가설 검정 (estimation & testing for the variances)모평균의 추정과 가설검정 (estimation & testing for the means)

7.1 머리말 (Introduction)

motivation

비교하고 싶은 그룹이 두 개이면 (comparisons of two groups) -> t-test

비교하고 싶은 그룹이 두 개 이상이면 (more than two groups) -> 두 개 그룹씩 뽑아서 쌍을 만든 후에여러 개의 t-test를 실시한다. (pairwise t-tests)

번거롭기도 하고 이론적으로 틀린 결론에 도달할 수있다. (cumbersome & theoretically wrong -> 다중비교의 문제 (multiple-comparisons problems)

전체 자료를 사용하지 않고 자료의 부분 만을 사용하므로 효율이 떨어진다. (efficiency problems due to the usage of partial data)

전체 자료를 이용하여서 세 그룹이상을 비교하는 분석(more than 3 groups using whole data) -> 분산 분석 ANOVA (종속변수는 연속형, 독립변수는 이산형) response var: conti, explanatory var: categorical

7.2 일원배치 분산분석(one-way analysis of variance)

하나의 설명변수 (one explanatory variable)

완전확률계획법(Completely Randomized Design)

정의 : 처리방법을 확률적으로 할당하고 그 처리 효과를 randomization)

처리변수 (treatment)

𝟏 𝟐 𝟑 ⋯ 𝒌

𝑥11 𝑥12 𝑥13 ⋯ 𝑥1𝑘

𝑥21 𝑥22 𝑥23 ⋯ 𝑥2𝑘

𝑥31 𝑥32 𝑥33 ⋯ 𝑥3𝑘

⋮ ⋮ ⋮ ⋮ ⋮

𝑥𝑛11 𝑥𝑛22 𝑥𝑛33 ⋯ 𝑥𝑛𝑘𝑘

합 (total) 𝑇.1 𝑇.2 𝑇.3 ⋯ 𝑇.𝑘 𝑇..

평균 (sum) 𝑥.1 𝑥.2 𝑥.3 ⋯ 𝑥.𝑘 𝑥..

완전확률화 계획법으로 관측한 표본

일원분산분석(one-way analysis of variance)

보기 8.2.1소의 연령에 따른 육류의 셀레니움 농도 비교

Comparison of selenium concentration of meat according to age of cattle

나이 그룹 (age group of cattle)A B C D

1820 1483 191 724 1020 1652 775 7522588 1723 1098 613 805 1309 1393 8042670 727 644 918 631 1002 533 11821022 1463 136 949 641 966 734 12431555 1777 1605 877 760 788 485 985222 1129 1247 1368 1085 472 449 1295

1197 1529 1692 775 471 236 1676

1249 1422 697 1307 771 831 754

1520 445 849 344 869 698 937

489 990 1199 961 513 167 1022

2575 489 429 239 731 824 1073

1426 2408 798 944 1130 448 948

1846 1064 631 1096 1034 991 222

1088 629 1016 1261 590 721

912 1025 42 994 375

1383 948 767 1781 1187

모형 (model)

1

:

ij j ijij

j ij

k

jj

k

j j

x

효과

번째측정치 j처리의 평균 ij번째 오차

: 전체평균

j번째 처리

ij-th observation mean of j-th treatment group error of ij-th observation

Grand mean

Effect of j-th treatment group

모형의 가정

1. 2. 평균, 등분산, 정규성, 독립적

(mean, variance (homogeneity), normality, independence)

모형의 가설 (Hypothesis of the model)

2~ (0, )

ijN independent

:0 1 2

:

(All the 's are not the same)

.

H k

H A j

j

모든 가 같은 것은 아니다

Same variances

& same means

Same variances but

different means

2

1 1

2

2

1 1

( )..

..

nk

ijj i

njk

ijj i

j

SST x x

Tx

N

• 총제곱합 (sum of squares, total)

2

..

1 1

2

. . ..

1 1

2 2

. . . .. . ..

1 1 1 1 1 1

2 2

. . ..

1 1 1

( )

( )

( ) 2 ( )( ) ( )

( ) ( )

j

j

j j j

j

nk

ij

j i

nk

ij j j

j i

n n nk k k

ij j ij j j j

j i j i j i

nk k

ij j j j

j i j

SST x x

x x x x

x x x x x x x x

x x n x x

within among groupSST SSW SSA

제곱 제곱집단내 합 집단간 합

MSAvariance ratio=

MSW

산분 비 = 집단간 평균제곱

집단내 평균제곱

->분산비가 커지면 집단간의 variation이 크다.집단간의 성질이 다르다. 집단의 효과가 크다.

Within-group SS Among(Between)-group SS

->larger VR -> larger between-group SS 0 ->groups are different -> bigger group effect !

유전율 예제 (Heritability Example)

요인 제곱합 자유도 평균제곱합 F

집단 간 제곱합𝑆𝑆𝐴 =

𝑗=1

𝑘

𝑛𝑗 𝑥.𝑗 − 𝑥..2

𝑘 − 1 𝑀𝑆𝐴 = 𝑆𝑆𝐴/(𝑘 − 1)𝑀𝑆𝐴

𝑀𝑆𝑊

집단 내 제곱합𝑆𝑆𝑊 =

𝑗=1

𝑘

𝑖=1

𝑛𝑗

𝑥𝑖𝑗 − 𝑥.𝑗2

𝑁 − 𝑘 𝑀𝑆𝑊 = 𝑆𝑆𝑊/(𝑁 − 𝑘)

총 제곱합𝑆𝑆𝑇 =

𝑗=1

𝑘

𝑖=1

𝑛𝑗

𝑥𝑖𝑗 − 𝑥..2

𝑁 − 1

ANOVA Table

factor

Within group

Between group

total

Sum of squares dfMean square Variance ratio

ANOVA Table

2 2 2 2

1

2

1(MSA) ,

1

(MSW)

k

A A j

j

E kk

E

The null hypothesis

indicates the equivalence of variancesestimated with MSA and MSW.

However, under the alternative test, varianceestimate from MSA is bigger than that fromMSW.

※ 오른쪽 검정(right-tailed test)

1 1( 0)k

program PROC IMPORT OUT= WORK.sele

DATAFILE= "E:\kim\yes\myweb\int\2018\newlectureNote\data\sel

e.csv"

DBMS=CSV REPLACE;

GETNAMES=YES;

DATAROW=2;

RUN;

* SAS 코드;

proc anova data=sele;

class group;

model value=group;

run;

# R code

> sele<-read.table('E:\\kim\\yes\\myweb\\int\\2018\\newlectureNote\\data\\sele.csv',sep=',',header=T)

aov(value~Group,data=sele)

> boxplot(value~Group,data=sele)

Betw

With

total

VR

Call:

aov(formula = value ~ Group, data = sele)

Terms:

Group Residuals

Sum of Squares 5931208 23026500

Deg. of Freedom 3 109

Residual standard error: 459.6219

Estimated effects may be unbalanced

>

Multiple Comparisons (다중비교)

ex) significance level = for a test

In general, if we want to test ,then

overall is 0.1855, not 0.05 -> inflated type I error !!

01 1 01 01

02 2 02 02

0 0 0 01 02

01

: 0 ( ) 1

: 0 ( ) 1

( ) where and

(

Let H p do not reject H H is true

H p do not reject H H is true

then p do not reject H H H H H

p do not reject H and do not reje

02 0

2

)

(1- ) (1- ) (1- )

ct H H

1 2 3 0k

4

(1 ) (1 )

1 0.1855 0.8145 ( .95) .95

k

Bonferroni Correction : Set individual significance

the overall significance level is about for m multiple tests.

m=4

example) When we have 10 hypotheses,

Individual p=0.05 -> multiple comparisons problem

(too many false findings)

Individual p=

This is often called “Bonferroni corrected p-value”.

m

40.05

1 0.95 1 0.054

0.050.005

10

[처리그룹 쌍별 두 모평균 차이의 검정]Detecting pairwise differences

After rejecting , which pairs have larger differences?

1. LSD (least significant difference, 최소 유의차 검정법)2. Duncan’s new multiple range test

Duncan의 새로운 다중범위 검정법

3. Tukey’s HSD

0 1 2 5:H

Liberal Conservative

DuncanLSD

SNK Tukey HSD

Scheffe

3. Tukey의 HSD (honestly significance difference) 검정

, ,

* *

, , *

's are the same

: sample size of smaller cell

k N k j

j

k N k j

j

MSEHSD q n

n

MSEHSD q n

n

=

max min, , : dist of ,

2 /

: significance level, : number of gropus, : df

k N k

y yq

S n

k N K

•보기 8.2.2

A B C D

A - 455.72 574.54 596.63

B - 118.82 140.91

C - 22.10

D -

Pair-wise differences

•표 8.2.6Pairwise comparisons by Tukey’s HSD test

개별 영가설 HSD* 검정결과

𝐻0: 𝜇𝐴 = 𝜇𝐵𝐻𝑆𝐷∗ = 3.690

211252

2

1

22+1

14= 409.99 455.72 > 409.99이므로 𝐻0을 기각함.

𝐻0: 𝜇𝐴 = 𝜇𝐶𝐻𝑆𝐷∗ = 3.690

211252

2

1

22+1

29= 339.05 574.54 > 339.05이므로 𝐻0을 기각함.

𝐻0: 𝜇𝐴 = 𝜇𝐷𝐻𝑆𝐷∗ = 3.690

211252

2

1

22+1

48= 308.75 596.63 > 308.75이므로 𝐻0을 기각함.

𝐻0: 𝜇𝐵 = 𝜇𝐶𝐻𝑆𝐷∗ = 3.690

211252

2

1

14+1

29= 390.27 118.82 < 390.27이므로 𝐻0을 기각하지 못함.

𝐻0: 𝜇𝐵 = 𝜇𝐷𝐻𝑆𝐷∗ = 3.690

211252

2

1

14+1

48= 364.25 140.91 < 364.25이므로 𝐻0을 기각하지 못함.

𝐻0: 𝜇𝐶 = 𝜇𝐷𝐻𝑆𝐷∗ = 3.690

211252

2

1

29+1

48= 282.05 22.10 < 282.05이므로 𝐻0을 기각하지 못함.

proc anova ;

class group ;

model value= group ;

means group /Tukey ;

run;

Homework

1-8

9->다음 문제들을 공식을 이용해서 분산분석표를 계산하시오 (엑셀 사용 가능). 그리고 SAS를 이용한 결과와 비교하시오

9-> Make Anova tables using the formulae (you mayuse MS Excel). Compare your results with the resultsfrom SAS

처리

블록 𝟏 𝟐 𝟑 ⋯ 𝒌 합 평균

1 𝑥11 𝑥12 𝑥13 ⋯ 𝑥1𝑘 𝑇1. 𝑥1.

2 𝑥21 𝑥22 𝑥23 ⋯ 𝑥2𝑘 𝑇2. 𝑥2.

3 𝑥31 𝑥32 𝑥33 ⋯ 𝑥3𝑘 𝑇3. 𝑥3.

⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮

𝑛 𝑥𝑛1 𝑥𝑛2 𝑥𝑛3 ⋯ 𝑥𝑛𝑘 𝑇𝑛. 𝑥𝑛.

합 𝑇.1 𝑇.2 𝑇.3 ⋯ 𝑇.𝑘 𝑇..

평균 𝑥.1 𝑥.2 𝑥.3 ⋯ 𝑥.𝑘 𝑥..

8.3 확률화 완전 블록 계획법과 이원배치 분산분석(Randomized complete block design

and two-way ANOVA)

R.A.Fisher (1925) : to compare the yields of certain species 땅을 블록(block=land)으로 나누고 블록 안에서Randomize (other factors) in a block 하는 것이다.

treatments

block

total

average

averagetotal

보기 8.3.1

약에 따른 치료시간의 차이 (treatment duration (days) by drug)

Drug

Age

group

약의 종류 Sum Average

나이 그룹 A B C 합 평균

20 미만 11 8 10 29 9.667

20 이상 29 미만 6 5 11 22 7.333

30 이상 39 미만 7 10 13 30 10

40 이상 49 미만 9 12 13 34 11.333

50 이상 10 17 15 42 14

합 (sum) 43 52 62 157

평균 (avegage) 8.6 10.4 12.4 10.467

2

0

(model)

( ) ~ (0, )

(hypothesis)

: 0 1,2, ,

:All 0 is not true. Some 0.

ij i j ij

ij ij i j

j

A j j

x

x N

H j k

H

블럭효과 처리효과

모형

가설

block effect trt effect

2

..

1 1

2 2 2

. .. . .. . . ..

1 1 1 1 1 1

( )

( ) ( ) ( )

: 1 ( 1) ( 1) ( 1)( 1)

j

k n

ij

j i

nk n k k n

i j ij i j

j i j i j i

SST x x

x x x x x x x x

SST SSBl SSTr SSE

df nk n k n k

*

ANOVA table

factor

trt

block

error

total

요인 제곱합 자유도 평균제곱 F

처리 𝑆𝑆𝑇𝑟 (𝑘 − 1) 𝑀𝑆𝑇𝑟 = 𝑆𝑆𝑇𝑟/(𝑘 − 1) 𝑀𝑆𝑇𝑟 𝑀𝑆𝐸

블록 𝑆𝑆𝐵𝑙 (𝑛 − 1) 𝑀𝑆𝐵𝑙 = 𝑆𝑆𝐵𝑙/(𝑛 − 1)

잔차 𝑆𝑆𝐸 (𝑛 − 1)(𝑘 − 1) 𝑀𝑆𝐸 = 𝑆𝑆𝐸/(𝑛 − 1)(𝑘 − 1)

합 𝑆𝑆𝑇 𝑘𝑛 − 1

Sum of

squares

Degree of

freedom

Mean

square

> response<-c(11, 6, 7, 9, 10, 8, 5, 10, 12, 17, 10, 11, 13, 13, 15)

> drug<-factor(rep(c('A','B','C'),each=5))

> age<-factor(rep(1:5),3) # 교과서 오류

> dat<-data.frame(response=response,drug=drug,age=age)

> anova(lm(response~drug+age,data=dat))

Analysis of Variance Table

Response: response

Df Sum Sq Mean Sq F value Pr(>F)

drug 2 36.133 18.0667 3.4522 0.08300 .

age 4 71.733 17.9333 3.4268 0.06505 .

Residuals 8 41.867 5.2333

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 >

data d;

do j=1 to 3;

do i=1 to 5;

drug=j ; output;

end;

end;

run;

data a;

do i=1 to 3;

do j=1 to 5;

age=j ; output;

end;

end;

run;

Data cc;

Input response @@;

Cards;

11 6 7 9 10 8 5 10 12 17 10 11 13 13 15

;run;

Data res;

Merge a(keep=age) d(keep=drug) cc;

run;

proc print data=res;run;

proc anova ;

class age drug ;

model response= drug age ;

run;

8.4 요인 실험 (Factorial Design)과이원배치 분산분석 (two-way ANOVA)

반응시간 (reduction of response time )= 약품수준(소량, 중간, 다량)*연령층(중년, 노년)drug level (min, med, max)*age(mid, old)

• 교호작용이 없을 때 (Without interaction)

요인B – 약품용량 (Factor-B, drug level)

요인A – 연령Factor A-age j=1 j=2 j=3

중년층(Mid) i=1 5 10 20

노년층(old) i=2 10 15 25

•교호작용이 있을 때 (With interaction)

요인B – 약품용량

요인A - 연령

j=1 j=2 j=3 j=2-1 j=3-2

중년층(i=1) 5 10 20 5 10

노년층(i=2) 15 10 5 -5 -5

2요인 완전확률화할당계획법(2 factors)

Factor A

Factor B

요인 B

요인 A 1 2 ⋯ 𝑏 합 평균1 𝑥111 𝑥121 ⋯ 𝑥1𝑏1

𝑇1.. 𝑥1..⋮𝑥11𝑛

⋮𝑥12𝑛

⋮⋯

⋮𝑥1𝑏𝑛

2 𝑥211 𝑥221 ⋯ 𝑥2𝑏1

𝑇2.. 𝑥2..⋮𝑥21𝑛

⋮𝑥22𝑛

⋮⋯

⋮𝑥2𝑏𝑛

⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮

𝑎 𝑥𝑎11 𝑥𝑎21 ⋯ 𝑥𝑎𝑏1

𝑇𝑎.. 𝑥𝑎..⋮𝑥𝑎1𝑛

⋮𝑥𝑎2𝑛

⋮⋯

⋮𝑥𝑎𝑏𝑛

합 𝑇.1. 𝑇.2. ⋯ 𝑇.𝑏. 𝑇...

평균 𝑥.1. 𝑥.2. ⋯ 𝑥.𝑏. 𝑥...

EX 8.4.2

간호사의 가정방문시간 (time of staying home for a nurse) =간호사의 연령 , 환자의 질환

(age of the nurse, disease of the patient)

( )

1, , 1, , 1, ,

(Model)

xijk i j ij ijk

i a j b k n

모형

0

0

0

0

0

0

: 0 1, ,

: Not 0 for some .

: 0 1, ,

: Not 0 for some .

:( ) 0 1, , 1, ,

:Not ( ) 0 for some , .

SST=SSA+SSB+SSAB+SSE

i

A i

j

A j

ij

A ij

H i a

H H i

H j b

H H j

H i a j b

H H i j

Hypotheses(가설)

> qf(0.95,3,64)

[1] 2.748191

> qf(0.95,9,64)

[1] 2.029792

> qf(0.95,15,64)

[1] 1.825586

> 1-pf(67.95,3,64)

[1] 0

> 1-pf(27.27,9,64)

[1] 0

> 1-pf(4.61,15,64)

[1] 7.473861e-06

PROC IMPORT OUT= WORK.nurse

DATAFILE= "E:\kim\yes\myweb\int\2018\newlectureNote\data\nurse.csv"

DBMS=CSV REPLACE;

GETNAMES=YES;

DATAROW=2;

RUN;

proc anova;

class a b;

model time= a b a*b ;

run;

miscellaneous (기타)

Log transformation: when normal assumption isviolated.

Normality is still problematic even after thevariable transformation. Sample size is too smallto check normality -> Nonparametric approach

e.g. income, concentration

One Way ANOVA

Type of Sum of Squares

* Type Ⅰ:sequential (if we know the relative importance of the variables)

Type Ⅱ: partial without interaction terms

**TypeⅢ:partial with interactions(If we don’t know the relative importance of the variables)

TypeⅣ: There are missing cells (if none, same as TypeⅢ)

* , ** : defaults

model i ijY A :

One Way ANOVA, mod12.sas

/* File : mod12.sas

To demonstrate one way ANOVA */

filename in 'd:\intro\taillite.dat';

data one;

infile in;

input id vehtype group position

speedzn resptime follotme

folltmec ;

if group = 1;

run;

proc sort ;by vehtype ;

proc means;

var resptime;

by vehtype ;

title 'Means of Response Time by

Vehicle Type';

run;

proc gplot ;

plot resptime*vehtype ;

symbol i=box;

title 'Box Plot Response Time by

Vehicle Type';

run;

proc anova;

class vehtype;

model resptime = vehtype ;

means vehtype /tukey lines bon cldiff

scheffe snk lsd ;

title 'One way Aonva for Tail Light

Study';

title2 ;

run;

Two Way ANOVA, mod13.sas

/* File : mod13.sas

To demonstrate Two way ANOVA */

filename stiff 'd:\intro\dummy.dat';

data one;

infile stiff;

input species $ impactor $ stiff1 stiff2 calcium magnesm ;

run;

proc gchart ;

block species / group=impactor sumvar=stiff1 type=mean ;

title 'Block Chart of Stiff1 by Impactor and Species';

run;

proc anova;

class species impactor;

model stiff1 = species impactor species*impactor ;

means species impactor / duncan lines ;

title 'Two way Aonva Dummy Data';

run;