Introduction to the Gauss-Markov Linear Modeldnett/S511/01Introduction.pdf · The Gauss-Markov...

Introduction to the Gauss-Markov LinearModel

Copyright c©2012 Dan Nettleton (Iowa State University) Statistics 511 1 / 36

Random Vectors

y =

y1y2...

yn

is a random vector if and only if each element of y is a

random variable (i.e., yi is a random variable ∀ i = 1, . . . , n).

The mean of the random vector y is E(y) =

E(y1)E(y2)

...E(yn)

.

The variance of the random vector y is the matrix whose i, jthelement is Cov(yi, yj) = E(yiyj)− E(yi)E(yj).


Example: Variance of a Random Vector

For example, the variance of y =

y1

y2

y3

is

Var(y) =

Cov(y1, y1) Cov(y1, y2) Cov(y1, y3)Cov(y2, y1) Cov(y2, y2) Cov(y2, y3)Cov(y3, y1) Cov(y3, y2) Cov(y3, y3)

=

Var(y1) Cov(y1, y2) Cov(y1, y3)Cov(y2, y1) Var(y2) Cov(y2, y3)Cov(y3, y1) Cov(y3, y2) Var(y3)

.


The Gauss-Markov Linear Model

y = Xβ + ε

y is an n× 1 random vector of responses.

X is an n× p matrix of constants with columns corresponding toexplanatory variables. X is sometimes referred to as the designmatrix.

β is an unknown parameter vector in IRp.

ε is an n× 1 random vector of errors.

E(ε) = 0 and Var(ε) = σ2I, where σ2 is an unknown parameter inIR+.


The Gauss-Markov Linear Model

Note that the model is not completely specified because thedistribution of y is not completely specified.

y = Xβ + ε, E(ε) = 0, Var(ε) = σ2I

=⇒ E(y) = Xβ, Var(y) = σ2I=⇒ y ∼ (Xβ, σ2I)

“y has a distribution with mean Xβ and variance σ2I.”


The Normal Theory Gauss-Markov Linear Model

We often add an assumption of multivariate normality to theGauss-Markov linear model: ε ∼ N(0, σ2I).

The assumption ε ∼ N(0, σ2I) is equivalent toε1, . . . , εn

i.i.d.∼ N(0, σ2).

The assumption ε ∼ N(0, σ2I) =⇒ y ∼ N(Xβ, σ2I), i.e.,y1, . . . , yn are independent normal random variables,

Var(yi) = σ2 ∀ i = 1, . . . , n, and

E(yi) = x′(i)β (where x′(i) is the ith row of X) ∀ i = 1, . . . , n.


Goal of Analysis

y = Xβ + ε

The goal of analysis often focuses on answering questionsabout certain linear functions of β of the form Cβ for aspecified matrix C.

The normality assumption is useful for constructingconfidence intervals and performing tests concerning Cβ.


Example 1Researchers harvested five randomly selected ears of corn from afield. For i = 1, . . . , 5; let yi denote the weight in grams of the ith ear.

y1, . . . , y5i.i.d.∼ N(µ, σ2)

yi = µ+ εi, i = 1, . . . , 5; ε1, . . . , ε5i.i.d.∼ N(0, σ2)

y1 = µ+ ε1

y2 = µ+ ε2

y3 = µ+ ε3 ε1, . . . , ε5i.i.d.∼ N(0, σ2)

y4 = µ+ ε4

y5 = µ+ ε5


Example 1 (continued)

y1 = µ+ ε1

y2 = µ+ ε2

y3 = µ+ ε3 ε1, . . . , ε5i.i.d.∼ N(0, σ2)

y4 = µ+ ε4

y5 = µ+ ε5

y1y2y3y4y5

=

µµµµµ

+

ε1ε2ε3ε4ε5

,ε1ε2ε3ε4ε5

∼ N(0, σ2I)



y1y2y3y4y5

=

µµµµµ

+

ε1ε2ε3ε4ε5

,ε1ε2ε3ε4ε5

∼ N(0, σ2I)

y1y2y3y4y5

=

11111

[µ] +

ε1ε2ε3ε4ε5

,ε1ε2ε3ε4ε5

∼ N(0, σ2I)



y1y2y3y4y5

=

11111

[µ] +

ε1ε2ε3ε4ε5

,ε1ε2ε3ε4ε5

∼ N(0, σ2I)

y = Xβ + ε, ε ∼ N(0, σ2I)

Cβ = [1][µ] = µ


Example 2

Researchers randomly assigned eight experimental units to twotreatments and measured a response of interest. For i = 1, 2; letyi1, yi2, yi3, yi4 denote the responses of the experimental units in the ith

treatment group.

y11, y12, y13, y14i.i.d.∼ N(µ1, σ

2)

independent of

y21, y22, y23, y24i.i.d.∼ N(µ2, σ

2)

yij = µi + εij, i = 1, 2; j = 1, . . . , 4

ε11, ε12, ε13, ε14, ε21, ε22, ε23, ε24i.i.d.∼ N(0, σ2)



y11 = µ1 + ε11

y12 = µ1 + ε12

y13 = µ1 + ε13

y14 = µ1 + ε14

y21 = µ2 + ε21

y22 = µ2 + ε22

y23 = µ2 + ε23

y24 = µ2 + ε24

ε11, ε12, ε13, ε14, ε21, ε22, ε23, ε24i.i.d.∼ N(0, σ2)



y11y12y13y14y21y22y23y24

=

µ1µ1µ1µ1µ2µ2µ2µ2

+

ε11ε12ε13ε14ε21ε22ε23ε24

,

ε11ε12ε13ε14ε21ε22ε23ε24

∼ N(0, σ2I)



y11y12y13y14y21y22y23y24

=

1 01 01 01 00 10 10 10 1

[µ1µ2

]+

ε11ε12ε13ε14ε21ε22ε23ε24

,

ε11ε12ε13ε14ε21ε22ε23ε24

∼ N(0, σ2I)



y11y12y13y14y21y22y23y24

=

1 01 01 01 00 10 10 10 1

[µ1µ2

]+

ε11ε12ε13ε14ε21ε22ε23ε24

,

ε11ε12ε13ε14ε21ε22ε23ε24

∼ N(0, σ2I)

y = Xβ + ε, ε ∼ N(0, σ2I)

Cβ = [1,−1][µ1µ2

]= µ1 − µ2


Example 3Suppose eight fertilizer amounts denoted x1, . . . , x8 were randomlyassigned to eight field plots. For i = 1, . . . , 8; let yi denote the yield ofthe plot that received fertilizer amount xi.

yi = β0 + β1xi + εi, i = 1, . . . , 8

ε1, . . . , ε8i.i.d.∼ N(0, σ2)

y1 = β0 + β1x1 + ε1

y2 = β0 + β1x2 + ε2

y3 = β0 + β1x3 + ε3

y4 = β0 + β1x4 + ε4

y5 = β0 + β1x5 + ε5

y6 = β0 + β1x6 + ε6

y7 = β0 + β1x7 + ε7

y8 = β0 + β1x8 + ε8Copyright c©2012 Dan Nettleton (Iowa State University) Statistics 511 17 / 36


y1y2y3y4y5y6y7y8

=

β0 + β1x1β0 + β1x2β0 + β1x3β0 + β1x4β0 + β1x5β0 + β1x6β0 + β1x7β0 + β1x8

+

ε1ε2ε3ε4ε5ε6ε7ε8

,


∼ N(0, σ2I)



y1y2y3y4y5y6y7y8

=

1 x11 x21 x31 x41 x51 x61 x71 x8

[β0β1

]+


,


∼ N(0, σ2I)



y1y2y3y4y5y6y7y8

=

1 x11 x21 x31 x41 x51 x61 x71 x8

[β0β1

]+


,


∼ N(0, σ2I)

y = Xβ + ε, ε ∼ N(0, σ2I)

Cβ = [0, 1][β0β1

]= β1


Example 4

Eight hogs were randomly assigned to two diets and two inoculationssuch that two hogs received each combination of diet and inoculation.

This experiment involves two factors: diet and inoculation.In this case, each factor has two levels (denoted here genericallyas 1 and 2).A combination of one level from each factor forms a treatment.In this case, we have four treatments:

Treatment Diet Inoculation1 1 12 1 23 2 14 2 2



For i = 1, 2; j = 1, 2; and k = 1, 2; let yijk denote the average daily gainof the kth hog that received diet i and inoculation j.

yijk = µ+ εijk i = 1, 2; j = 1, 2; k = 1, 2;

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)

Under this model, neither diet nor inoculation affects average dailygain.




yijk = µ+ αi + εijk i = 1, 2; j = 1, 2; k = 1, 2;

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)

Under this model, only diet affects average daily gain.




yijk = µ+ βj + εijk i = 1, 2; j = 1, 2; k = 1, 2;

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)

Under this model, only inoculation affects average daily gain.



yijk = µ+ αi + βj + εijk i = 1, 2; j = 1, 2; k = 1, 2;

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)

Under this model, factors diet and inoculation affect the meanaverage daily gain in an additive manner.There is no interaction between the factors diet and inoculation.

inoculationdiet 1 2 inoculation difference1 µ+ α1 + β1 µ+ α1 + β2 β1 − β22 µ+ α2 + β1 µ+ α2 + β2 β1 − β2

diet difference α1 − α2 α1 − α2



yijk = µ+ αi + βj + γij + εijk i = 1, 2; j = 1, 2; k = 1, 2;

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)

Under this model, there is one mean for each combination of dietand inoculation.Those four means are free to take any four values with norestrictions.

inoculationdiet 1 2 ∆inoculation1 µ+ α1 + β1 + γ11 µ+ α1 + β2 + γ12 β1 − β2 + γ11 − γ122 µ+ α2 + β1 + γ21 µ+ α2 + β2 + γ22 β1 − β2 + γ21 − γ22

∆diet α1 − α2 + γ11 − γ21 α1 − α2 + γ12 − γ22



An equivalent model is the so called cell means model:

yijk = µij + εijk i = 1, 2; j = 1, 2; k = 1, 2;

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)

inoculationdiet 1 2 ∆inoculation1 µ11 µ12 µ11 − µ122 µ21 µ22 µ21 − µ22

∆diet µ11 − µ21 µ12 − µ22



yijk = µ+ αi + βj + γij + εijk i = 1, 2; j = 1, 2; k = 1, 2;

y111 = µ+ α1 + β1 + γ11 + ε111

y112 = µ+ α1 + β1 + γ11 + ε112

y121 = µ+ α1 + β2 + γ12 + ε121

y122 = µ+ α1 + β2 + γ12 + ε122

y211 = µ+ α2 + β1 + γ21 + ε211

y212 = µ+ α2 + β1 + γ21 + ε212

y221 = µ+ α2 + β2 + γ22 + ε221

y222 = µ+ α2 + β2 + γ22 + ε222

ε111, ε112, ε121, ε122, ε211, ε212, ε221, ε222i.i.d.∼ N(0, σ2)



y111y112y121y122y211y212y221y222

=

µ+ α1 + β1 + γ11µ+ α1 + β1 + γ11µ+ α1 + β2 + γ12µ+ α1 + β2 + γ12µ+ α2 + β1 + γ21µ+ α2 + β1 + γ21µ+ α2 + β2 + γ22µ+ α2 + β2 + γ22

+

ε111ε112ε121ε122ε211ε212ε221ε222

,

ε111ε112ε121ε122ε211ε212ε221ε222

∼ N(0, σ2I)



y111y112y121y122y211y212y221y222

=

1 1 0 1 0 1 0 0 01 1 0 1 0 1 0 0 01 1 0 0 1 0 1 0 01 1 0 0 1 0 1 0 01 0 1 1 0 0 0 1 01 0 1 1 0 0 0 1 01 0 1 0 1 0 0 0 11 0 1 0 1 0 0 0 1

µα1α2β1β2γ11γ12γ21γ22

+

ε111ε112ε121ε122ε211ε212ε221ε222

y = Xβ + ε, ε ∼ N(0, σ2I)



β = [µ, α1, α2, β1, β2, γ11, γ12, γ21, γ22]′

inoculationdiet 1 21 µ+ α1 + β1 + γ11 µ+ α1 + β2 + γ122 µ+ α2 + β1 + γ21 µ+ α2 + β2 + γ22

∆diet α1 − α2 + γ11 − γ21 α1 − α2 + γ12 − γ22

Is the difference between diet means for inoculation 1 the same as thedifference between diet means for inoculation 2?

Cβ = [0, 0, 0, 0, 0, 1,−1,−1, 1]β = γ11 − γ12 − γ21 + γ22 = 0?

This questions asks if there is interaction between the factors diet andinoculation.



β = [µ, α1, α2, β1, β2, γ11, γ12, γ21, γ22]′


Is the difference between inoculation means for diet 1 the same as thedifference between inoculation means for diet 2?

Cβ = [0, 0, 0, 0, 0, 1,−1,−1, 1]β = γ11 − γ12 − γ21 + γ22 = 0?

This questions also asks if there is interaction between the factors dietand inoculation.



β = [µ, α1, α2, β1, β2, γ11, γ12, γ21, γ22]′

inoculationdiet 1 2 Diet Means1 µ+ α1 + β1 + γ11 µ+ α1 + β2 + γ12 µ+ α1 + β· + γ1·2 µ+ α2 + β1 + γ21 µ+ α2 + β2 + γ22 µ+ α2 + β· + γ2·

Is the average over inoculation means for diet 1 different than theaverage over inoculation means for diet 2?

Cβ = [0, 1,−1, 0, 0, .5, .5,−.5,−.5]β = α1 − α2 + γ1· − γ2· = 0?

This question asks about the main effect of the factor diet.



β = [µ, α1, α2, β1, β2, γ11, γ12, γ21, γ22]′


Inoculation Means µ+ α· + β1 + γ·1 µ+ α· + β2 + γ·2

Is the average over diet means for inoculation 1 different than theaverage over diet means for inoculation 2?

Cβ = [0, 0, 0, 1,−1, .5,−.5, .5,−.5]β = β1 − β2 + γ·1 − γ·2 = 0?

This question asks about the main effect of the factor inoculation.



β = [µ, α1, α2, β1, β2, γ11, γ12, γ21, γ22]′


∆diet α1 − α2 + γ11 − γ21

Is there a difference between the diet means for inoculation 1?

Cβ = [0, 1,−1, 0, 0, 1, 0,−1, 0]β = α1 − α2 + γ11 − γ21 = 0?

This question asks about the simple effect of the factor diet for the firstlevel of the factor inoculation.



β = [µ, α1, α2, β1, β2, γ11, γ12, γ21, γ22]′


∆diet α1 − α2 + γ11 − γ21 α1 − α2 + γ12 − γ22

Are all four treatment means identical?

Cβ =

0 0 0 1 −1 1 −1 0 00 0 0 1 −1 0 0 1 −10 1 −1 0 0 1 0 −1 0

β

=

β1 − β2 + γ11 − γ12β1 − β2 + γ21 − γ22α1 − α2 + γ11 − γ21

=

000

?


Introduction to the Gauss-Markov Linear Modeldnett/S511/01Introduction.pdf · The Gauss-Markov...

Documents

Transcript of Introduction to the Gauss-Markov Linear Modeldnett/S511/01Introduction.pdf · The Gauss-Markov...