Mixed Models and Hierarchical Models - A Short Overview

OutlineThe Mixed Effects Model Through Examples

The Mathematical ModelGeneralized Linear Mixed Models

Mixed Models and Hierarchical Models - AShort Overview

Dipayan Maiti

LISA Short CourseLaboratory for Interdisciplinary Statistical Analysis

Department of Statistics

Virginia Tech

http://www.lisa.stat.vt.edu

July 8, 2009

Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview

http://www.lisa.stat.vt.edu



Outline

The Mixed Effects Model Through ExamplesThe High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data

The Mathematical ModelThe RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model

Generalized Linear Mixed ModelsThe Respiratory Illness ExampleA Bayesian Hierarchical Model




The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data

Outline








The Data

Data is from a 1982 survey of 7185 High School students from 160 Schools.

The data has some student specific attributes and some school specific attributes.

You can visualize the schools as groups and within each group (school) there are students.





Figure: High School data





Description of Variables

I school: school ID for student´s school

I ses: socioeconomic status of student´s family

I mathach: students score on a math-achievement test

I meanses: mean SES score for each school

I cses: socioeconomic status of student´s family centered by mean SES score for his school

I sector: public or catholic school





Figure: Plot of Math Achievement Score by Socio Economic Status forCatholic Schools





Figure: Plot of Math Achievement Score by Socio Economic Status forPublic Schools





Figure: Plot of Intercepts and Slopes for Catholic SchoolsDipayan Maiti Mixed Models and Hierarchical Models - A Short Overview




Figure: Plot of Intercepts and Slopes for Public SchoolsDipayan Maiti Mixed Models and Hierarchical Models - A Short Overview




Figure: Box Plot Comparisons of Intercepts and Slopes for Catholic andPublic Schools





What does this tell us about the model?

I If we model the math score (mathach) by centered socioeconomic status (cses) for students of a specificschool the intercept is on the average 14 for Catholic Schools and 12 for Public Schools

I The slope is on the average 1.5 for Catholic Schools and just about less than 3 for Public Schools

I The slope signifies the rate of change of math score (mathach) with respect to the centered socioeconomicstatus (cses)

I However both the intercept and the slope seem to have a random distribution around these expected values





How do we build a model that takes into consideration all the salient features of the data that we have been ableto obtain in this exploratory study?

Can we build a hierarchical model that makes the intercept and slope random?

Maybe the slope and intercept for each school can be modeled based on the mean socioeconomic status for theschool and if the school is Catholic or Public.

This slope and intercept can be used for that specific school to model math score of the students based on theirindividual centered socioeconomic status (cses).





A Possible Hierarchical ModelFor student j in school i

mathachij = β0 i + β1 i Xcses ij + εij (1)

β0 i = α0 i + α1 i Xsector i + α2 i Xmeanses i + εβ0 i(2)

β1 i = γ0 i + γ1 i Xsector i + γ2 i Xmeanses i + εβ1 i(3)





What does the complete model look like?

If we plug the values of β0 i and β1 i in the first model we obtain:

mathachij = β0 + β1Xcses ij + β2Xsector i + β4Xmeanses i +

β5Xmeanses i ∗ Xsector i + β6Xmeanses i ∗ Xcses ij +

γ1 + γ2Xcses ij + εij





Let us code Xsector such that 0 is Public and 1 Catholic.For students in Public School i , the equation becomes,

mathachij = (β0 + γ1) + (β1 + γ2)Xcses ij + β4Xmeanses i +

β6Xmeanses i ∗ Xcses ij + εij

For students in Catholic School i , the equation becomes,

mathachij = (β0 + β2 + γ1) + (β1 + γ2)Xcses ij + (β4 + β5)Xmeanses i

+β6Xmeanses i ∗ Xcses ij + εij





R-Output: Random Effects Covariance Matrix

> mixedmodel.lme.1 <- lme(mathach ~ meanses + cses + sector + meanses*cses +

sector*cses, random = ~ cses | school, data= mixedmodel)

> summary(mixedmodel.lme.1)

Linear mixed-effects model fit by REML

Data: mixedmodel

AIC BIC logLik

46523.66 46592.45 -23251.83

Random effects:

Formula: ~cses | school

Structure: General positive-definite, Log-Cholesky parametrization

StdDev Corr

(Intercept) 1.5426480 (Intr)

cses 0.3178422 0.391

Residual 6.0598009





R-Output: Fixed Effects Estimates

Fixed effects: mathach ~ meanses + cses + sector + meanses * cses + sector * cses

Value Std.Error DF t-value p-value

(Intercept) 12.127931 0.1992951 7022 60.85415 0e+00

meanses 5.332878 0.3691743 157 14.44542 0e+00

cses 2.945043 0.1555906 7022 18.92815 0e+00

sectorCatholic 1.226578 0.3062783 157 4.00478 1e-04

meanses:cses 1.039249 0.2988794 7022 3.47715 5e-04

cses:sectorCatholic -1.642680 0.2397649 7022 -6.85121 0e+00





R-Output: Fixed Effect Correlations & Within GroupVariance Estimates

(Intr) meanss cses sctrCt mnss:c

meanses 0.256

cses 0.075 0.019

sectorCatholic -0.699 -0.356 -0.053

meanses:cses 0.019 0.074 0.293 -0.026

cses:sectorCatholic -0.052 -0.027 -0.696 0.077 -0.351

Standardized Within-Group Residuals:

Min Q1 Med Q3 Max

-3.15926307 -0.72319713 0.01704114 0.75443404 2.95823100

Number of Observations: 7185

Number of Groups: 160





R-Output: Comparing Mixed Effects Models

> mixedmodel.lme.2 <- lme(mathach ~ meanses + cses + sector +

meanses*cses + sector*cses, random = ~ 1| school, data= mixedmodel)

> summary(mixedmodel.lme.2) # omitting random effect of cses

-------------------------------------------------------------------------------------------------------


AIC BIC logLik

46520.79 46575.82 -23252.39

Random effects:

Formula: ~1 | school

(Intercept) Residual

StdDev: 1.541213 6.063503

> mixedmodel.lme.3 <- lme(mathach ~ meanses + cses + sector +

meanses*cses + sector*cses, random = ~ cses -1 | school, data= mixedmodel)

> summary(mixedmodel.lme.3) # omitting random intercept

-----------------------------------------------------------------------------------------------------


AIC BIC logLik

46740.23 46795.26 -23362.11

Random effects:

Formula: ~cses - 1 | school

cses Residual

StdDev: 0.006713239 6.247727





Another Example: A scientist tracks the growth of pigs for nine consecutive weeks. He wants to estimate thetypical growth curve of the population of pigs.

Figure: Pig Growth over nine weeksDipayan Maiti Mixed Models and Hierarchical Models - A Short Overview




I Although the growth rate is linear the starting weight of the pigs differ considerably

I If the starting weight was higher it tends to remain higher and maybe even more so at the end of the nineweeks

I The growth rate probably should be explained by not only pig-specific starting weight but also pig-specificgrowth rates.





The growth model of the pigs can be explained using the following model. yit is the weight of the i th pig at the tth

week.

yit = α + ai + (β + bi )t + εit

i = 1, . . . 48 t = 1, . . . 9

I α = mean weight of pigs at week zero

I ai ∼ N(0, σ2a ) = random deviation due to i th pig from mean weight at week zero

I β = mean growth rate across all pigs

I bi ∼ N(0, σ2b) = random deviation due to i th pig from mean growth rate

I εit ∼ N(0, σ2) = random residual error

I ai and bi are uncorrelated with the random error term εit

I However there might be some correlation between ai and bi = Cov(ai , bi )





According to the above model the following variance and covariance formula can be derived

Var(yit , yit ) = σ2a + t2

σ2b + 2tCov(ai , bi ) + σ

2

Cov(yi1, yit ) = σ2a + tσ2

b + (t + 1)Cov(ai , bi ) t = 2, . . . , 9





Mixed Model Analysis of the Pig data

library(lme4)

library(lattice)

pigweight <- read.table("pigweight.txt", header = TRUE)

week1 week2 week3 week4 week5 week6 week7 week8 week9

1 24.0 32.0 39.0 42.5 48.0 54.5 61.0 65.0 72.0

2 22.5 30.5 40.5 45.0 51.0 58.5 64.0 72.0 78.0

3 22.5 28.0 36.5 41.0 47.5 55.0 61.0 68.0 76.0

4 24.0 31.5 39.5 44.5 51.0 56.0 59.5 64.0 67.0

5 24.5 31.5 37.0 42.5 48.0 54.0 58.0 63.0 65.5

6 23.0 30.0 35.5 41.0 48.0 51.5 56.5 63.5 69.5

7 22.5 28.5 36.0 43.5 47.0 53.5 59.5 67.5 73.5

8 23.5 30.5 38.0 41.0 48.5 55.0 59.5 66.5 73.0

9 20.0 27.5 33.0 39.0 43.5 49.0 54.5 59.5 65.0

10 25.5 32.5 39.5 47.0 53.0 58.5 63.0 69.5 76.0

11 24.5 31.0 40.5 46.0 51.5 57.0 62.5 69.5 76.0

12 24.0 29.0 39.0 44.0 50.5 57.0 61.5 68.0 73.5

pigweightL <- reshape(pigweight, direction = "long",varying = list(colnames(pigweight)),

v.name = "weight",timevar = "week", idvar = "pigid")

pigweightL <- with(pigweightL, pigweightL[order(pigid,week),])

mixedmodel.pig <- lmer(weight ~ 1+week+(1+week|pigid),data=piweightL)

print(xyplot(weight ~ week, groups = pigid, data = pigweightL,type = "l", col = 1))






pigweightL <- reshape(pigweight, direction = "long",varying = list(colnames(pigweight)),

v.name = "weight",timevar = "week", idvar = "pigid")

week weight pigid

1.1 1 24.0 1

1.2 2 32.0 1

1.3 3 39.0 1

1.4 4 42.5 1

1.5 5 48.0 1

1.6 6 54.5 1

1.7 7 61.0 1

1.8 8 65.0 1

1.9 9 72.0 1

2.1 1 22.5 2

2.2 2 30.5 2

2.3 3 40.5 2

2.4 4 45.0 2

pigweightL <- with(pigweightL, pigweightL[order(pigid,week),])

mixedmodel.pig <- lmer(weight ~ 1+week+(1+week|pigid),data=piweightL)

print(xyplot(weight ~ week, groups = pigid, data = pigweightL,type = "l", col = 1))






Linear mixed model fit by REML

Formula: weight ~ 1 + week + (1 + week | pigid)

Data: pigweightL

AIC BIC logLik deviance REMLdev

1753 1777 -870.4 1738 1741

Random effects:

Groups Name Variance Std.Dev. Corr

pigid (Intercept) 6.9865 2.64319

week 0.3800 0.61644 -0.064

Residual 1.5968 1.26366

Number of obs: 432, groups: pigid, 48

Fixed effects:

Estimate Std. Error t value

(Intercept) 19.35561 0.40387 47.93

week 6.20990 0.09204 67.47

Correlation of Fixed Effects:

(Intr)

week -0.133





The growth of rats is tracked for four weeks (with a baseline measure at week zero) under the influence of twosubstances thiouracil and thyroxinil. The growth measurements are also obtained for control rats for the same timeperiod.

Figure: Rat Growth over four weeks for control, thiouracil and thyroxinilgroups





Mixed Model Analysis of the Rat data

ratsbodyweight <- read.table("ratsbodyweight.txt", header = TRUE)

ratid treat week0 week1 week2 week3 week4

1 1 control 57 86 114 139 172

2 2 control 60 93 123 146 177

3 3 control 52 77 111 144 185

4 4 control 49 67 100 129 164

5 5 control 56 81 104 121 151

6 6 control 46 70 102 131 153

7 7 control 51 71 94 110 141

8 8 control 63 91 112 130 154

9 9 control 49 67 90 112 140

10 10 control 57 82 110 139 169

11 11 thiouracil 61 86 109 120 129

12 12 thiouracil 59 80 101 111 122

13 13 thiouracil 53 79 100 106 133

14 14 thiouracil 59 88 100 111 122






rat <- reshape(ratsbodyweight,direction="long",varying=list(paste("week",c(0:4),sep="")),

v.name="weight",idvar="ratid",timevar="week",time=c(0:4))

rat <- transform(rat, treat = factor(treat))

ratid treat week weight

1.0 1 control 0 57

2.0 2 control 0 60

3.0 3 control 0 52

4.0 4 control 0 49

5.0 5 control 0 56

6.0 6 control 0 46

7.0 7 control 0 51

8.0 8 control 0 63

9.0 9 control 0 49

10.0 10 control 0 57

11.0 11 thiouracil 0 61




library(lattice)

print(xyplot(weight ~ week | treat, groups = ratid,data=rat,type="l"))





The growth model of the rats can be explained using the following model. yit is the weight of the i th rat at the tth

week.

yit = α + ai + γtreat(i) + (β + bi + δtreat(i))t + εit

i = 1, . . . 48 t = 1, . . . 9

I α = mean weight of rats at week zero

I ai ∼ N(0, σ2a ) = random deviation due to i th rat from mean weight at week zero

I β = mean growth rate across all rats

I bi ∼ N(0, σ2b) = random deviation due to i th rat from mean growth rate

I γtreat(i) = effect of the treatment factor (control/thiouracil/thyroxinil) that has been applied to the ith rat

I δtreat(i) = interaction effect between the treatment factor (control/thiouracil/thyroxinil) that has been

applied to the ith rat and time t

I εit ∼ N(0, σ2) = random residual error

I ai and bi are uncorrelated with the random error term εit

I However there might be some correlation between ai and bi = Cov(ai , bi )






mixedmodel.rat <- lmer(weight ~ treat + week + treat:week + (1 + week | ratid), data=rat)


Formula: weight ~ treat + week + treat:week + (1 + week | ratid)

Data: rat


898 927 -439 895.2 878

Random effects:


ratid (Intercept) 32.404 5.6925

week 13.963 3.7367 -0.127

Residual 18.806 4.3366

Number of obs: 135, groups: ratid, 27






Fixed effects:


(Intercept) 52.8800 2.0902 25.299

treatthiouracil 4.8200 2.9559 1.631

treatthyroxin -0.7943 3.2573 -0.244

week 26.4800 1.2587 21.037

treatthiouracil:week -9.4300 1.7801 -5.297

treatthyroxin:week 0.6629 1.9616 0.338


(Intr) trtthr trtthy week trtthr:

treatthircl -0.707

treatthyrxn -0.642 0.454

week -0.245 0.173 0.157

trtthrcl:wk 0.173 -0.245 -0.111 -0.707

trtthyrxn:w 0.157 -0.111 -0.245 -0.642 0.454





Comparing Two Mixed Models for the Rat data

mixedmodel2.rat <- lmer(weight ~ week + (1 + week | ratid), data=rat)


Formula: weight ~ week + (1 + week | ratid)

Data: rat


928.2 945.6 -458.1 920.5 916.2

Random effects:


ratid (Intercept) 35.559 5.9631

week 35.615 5.9679 -0.404

Residual 18.806 4.3366

Number of obs: 135, groups: ratid, 27

Fixed effects:


(Intercept) 54.459 1.317 41.35

week 23.159 1.178 19.65


(Intr)

week -0.433





Comparing Two Mixed Models for the Rat data

anova(mixedmodel1.rat,mixedmodel2.rat)

Data: rat

Models:

mixedmodel2.rat: weight ~ week + (1 + week | ratid)

mixedmodel1.rat: weight ~ treat + week + treat:week + (1 + week | ratid)

Df AIC BIC logLik Chisq Chi Df Pr(>Chisq)

mixedmodel2.rat 6 932.51 949.94 -460.26

mixedmodel1.rat 10 915.17 944.22 -447.58 25.341 4 4.296e-05 ***




The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model

Outline








I Assuming that we have a X matrix, a Y vector, we are interested in finding β such that E [Y |X ] = Xβand the errors between the estimated Y and the observed Y are reduced or in some sense this is the ”mostlikely” β

I A random model assumes that some part of β is not fixed, i.e. there are βi in β which are random

I We will briefly touch the subject why some βs may be random





I What does it mean when some factor and its corresponding βi is random?

I It means in some sense that βi ”fluctuates” around a value (typically zero) with some variance.

I This motivates us to write the model in a different way than E [Y |X ] = Xβ





I Maybe now that we have assumed that β is random with a random part b (let β be completely random)we can build a model given the random value of β.

I Note since β is random we can build a model conditionally knowing the random value of β.

I The model is E [Y |b] = X (β + b) = Xβ + Xb

I Typically we can add random components outside β also and the model is written as E [Y |b] = Xβ + Zb





The Hierarchical Model

For an arbitrary error distribution G , the model now can be written as

Yi |bi = Xiβ + Zi bi + εi

bi ∼ G(0,D)

εi ∼ G(0, Ri )

Cov(ei , bi ) = 0





But which variables are random? Typically

I if the levels of the effect were chosen at random from a population list of possible levels - e.g. subject effect

I or we have enough reason to believe that the effect on the outcome is stochastic in nature (apart frompure error) - e.g. repeated measures at different time points imparting autoregressive variance component





Suppose our design is as follows:

Figure: Five planks of wood were randomly assigned to a subject. Five different treatments (loading scenarios- color coded) were randomly assigned to each of the five planks of wood. The subject was asked to walk on theplank of wood and at location d1, . . . , d4 of the plank, a response was measured. No effect overflow is assumedsince the plank changes and effect of loading scenario on a subject is not possible.The same process is repeated on ten subjects each time with five different planks of wood.





Figure: Plot of response with distance at different loadings and fordifferent subjects





Figure: Fitted linear regression line on response with distance as thepredictor at different loadings and for different subjects





R-Code: Plots

xyplot(mechanicalresp ~ location | subject, data=mechresp, groups=loading,

panel="panel.superpose",panel.groups="panel.lmline",

key = list(lines = Rows(trellis.par.get("superpose.line"),

c(1:5, 1)),text = list(lab = as.character(unique(mechresp $loading))),columns = 3, title = "Loading"))

xyplot(mechanicalresp ~ location | subject, data=mechresp, groups=loading,

panel="panel.superpose",panel.groups="panel.xyplot",

key = list(lines = Rows(trellis.par.get("superpose.line"),

c(1:5, 1)), text = list(lab = as.character(unique(mechresp $loading))),columns = 3, title = "Loading"))

loadingA.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="A", data= mechresp)

loadingB.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="B", data= mechresp)

loadingC.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="C", data= mechresp)

loadingD.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="D", data= mechresp)

loadingE.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="E", data= mechresp)

loadingA.coef <- coef(loadingA.list)

loadingB.coef <- coef(loadingB.list)

loadingC.coef <- coef(loadingC.list)

loadingD.coef <- coef(loadingD.list)

loadingE.coef <- coef(loadingE.list)

old <- par(mfrow=c(1,2))

boxplot(loadingA.coef[,1], loadingB.coef[,1], loadingC.coef[,1], loadingD.coef[,1], loadingE.coef[,1],

main="Box Plot of Intercepts",names=c("A","B","C","D","E"))

boxplot(loadingA.coef[,2], loadingB.coef[,2], loadingC.coef[,2], loadingD.coef[,2], loadingE.coef[,2],

main="Box Plot of Slopes(location)",names=c("A","B","C","D","E"))

par(old)





Looking at the plots we would probably want to build a model where the response depended on the loading, thedistance and the interaction between loading and distance.

However, due to the subjects these linear regression lines would probably have a random intercept and a randomslope.

We would be interested in knowing what the average intercept and the average slope might be for the model.





A probable model for explaining the mechanical response might be the following. yijd is the response of the i th

subject due to the jth loading at location d . Note also the residual errors at two locations of the same plank ofwood is correlated since they are at different locations on the same plank. The two responses closer in location onthe same plank are probably highly correlated.

yijd = α + ai + γj + bij + (β + δj )d + εijd

i = 1, . . . 10 j = A, . . . , E d = d1, . . . d4

I α = mean mechanical response over all loadings, locations and subjects

I ai ∼ N(0, σ2a ) = random deviation due to i th subject from mean response

I γj = effect of the treatment factor (loading) at level j . Loading is fully nested within subject.j = A, B, C ,D, E .

I β = mean rate of change of response with respect to location

I bij ∼ N(0, σ2b) = random deviation due to ijth plank

I δj = interaction effect between the treatment factor (loading) and location

I εijd ∼ N(0, σ2) = random residual error which are correlated on the same plank ij





But what kind of covariance structure do we assume for the random effects?Y will have the corresponding block covariance structure, with blocks being defined by clusters. We will look atclusters in our problem. Let us focus on the general problem when clusters have been identified.





Covariance Structures





SAS Code for Loading Data

proc mixed data=mixed2 asycov;

class subject loading locclass;

model mechanicalresp = loading location loading*location/ solution;

random subject subject*loading;

repeated locclass / subject=subject*loading type=ar(1);

/* locclass is a class variable created from the locations to assign a covariance structure

to residual errors at the different locations for the same plank */

lsmeans loading/at location=0.125;




run;






Fit Statistics

-2 Res Log Likelihood -232.2

AIC (smaller is better) -226.2

AICC (smaller is better) -226.0

BIC (smaller is better) -225.3

Solution for Fixed Effects

Standard

Effect loading Estimate Error DF t Value Pr > |t|

Intercept 0.1023 0.07366 9 1.39 0.1984

loading A 0.2770 0.08777 36 3.16 0.0032

loading B 0.1113 0.08777 36 1.27 0.2130

loading C 0.04727 0.08777 36 0.54 0.5935

loading D 0.04231 0.08777 36 0.48 0.6327

loading E 0 . . . .

location 0.7076 0.1530 145 4.62 <.0001

location*loading A 2.1634 0.2164 145 10.00 <.0001

location*loading B 1.2185 0.2164 145 5.63 <.0001

location*loading C 0.7067 0.2164 145 3.27 0.0014

location*loading D 0.5084 0.2164 145 2.35 0.0201

location*loading E 0 . . . .

Type 3 Tests of Fixed Effects

Num Den

Effect DF DF F Value Pr > F

loading 4 36 3.08 0.0279

location 1 145 565.32 <.0001

location*loading 4 145 28.79 <.0001






Covariance Parameter Estimates

Cov Parm Subject Estimate

subject 0.01575

subject*loading 3.51E-36

AR(1) subject*loading 0.6868

Residual 0.02437

Asymptotic Covariance Matrix of Estimates

Row Cov Parm CovP1 CovP2 CovP3 CovP4

1 subject 0.000080 -0.00005 -3.41E-6

2 subject*loading

3 AR(1) -0.00005 0.002918 0.000158

4 Residual -3.41E-6 0.000158 0.000015





What is a split-plot design?

A large experimental unit, the whole plot, is split into smaller plots. The factor that is applied to the whole plot iscalled the whole plot factor.

Once the whole plot factor has been administered, all levels of the second factor is administered into the split plotswithin the whole plot.

Each whole unit is thus a complete replicate of all levels of the split plot factor factor.





When do we use it?

I Split plot unit variance is smaller that whole plot factor variance.

I Hence split plot and its interactions tested with greater sensitivity

I So in general when testing for one factor is more sensitive than the other factor.

I When a larger sized experimental unit is required for one factor than the other factor.





High variance and few replications of whole plot units frequently leads to poor sensitivity on the whole unit factor.





The Shoe Example

A shoe company ran an experiment on human athlete legs to find the effect of low ankle and high ankle design ofshoes and presence or absence of laces on both legs (right and left) on the force that a shoe can withstand. Theathletes can be treated as random blocking factors. Each leg can be thought of the whole plot with left and rightbeing the whole plot factor - within each leg, a full factorial of the split plot factors i.e. the ankle design andlace/no-lace has been replicated once. The response, force (in lbf) is measured three times on each shoe. This iscalled subsampling.





The Shoe Example

data shoes;

input subject age bodymass bodyht leg side $ shoe $ lace $ trial force_lbf angle_deg;

cards;

1 62 59.5 177.8 1 right Low Tied 1 170.46 49.07

1 62 59.5 177.8 1 right Low Tied 2 172.68 50.34

1 62 59.5 177.8 1 right Low Tied 3 171.93 50.62

1 62 59.5 177.8 1 right High Tied 1 133.26 45.93

1 62 59.5 177.8 1 right High Tied 2 138.84 47.29

1 62 59.5 177.8 1 right High Tied 3 142.61 47.72

1 62 59.5 177.8 1 right Low Untied 1 167.84 51.38

1 62 59.5 177.8 1 right Low Untied 2 166.81 50.80

1 62 59.5 177.8 1 right Low Untied 3 169.12 51.31

1 62 59.5 177.8 1 right High Untied 1 145.55 46.46



1 62 59.5 177.8 2 left Low Tied 1 95.44 52.48

1 62 59.5 177.8 2 left Low Tied 2 98.09 52.58

1 62 59.5 177.8 2 left Low Tied 3 97.71 53.23

1 62 59.5 177.8 2 left High Tied 1 88.31 44.78

1 62 59.5 177.8 2 left High Tied 2 91.85 46.30

1 62 59.5 177.8 2 left High Tied 3 91.87 47.41

1 62 59.5 177.8 2 left Low Untied 1 96.28 50.91

1 62 59.5 177.8 2 left Low Untied 2 104.31 52.59

1 62 59.5 177.8 2 left Low Untied 3 99.89 52.96

1 62 59.5 177.8 2 left High Untied 1 93.30 48.25

1 62 59.5 177.8 2 left High Untied 2 94.57 49.54

1 62 59.5 177.8 2 left High Untied 3 94.85 50.52





The Shoe Example - SAS Code and Ouput

proc mixed data=shoesmean_over_subsampling;

class subject side ankle lace;

model meanforce = side ankle lace ankle*lace side*ankle*lace;

random subject subject*side / solution;

ods output tests3=b;

run;

%data power;

%set b;

%alpha=0.05;

%fcrit=finv(1-alpha,numdf,dendf);

%power=1-probf(fcrit,numdf,dendf,numdf*fvalue);

%run;






The Mixed Procedure

Covariance Parameter

Estimates

Cov Parm Estimate

subject 2195.09

subject*side 2437.64

Residual 239.79

Fit Statistics

-2 Res Log Likelihood 80.6

AIC (smaller is better) 86.6

AICC (smaller is better) 92.6

BIC (smaller is better) 82.6






Solution for Random Effects

Std Err

Effect side subject Estimate Pred DF t Value Pr > |t|

subject 1 -26.4492 38.6720 6 -0.68 0.5195

subject 2 26.4492 38.6720 6 0.68 0.5195

subject*side left 1 -39.0740 40.3383 6 -0.97 0.3701

subject*side right 1 9.7023 40.3383 6 0.24 0.8179

subject*side left 2 39.0740 40.3383 6 0.97 0.3701

subject*side right 2 -9.7023 40.3383 6 -0.24 0.8179

Type 3 Tests of Fixed Effects

Num Den

Effect DF DF F Value Pr > F

side 1 1 0.05 0.8538

ankle 1 6 0.86 0.3882

lace 1 6 5.02 0.0663

ankle*lace 1 6 1.51 0.2652

side*ankle*lace 3 6 0.28 0.8375






Proc mixed for force(lbf)

Num Den

Obs Effect DF DF FValue ProbF alpha fcrit power

1 side 1 1 0.05 0.8538 0.05 161.448 0.05135

2 ankle 1 6 0.86 0.3882 0.05 5.987 0.12384

3 lace 1 6 5.02 0.0663 0.05 5.987 0.46887

4 ankle*lace 1 6 1.51 0.2652 0.05 5.987 0.18026

5 side*ankle*lace 3 6 0.28 0.8375 0.05 4.757 0.07973




The Respiratory Illness ExampleA Bayesian Hierarchical Model

Outline








The data are from a clinical trial of patients with respiratory illness, where 111 patients from two different clinicswere randomized to receive either placebo or an active treatment. Patents were examined at baseline and fourvisits during treatment. At each visit, respiratory status was recorded as 1=good and 0=poor. The other recordedvariables were center (=1 or 2), treatment (=A for Active or P for Placebo), Gender (=M for male and F forfemale), age (in years at baseline) and baseline response.





The Respiratory Illness Example - Data

respiratory <- read.table("respiratory.txt", header = TRUE)

respiratory <- transform(respiratory, age2 = (age/10)^2,center = factor(center),

visit = factor(visit),patid = interaction(center, id))

center id treat sex age baseline visit outcome age2 patid

1 1 1 P M 46 0 1 0 21.16 1.1

2 1 1 P M 46 0 2 0 21.16 1.1

3 1 1 P M 46 0 3 0 21.16 1.1

4 1 1 P M 46 0 4 0 21.16 1.1

5 1 2 P M 28 0 1 0 7.84 1.2

6 1 2 P M 28 0 2 0 7.84 1.2

7 1 2 P M 28 0 3 0 7.84 1.2

8 1 2 P M 28 0 4 0 7.84 1.2

9 1 3 A M 23 1 1 1 5.29 1.3

10 1 3 A M 23 1 2 1 5.29 1.3

11 1 3 A M 23 1 3 1 5.29 1.3

12 1 3 A M 23 1 4 1 5.29 1.3

13 1 4 P M 44 1 1 1 19.36 1.4

14 1 4 P M 44 1 2 1 19.36 1.4





Figure: Mean outcome across visits for patients against age





A probable hierarchical model for binary response can model the logit of the probability of observing 1 in thepatient’s visit with a linear predictor.

yiv = Bin(1, piv ), independent for v = 1, . . . , 4

log(piv

1− piv

) = β0 + β1baseline(i) + β2center(i) + β3gender(i) + β4treat(i)

β5age(i) + β6(age(i)

10)2 + spatient(i)

spatient(i) ∼ N(0, σs2), independent for different i

i = 1, . . . 111 v = 1, . . . , 4

I spatient(i) ∼ N(0, σ2s ) = random deviation due to i th patient from mean response





The Respiratory Illness Example - R Code and Output

mixedmodel.glm <- lmer(outcome ~ baseline + center + sex +treat +

age + age2 + (1 | patid), data = respiratory,family=binomial)

Generalized linear mixed model fit by the Laplace approximation

Formula: outcome ~ baseline + center + sex + treat + age + age2 + (1 | patid)

Data: respiratory

AIC BIC logLik deviance

440.9 473.7 -212.4 424.9

Random effects:

Groups Name Variance Std.Dev.

patid (Intercept) 3.1998 1.7888

Number of obs: 444, groups: patid, 111

Fixed effects:

Estimate Std. Error z value Pr(>|z|)

(Intercept) 5.53290 1.87130 2.957 0.00311 **

baseline 2.86340 0.49680 5.764 8.23e-09 ***

center2 0.75956 0.50042 1.518 0.12906

sexM -0.60686 0.64753 -0.937 0.34866

treatP -2.01317 0.48807 -4.125 3.71e-05 ***

age -0.30711 0.09476 -3.241 0.00119 **

age2 0.38901 0.13030 2.985 0.00283 **





The Respiratory Illness Example - R Code and Output


(Intr) baseln centr2 sexM treatP age

baseline 0.041

center2 -0.167 -0.256

sexM -0.567 -0.145 0.190

treatP -0.250 -0.199 0.007 0.248

age -0.902 -0.078 0.059 0.258 0.086

age2 0.838 0.094 -0.101 -0.210 -0.092 -0.978





A Bayesian Model with Binary Response

A binary response was tracked on 300 individuals over 4 time points. Along with this response two body factorswere also measured at each of those time points. The researcher wants to know if the two body factors that weretracked over time affects the response. However, he also wants to take into consideration the random effects dueto the subjects in the study.





The key concept in Bayesian model is the likelihood function. The likelihood expresses our notion about we how”likely” the observed data is but itis a function of our parameters. Change the values of the parameters and the likelihood of the observed data changes.

It is obvious that a model with random effects due to the subjects should be used to model the data.





The Model

The binary variable will be modeled using probit regression i.e. if πit is the probability that the response for i th

individual at time t will be 1 then,

πit = φ(xTit β + zT

i θ)

The Bayesian model will use a latent factor to model the response. The latent factor structure maintains thelikelihood function as required by our model and also makes calculations simpler.





The hierarchical structure in a Bayesian model also includes our prior belief about β, θ and ψ−1.

yit = 1, if γit > 0

= 0, if γit ≤ 0

γit ∼ N(xTit β + zT

i θ)

β ∼ MVN(β0,Σ0)

θ ∼ MVN(0, ψ−1I)

ψ ∼ Gamma(ε, ε)





Estimates obtained from the Gibb’s Sampler.





References

http://cran.r-project.org/doc/contrib/Fox-Companion/appendix-mixed-models.pdfContemporary Statistical Models for the Plant and Soil Sciences by Oliver Schabenberger, Francis J. PierceSAS for mixed models by Ramon C. Littell


Mixed Models and Hierarchical Models - A Short Overview

Documents

Transcript of Mixed Models and Hierarchical Models - A Short Overview