OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
Mixed Models and Hierarchical Models - AShort Overview
Dipayan Maiti
LISA Short CourseLaboratory for Interdisciplinary Statistical Analysis
Department of Statistics
Virginia Tech
http://www.lisa.stat.vt.edu
July 8, 2009
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
Outline
The Mixed Effects Model Through ExamplesThe High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The Mathematical ModelThe RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Generalized Linear Mixed ModelsThe Respiratory Illness ExampleA Bayesian Hierarchical Model
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Outline
The Mixed Effects Model Through ExamplesThe High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The Mathematical ModelThe RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Generalized Linear Mixed ModelsThe Respiratory Illness ExampleA Bayesian Hierarchical Model
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The Data
Data is from a 1982 survey of 7185 High School students from 160 Schools.
The data has some student specific attributes and some school specific attributes.
You can visualize the schools as groups and within each group (school) there are students.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Figure: High School data
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Description of Variables
I school: school ID for student´s school
I ses: socioeconomic status of student´s family
I mathach: students score on a math-achievement test
I meanses: mean SES score for each school
I cses: socioeconomic status of student´s family centered by mean SES score for his school
I sector: public or catholic school
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Figure: Plot of Math Achievement Score by Socio Economic Status forCatholic Schools
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Figure: Plot of Math Achievement Score by Socio Economic Status forPublic Schools
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Figure: Plot of Intercepts and Slopes for Catholic SchoolsDipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Figure: Plot of Intercepts and Slopes for Public SchoolsDipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Figure: Box Plot Comparisons of Intercepts and Slopes for Catholic andPublic Schools
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
What does this tell us about the model?
I If we model the math score (mathach) by centered socioeconomic status (cses) for students of a specificschool the intercept is on the average 14 for Catholic Schools and 12 for Public Schools
I The slope is on the average 1.5 for Catholic Schools and just about less than 3 for Public Schools
I The slope signifies the rate of change of math score (mathach) with respect to the centered socioeconomicstatus (cses)
I However both the intercept and the slope seem to have a random distribution around these expected values
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
How do we build a model that takes into consideration all the salient features of the data that we have been ableto obtain in this exploratory study?
Can we build a hierarchical model that makes the intercept and slope random?
Maybe the slope and intercept for each school can be modeled based on the mean socioeconomic status for theschool and if the school is Catholic or Public.
This slope and intercept can be used for that specific school to model math score of the students based on theirindividual centered socioeconomic status (cses).
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
A Possible Hierarchical ModelFor student j in school i
mathachij = β0 i + β1 i Xcses ij + εij (1)
β0 i = α0 i + α1 i Xsector i + α2 i Xmeanses i + εβ0 i(2)
β1 i = γ0 i + γ1 i Xsector i + γ2 i Xmeanses i + εβ1 i(3)
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
What does the complete model look like?
If we plug the values of β0 i and β1 i in the first model we obtain:
mathachij = β0 + β1Xcses ij + β2Xsector i + β4Xmeanses i +
β5Xmeanses i ∗ Xsector i + β6Xmeanses i ∗ Xcses ij +
γ1 + γ2Xcses ij + εij
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Let us code Xsector such that 0 is Public and 1 Catholic.For students in Public School i , the equation becomes,
mathachij = (β0 + γ1) + (β1 + γ2)Xcses ij + β4Xmeanses i +
β6Xmeanses i ∗ Xcses ij + εij
For students in Catholic School i , the equation becomes,
mathachij = (β0 + β2 + γ1) + (β1 + γ2)Xcses ij + (β4 + β5)Xmeanses i
+β6Xmeanses i ∗ Xcses ij + εij
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
R-Output: Random Effects Covariance Matrix
> mixedmodel.lme.1 <- lme(mathach ~ meanses + cses + sector + meanses*cses +
sector*cses, random = ~ cses | school, data= mixedmodel)
> summary(mixedmodel.lme.1)
Linear mixed-effects model fit by REML
Data: mixedmodel
AIC BIC logLik
46523.66 46592.45 -23251.83
Random effects:
Formula: ~cses | school
Structure: General positive-definite, Log-Cholesky parametrization
StdDev Corr
(Intercept) 1.5426480 (Intr)
cses 0.3178422 0.391
Residual 6.0598009
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
R-Output: Fixed Effects Estimates
Fixed effects: mathach ~ meanses + cses + sector + meanses * cses + sector * cses
Value Std.Error DF t-value p-value
(Intercept) 12.127931 0.1992951 7022 60.85415 0e+00
meanses 5.332878 0.3691743 157 14.44542 0e+00
cses 2.945043 0.1555906 7022 18.92815 0e+00
sectorCatholic 1.226578 0.3062783 157 4.00478 1e-04
meanses:cses 1.039249 0.2988794 7022 3.47715 5e-04
cses:sectorCatholic -1.642680 0.2397649 7022 -6.85121 0e+00
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
R-Output: Fixed Effect Correlations & Within GroupVariance Estimates
(Intr) meanss cses sctrCt mnss:c
meanses 0.256
cses 0.075 0.019
sectorCatholic -0.699 -0.356 -0.053
meanses:cses 0.019 0.074 0.293 -0.026
cses:sectorCatholic -0.052 -0.027 -0.696 0.077 -0.351
Standardized Within-Group Residuals:
Min Q1 Med Q3 Max
-3.15926307 -0.72319713 0.01704114 0.75443404 2.95823100
Number of Observations: 7185
Number of Groups: 160
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
R-Output: Comparing Mixed Effects Models
> mixedmodel.lme.2 <- lme(mathach ~ meanses + cses + sector +
meanses*cses + sector*cses, random = ~ 1| school, data= mixedmodel)
> summary(mixedmodel.lme.2) # omitting random effect of cses
-------------------------------------------------------------------------------------------------------
Linear mixed-effects model fit by REML
AIC BIC logLik
46520.79 46575.82 -23252.39
Random effects:
Formula: ~1 | school
(Intercept) Residual
StdDev: 1.541213 6.063503
> mixedmodel.lme.3 <- lme(mathach ~ meanses + cses + sector +
meanses*cses + sector*cses, random = ~ cses -1 | school, data= mixedmodel)
> summary(mixedmodel.lme.3) # omitting random intercept
-----------------------------------------------------------------------------------------------------
Linear mixed-effects model fit by REML
AIC BIC logLik
46740.23 46795.26 -23362.11
Random effects:
Formula: ~cses - 1 | school
cses Residual
StdDev: 0.006713239 6.247727
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Another Example: A scientist tracks the growth of pigs for nine consecutive weeks. He wants to estimate thetypical growth curve of the population of pigs.
Figure: Pig Growth over nine weeksDipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
I Although the growth rate is linear the starting weight of the pigs differ considerably
I If the starting weight was higher it tends to remain higher and maybe even more so at the end of the nineweeks
I The growth rate probably should be explained by not only pig-specific starting weight but also pig-specificgrowth rates.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The growth model of the pigs can be explained using the following model. yit is the weight of the i th pig at the tth
week.
yit = α + ai + (β + bi )t + εit
i = 1, . . . 48 t = 1, . . . 9
I α = mean weight of pigs at week zero
I ai ∼ N(0, σ2a ) = random deviation due to i th pig from mean weight at week zero
I β = mean growth rate across all pigs
I bi ∼ N(0, σ2b) = random deviation due to i th pig from mean growth rate
I εit ∼ N(0, σ2) = random residual error
I ai and bi are uncorrelated with the random error term εit
I However there might be some correlation between ai and bi = Cov(ai , bi )
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
According to the above model the following variance and covariance formula can be derived
Var(yit , yit ) = σ2a + t2
σ2b + 2tCov(ai , bi ) + σ
2
Cov(yi1, yit ) = σ2a + tσ2
b + (t + 1)Cov(ai , bi ) t = 2, . . . , 9
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Pig data
library(lme4)
library(lattice)
pigweight <- read.table("pigweight.txt", header = TRUE)
week1 week2 week3 week4 week5 week6 week7 week8 week9
1 24.0 32.0 39.0 42.5 48.0 54.5 61.0 65.0 72.0
2 22.5 30.5 40.5 45.0 51.0 58.5 64.0 72.0 78.0
3 22.5 28.0 36.5 41.0 47.5 55.0 61.0 68.0 76.0
4 24.0 31.5 39.5 44.5 51.0 56.0 59.5 64.0 67.0
5 24.5 31.5 37.0 42.5 48.0 54.0 58.0 63.0 65.5
6 23.0 30.0 35.5 41.0 48.0 51.5 56.5 63.5 69.5
7 22.5 28.5 36.0 43.5 47.0 53.5 59.5 67.5 73.5
8 23.5 30.5 38.0 41.0 48.5 55.0 59.5 66.5 73.0
9 20.0 27.5 33.0 39.0 43.5 49.0 54.5 59.5 65.0
10 25.5 32.5 39.5 47.0 53.0 58.5 63.0 69.5 76.0
11 24.5 31.0 40.5 46.0 51.5 57.0 62.5 69.5 76.0
12 24.0 29.0 39.0 44.0 50.5 57.0 61.5 68.0 73.5
pigweightL <- reshape(pigweight, direction = "long",varying = list(colnames(pigweight)),
v.name = "weight",timevar = "week", idvar = "pigid")
pigweightL <- with(pigweightL, pigweightL[order(pigid,week),])
mixedmodel.pig <- lmer(weight ~ 1+week+(1+week|pigid),data=piweightL)
print(xyplot(weight ~ week, groups = pigid, data = pigweightL,type = "l", col = 1))
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Pig data
pigweightL <- reshape(pigweight, direction = "long",varying = list(colnames(pigweight)),
v.name = "weight",timevar = "week", idvar = "pigid")
week weight pigid
1.1 1 24.0 1
1.2 2 32.0 1
1.3 3 39.0 1
1.4 4 42.5 1
1.5 5 48.0 1
1.6 6 54.5 1
1.7 7 61.0 1
1.8 8 65.0 1
1.9 9 72.0 1
2.1 1 22.5 2
2.2 2 30.5 2
2.3 3 40.5 2
2.4 4 45.0 2
pigweightL <- with(pigweightL, pigweightL[order(pigid,week),])
mixedmodel.pig <- lmer(weight ~ 1+week+(1+week|pigid),data=piweightL)
print(xyplot(weight ~ week, groups = pigid, data = pigweightL,type = "l", col = 1))
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Pig data
Linear mixed model fit by REML
Formula: weight ~ 1 + week + (1 + week | pigid)
Data: pigweightL
AIC BIC logLik deviance REMLdev
1753 1777 -870.4 1738 1741
Random effects:
Groups Name Variance Std.Dev. Corr
pigid (Intercept) 6.9865 2.64319
week 0.3800 0.61644 -0.064
Residual 1.5968 1.26366
Number of obs: 432, groups: pigid, 48
Fixed effects:
Estimate Std. Error t value
(Intercept) 19.35561 0.40387 47.93
week 6.20990 0.09204 67.47
Correlation of Fixed Effects:
(Intr)
week -0.133
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The growth of rats is tracked for four weeks (with a baseline measure at week zero) under the influence of twosubstances thiouracil and thyroxinil. The growth measurements are also obtained for control rats for the same timeperiod.
Figure: Rat Growth over four weeks for control, thiouracil and thyroxinilgroups
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Rat data
ratsbodyweight <- read.table("ratsbodyweight.txt", header = TRUE)
ratid treat week0 week1 week2 week3 week4
1 1 control 57 86 114 139 172
2 2 control 60 93 123 146 177
3 3 control 52 77 111 144 185
4 4 control 49 67 100 129 164
5 5 control 56 81 104 121 151
6 6 control 46 70 102 131 153
7 7 control 51 71 94 110 141
8 8 control 63 91 112 130 154
9 9 control 49 67 90 112 140
10 10 control 57 82 110 139 169
11 11 thiouracil 61 86 109 120 129
12 12 thiouracil 59 80 101 111 122
13 13 thiouracil 53 79 100 106 133
14 14 thiouracil 59 88 100 111 122
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Rat data
rat <- reshape(ratsbodyweight,direction="long",varying=list(paste("week",c(0:4),sep="")),
v.name="weight",idvar="ratid",timevar="week",time=c(0:4))
rat <- transform(rat, treat = factor(treat))
ratid treat week weight
1.0 1 control 0 57
2.0 2 control 0 60
3.0 3 control 0 52
4.0 4 control 0 49
5.0 5 control 0 56
6.0 6 control 0 46
7.0 7 control 0 51
8.0 8 control 0 63
9.0 9 control 0 49
10.0 10 control 0 57
11.0 11 thiouracil 0 61
12.0 12 thiouracil 0 59
13.0 13 thiouracil 0 53
14.0 14 thiouracil 0 59
library(lattice)
print(xyplot(weight ~ week | treat, groups = ratid,data=rat,type="l"))
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The growth model of the rats can be explained using the following model. yit is the weight of the i th rat at the tth
week.
yit = α + ai + γtreat(i) + (β + bi + δtreat(i))t + εit
i = 1, . . . 48 t = 1, . . . 9
I α = mean weight of rats at week zero
I ai ∼ N(0, σ2a ) = random deviation due to i th rat from mean weight at week zero
I β = mean growth rate across all rats
I bi ∼ N(0, σ2b) = random deviation due to i th rat from mean growth rate
I γtreat(i) = effect of the treatment factor (control/thiouracil/thyroxinil) that has been applied to the ith rat
I δtreat(i) = interaction effect between the treatment factor (control/thiouracil/thyroxinil) that has been
applied to the ith rat and time t
I εit ∼ N(0, σ2) = random residual error
I ai and bi are uncorrelated with the random error term εit
I However there might be some correlation between ai and bi = Cov(ai , bi )
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Rat data
mixedmodel.rat <- lmer(weight ~ treat + week + treat:week + (1 + week | ratid), data=rat)
Linear mixed model fit by REML
Formula: weight ~ treat + week + treat:week + (1 + week | ratid)
Data: rat
AIC BIC logLik deviance REMLdev
898 927 -439 895.2 878
Random effects:
Groups Name Variance Std.Dev. Corr
ratid (Intercept) 32.404 5.6925
week 13.963 3.7367 -0.127
Residual 18.806 4.3366
Number of obs: 135, groups: ratid, 27
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Mixed Model Analysis of the Rat data
Fixed effects:
Estimate Std. Error t value
(Intercept) 52.8800 2.0902 25.299
treatthiouracil 4.8200 2.9559 1.631
treatthyroxin -0.7943 3.2573 -0.244
week 26.4800 1.2587 21.037
treatthiouracil:week -9.4300 1.7801 -5.297
treatthyroxin:week 0.6629 1.9616 0.338
Correlation of Fixed Effects:
(Intr) trtthr trtthy week trtthr:
treatthircl -0.707
treatthyrxn -0.642 0.454
week -0.245 0.173 0.157
trtthrcl:wk 0.173 -0.245 -0.111 -0.707
trtthyrxn:w 0.157 -0.111 -0.245 -0.642 0.454
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Comparing Two Mixed Models for the Rat data
mixedmodel2.rat <- lmer(weight ~ week + (1 + week | ratid), data=rat)
Linear mixed model fit by REML
Formula: weight ~ week + (1 + week | ratid)
Data: rat
AIC BIC logLik deviance REMLdev
928.2 945.6 -458.1 920.5 916.2
Random effects:
Groups Name Variance Std.Dev. Corr
ratid (Intercept) 35.559 5.9631
week 35.615 5.9679 -0.404
Residual 18.806 4.3366
Number of obs: 135, groups: ratid, 27
Fixed effects:
Estimate Std. Error t value
(Intercept) 54.459 1.317 41.35
week 23.159 1.178 19.65
Correlation of Fixed Effects:
(Intr)
week -0.433
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
Comparing Two Mixed Models for the Rat data
anova(mixedmodel1.rat,mixedmodel2.rat)
Data: rat
Models:
mixedmodel2.rat: weight ~ week + (1 + week | ratid)
mixedmodel1.rat: weight ~ treat + week + treat:week + (1 + week | ratid)
Df AIC BIC logLik Chisq Chi Df Pr(>Chisq)
mixedmodel2.rat 6 932.51 949.94 -460.26
mixedmodel1.rat 10 915.17 944.22 -447.58 25.341 4 4.296e-05 ***
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Outline
The Mixed Effects Model Through ExamplesThe High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The Mathematical ModelThe RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Generalized Linear Mixed ModelsThe Respiratory Illness ExampleA Bayesian Hierarchical Model
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
I Assuming that we have a X matrix, a Y vector, we are interested in finding β such that E [Y |X ] = Xβand the errors between the estimated Y and the observed Y are reduced or in some sense this is the ”mostlikely” β
I A random model assumes that some part of β is not fixed, i.e. there are βi in β which are random
I We will briefly touch the subject why some βs may be random
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
I What does it mean when some factor and its corresponding βi is random?
I It means in some sense that βi ”fluctuates” around a value (typically zero) with some variance.
I This motivates us to write the model in a different way than E [Y |X ] = Xβ
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
I Maybe now that we have assumed that β is random with a random part b (let β be completely random)we can build a model given the random value of β.
I Note since β is random we can build a model conditionally knowing the random value of β.
I The model is E [Y |b] = X (β + b) = Xβ + Xb
I Typically we can add random components outside β also and the model is written as E [Y |b] = Xβ + Zb
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Hierarchical Model
For an arbitrary error distribution G , the model now can be written as
Yi |bi = Xiβ + Zi bi + εi
bi ∼ G(0,D)
εi ∼ G(0, Ri )
Cov(ei , bi ) = 0
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
But which variables are random? Typically
I if the levels of the effect were chosen at random from a population list of possible levels - e.g. subject effect
I or we have enough reason to believe that the effect on the outcome is stochastic in nature (apart frompure error) - e.g. repeated measures at different time points imparting autoregressive variance component
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Suppose our design is as follows:
Figure: Five planks of wood were randomly assigned to a subject. Five different treatments (loading scenarios- color coded) were randomly assigned to each of the five planks of wood. The subject was asked to walk on theplank of wood and at location d1, . . . , d4 of the plank, a response was measured. No effect overflow is assumedsince the plank changes and effect of loading scenario on a subject is not possible.The same process is repeated on ten subjects each time with five different planks of wood.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Figure: Plot of response with distance at different loadings and fordifferent subjects
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Figure: Fitted linear regression line on response with distance as thepredictor at different loadings and for different subjects
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
R-Code: Plots
xyplot(mechanicalresp ~ location | subject, data=mechresp, groups=loading,
panel="panel.superpose",panel.groups="panel.lmline",
key = list(lines = Rows(trellis.par.get("superpose.line"),
c(1:5, 1)),text = list(lab = as.character(unique(mechresp $loading))),columns = 3, title = "Loading"))
xyplot(mechanicalresp ~ location | subject, data=mechresp, groups=loading,
panel="panel.superpose",panel.groups="panel.xyplot",
key = list(lines = Rows(trellis.par.get("superpose.line"),
c(1:5, 1)), text = list(lab = as.character(unique(mechresp $loading))),columns = 3, title = "Loading"))
loadingA.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="A", data= mechresp)
loadingB.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="B", data= mechresp)
loadingC.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="C", data= mechresp)
loadingD.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="D", data= mechresp)
loadingE.list <- lmList(mechanicalresp ~ location | subject, subset = loading=="E", data= mechresp)
loadingA.coef <- coef(loadingA.list)
loadingB.coef <- coef(loadingB.list)
loadingC.coef <- coef(loadingC.list)
loadingD.coef <- coef(loadingD.list)
loadingE.coef <- coef(loadingE.list)
old <- par(mfrow=c(1,2))
boxplot(loadingA.coef[,1], loadingB.coef[,1], loadingC.coef[,1], loadingD.coef[,1], loadingE.coef[,1],
main="Box Plot of Intercepts",names=c("A","B","C","D","E"))
boxplot(loadingA.coef[,2], loadingB.coef[,2], loadingC.coef[,2], loadingD.coef[,2], loadingE.coef[,2],
main="Box Plot of Slopes(location)",names=c("A","B","C","D","E"))
par(old)
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Looking at the plots we would probably want to build a model where the response depended on the loading, thedistance and the interaction between loading and distance.
However, due to the subjects these linear regression lines would probably have a random intercept and a randomslope.
We would be interested in knowing what the average intercept and the average slope might be for the model.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
A probable model for explaining the mechanical response might be the following. yijd is the response of the i th
subject due to the jth loading at location d . Note also the residual errors at two locations of the same plank ofwood is correlated since they are at different locations on the same plank. The two responses closer in location onthe same plank are probably highly correlated.
yijd = α + ai + γj + bij + (β + δj )d + εijd
i = 1, . . . 10 j = A, . . . , E d = d1, . . . d4
I α = mean mechanical response over all loadings, locations and subjects
I ai ∼ N(0, σ2a ) = random deviation due to i th subject from mean response
I γj = effect of the treatment factor (loading) at level j . Loading is fully nested within subject.j = A, B, C ,D, E .
I β = mean rate of change of response with respect to location
I bij ∼ N(0, σ2b) = random deviation due to ijth plank
I δj = interaction effect between the treatment factor (loading) and location
I εijd ∼ N(0, σ2) = random residual error which are correlated on the same plank ij
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
But what kind of covariance structure do we assume for the random effects?Y will have the corresponding block covariance structure, with blocks being defined by clusters. We will look atclusters in our problem. Let us focus on the general problem when clusters have been identified.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Covariance Structures
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
SAS Code for Loading Data
proc mixed data=mixed2 asycov;
class subject loading locclass;
model mechanicalresp = loading location loading*location/ solution;
random subject subject*loading;
repeated locclass / subject=subject*loading type=ar(1);
/* locclass is a class variable created from the locations to assign a covariance structure
to residual errors at the different locations for the same plank */
lsmeans loading/at location=0.125;
lsmeans loading/at location=0.250;
lsmeans loading/at location=0.375;
lsmeans loading/at location=0.500;
run;
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
SAS Code for Loading Data
Fit Statistics
-2 Res Log Likelihood -232.2
AIC (smaller is better) -226.2
AICC (smaller is better) -226.0
BIC (smaller is better) -225.3
Solution for Fixed Effects
Standard
Effect loading Estimate Error DF t Value Pr > |t|
Intercept 0.1023 0.07366 9 1.39 0.1984
loading A 0.2770 0.08777 36 3.16 0.0032
loading B 0.1113 0.08777 36 1.27 0.2130
loading C 0.04727 0.08777 36 0.54 0.5935
loading D 0.04231 0.08777 36 0.48 0.6327
loading E 0 . . . .
location 0.7076 0.1530 145 4.62 <.0001
location*loading A 2.1634 0.2164 145 10.00 <.0001
location*loading B 1.2185 0.2164 145 5.63 <.0001
location*loading C 0.7067 0.2164 145 3.27 0.0014
location*loading D 0.5084 0.2164 145 2.35 0.0201
location*loading E 0 . . . .
Type 3 Tests of Fixed Effects
Num Den
Effect DF DF F Value Pr > F
loading 4 36 3.08 0.0279
location 1 145 565.32 <.0001
location*loading 4 145 28.79 <.0001
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
SAS Code for Loading Data
Covariance Parameter Estimates
Cov Parm Subject Estimate
subject 0.01575
subject*loading 3.51E-36
AR(1) subject*loading 0.6868
Residual 0.02437
Asymptotic Covariance Matrix of Estimates
Row Cov Parm CovP1 CovP2 CovP3 CovP4
1 subject 0.000080 -0.00005 -3.41E-6
2 subject*loading
3 AR(1) -0.00005 0.002918 0.000158
4 Residual -3.41E-6 0.000158 0.000015
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
What is a split-plot design?
A large experimental unit, the whole plot, is split into smaller plots. The factor that is applied to the whole plot iscalled the whole plot factor.
Once the whole plot factor has been administered, all levels of the second factor is administered into the split plotswithin the whole plot.
Each whole unit is thus a complete replicate of all levels of the split plot factor factor.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
When do we use it?
I Split plot unit variance is smaller that whole plot factor variance.
I Hence split plot and its interactions tested with greater sensitivity
I So in general when testing for one factor is more sensitive than the other factor.
I When a larger sized experimental unit is required for one factor than the other factor.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
High variance and few replications of whole plot units frequently leads to poor sensitivity on the whole unit factor.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Shoe Example
A shoe company ran an experiment on human athlete legs to find the effect of low ankle and high ankle design ofshoes and presence or absence of laces on both legs (right and left) on the force that a shoe can withstand. Theathletes can be treated as random blocking factors. Each leg can be thought of the whole plot with left and rightbeing the whole plot factor - within each leg, a full factorial of the split plot factors i.e. the ankle design andlace/no-lace has been replicated once. The response, force (in lbf) is measured three times on each shoe. This iscalled subsampling.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Shoe Example
data shoes;
input subject age bodymass bodyht leg side $ shoe $ lace $ trial force_lbf angle_deg;
cards;
1 62 59.5 177.8 1 right Low Tied 1 170.46 49.07
1 62 59.5 177.8 1 right Low Tied 2 172.68 50.34
1 62 59.5 177.8 1 right Low Tied 3 171.93 50.62
1 62 59.5 177.8 1 right High Tied 1 133.26 45.93
1 62 59.5 177.8 1 right High Tied 2 138.84 47.29
1 62 59.5 177.8 1 right High Tied 3 142.61 47.72
1 62 59.5 177.8 1 right Low Untied 1 167.84 51.38
1 62 59.5 177.8 1 right Low Untied 2 166.81 50.80
1 62 59.5 177.8 1 right Low Untied 3 169.12 51.31
1 62 59.5 177.8 1 right High Untied 1 145.55 46.46
1 62 59.5 177.8 1 right High Untied 2 153.33 47.33
1 62 59.5 177.8 1 right High Untied 3 153.97 48.11
1 62 59.5 177.8 2 left Low Tied 1 95.44 52.48
1 62 59.5 177.8 2 left Low Tied 2 98.09 52.58
1 62 59.5 177.8 2 left Low Tied 3 97.71 53.23
1 62 59.5 177.8 2 left High Tied 1 88.31 44.78
1 62 59.5 177.8 2 left High Tied 2 91.85 46.30
1 62 59.5 177.8 2 left High Tied 3 91.87 47.41
1 62 59.5 177.8 2 left Low Untied 1 96.28 50.91
1 62 59.5 177.8 2 left Low Untied 2 104.31 52.59
1 62 59.5 177.8 2 left Low Untied 3 99.89 52.96
1 62 59.5 177.8 2 left High Untied 1 93.30 48.25
1 62 59.5 177.8 2 left High Untied 2 94.57 49.54
1 62 59.5 177.8 2 left High Untied 3 94.85 50.52
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Shoe Example - SAS Code and Ouput
proc mixed data=shoesmean_over_subsampling;
class subject side ankle lace;
model meanforce = side ankle lace ankle*lace side*ankle*lace;
random subject subject*side / solution;
ods output tests3=b;
run;
%data power;
%set b;
%alpha=0.05;
%fcrit=finv(1-alpha,numdf,dendf);
%power=1-probf(fcrit,numdf,dendf,numdf*fvalue);
%run;
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Shoe Example - SAS Code and Ouput
The Mixed Procedure
Covariance Parameter
Estimates
Cov Parm Estimate
subject 2195.09
subject*side 2437.64
Residual 239.79
Fit Statistics
-2 Res Log Likelihood 80.6
AIC (smaller is better) 86.6
AICC (smaller is better) 92.6
BIC (smaller is better) 82.6
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Shoe Example - SAS Code and Ouput
Solution for Random Effects
Std Err
Effect side subject Estimate Pred DF t Value Pr > |t|
subject 1 -26.4492 38.6720 6 -0.68 0.5195
subject 2 26.4492 38.6720 6 0.68 0.5195
subject*side left 1 -39.0740 40.3383 6 -0.97 0.3701
subject*side right 1 9.7023 40.3383 6 0.24 0.8179
subject*side left 2 39.0740 40.3383 6 0.97 0.3701
subject*side right 2 -9.7023 40.3383 6 -0.24 0.8179
Type 3 Tests of Fixed Effects
Num Den
Effect DF DF F Value Pr > F
side 1 1 0.05 0.8538
ankle 1 6 0.86 0.3882
lace 1 6 5.02 0.0663
ankle*lace 1 6 1.51 0.2652
side*ankle*lace 3 6 0.28 0.8375
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
The Shoe Example - SAS Code and Ouput
Proc mixed for force(lbf)
Num Den
Obs Effect DF DF FValue ProbF alpha fcrit power
1 side 1 1 0.05 0.8538 0.05 161.448 0.05135
2 ankle 1 6 0.86 0.3882 0.05 5.987 0.12384
3 lace 1 6 5.02 0.0663 0.05 5.987 0.46887
4 ankle*lace 1 6 1.51 0.2652 0.05 5.987 0.18026
5 side*ankle*lace 3 6 0.28 0.8375 0.05 4.757 0.07973
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
Outline
The Mixed Effects Model Through ExamplesThe High School Data ExampleModel Building for the High School DataThe Pig Growth Data ExampleThe One Factor Mixed Model Example with Rat Data
The Mathematical ModelThe RepresentationThe Loading Data Example - Repeated MeasuresA Split Plot of Design as a Mixed Model
Generalized Linear Mixed ModelsThe Respiratory Illness ExampleA Bayesian Hierarchical Model
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The data are from a clinical trial of patients with respiratory illness, where 111 patients from two different clinicswere randomized to receive either placebo or an active treatment. Patents were examined at baseline and fourvisits during treatment. At each visit, respiratory status was recorded as 1=good and 0=poor. The other recordedvariables were center (=1 or 2), treatment (=A for Active or P for Placebo), Gender (=M for male and F forfemale), age (in years at baseline) and baseline response.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The Respiratory Illness Example - Data
respiratory <- read.table("respiratory.txt", header = TRUE)
respiratory <- transform(respiratory, age2 = (age/10)^2,center = factor(center),
visit = factor(visit),patid = interaction(center, id))
center id treat sex age baseline visit outcome age2 patid
1 1 1 P M 46 0 1 0 21.16 1.1
2 1 1 P M 46 0 2 0 21.16 1.1
3 1 1 P M 46 0 3 0 21.16 1.1
4 1 1 P M 46 0 4 0 21.16 1.1
5 1 2 P M 28 0 1 0 7.84 1.2
6 1 2 P M 28 0 2 0 7.84 1.2
7 1 2 P M 28 0 3 0 7.84 1.2
8 1 2 P M 28 0 4 0 7.84 1.2
9 1 3 A M 23 1 1 1 5.29 1.3
10 1 3 A M 23 1 2 1 5.29 1.3
11 1 3 A M 23 1 3 1 5.29 1.3
12 1 3 A M 23 1 4 1 5.29 1.3
13 1 4 P M 44 1 1 1 19.36 1.4
14 1 4 P M 44 1 2 1 19.36 1.4
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
Figure: Mean outcome across visits for patients against age
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
A probable hierarchical model for binary response can model the logit of the probability of observing 1 in thepatient’s visit with a linear predictor.
yiv = Bin(1, piv ), independent for v = 1, . . . , 4
log(piv
1− piv
) = β0 + β1baseline(i) + β2center(i) + β3gender(i) + β4treat(i)
β5age(i) + β6(age(i)
10)2 + spatient(i)
spatient(i) ∼ N(0, σs2), independent for different i
i = 1, . . . 111 v = 1, . . . , 4
I spatient(i) ∼ N(0, σ2s ) = random deviation due to i th patient from mean response
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The Respiratory Illness Example - R Code and Output
mixedmodel.glm <- lmer(outcome ~ baseline + center + sex +treat +
age + age2 + (1 | patid), data = respiratory,family=binomial)
Generalized linear mixed model fit by the Laplace approximation
Formula: outcome ~ baseline + center + sex + treat + age + age2 + (1 | patid)
Data: respiratory
AIC BIC logLik deviance
440.9 473.7 -212.4 424.9
Random effects:
Groups Name Variance Std.Dev.
patid (Intercept) 3.1998 1.7888
Number of obs: 444, groups: patid, 111
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 5.53290 1.87130 2.957 0.00311 **
baseline 2.86340 0.49680 5.764 8.23e-09 ***
center2 0.75956 0.50042 1.518 0.12906
sexM -0.60686 0.64753 -0.937 0.34866
treatP -2.01317 0.48807 -4.125 3.71e-05 ***
age -0.30711 0.09476 -3.241 0.00119 **
age2 0.38901 0.13030 2.985 0.00283 **
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The Respiratory Illness Example - R Code and Output
Correlation of Fixed Effects:
(Intr) baseln centr2 sexM treatP age
baseline 0.041
center2 -0.167 -0.256
sexM -0.567 -0.145 0.190
treatP -0.250 -0.199 0.007 0.248
age -0.902 -0.078 0.059 0.258 0.086
age2 0.838 0.094 -0.101 -0.210 -0.092 -0.978
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
A Bayesian Model with Binary Response
A binary response was tracked on 300 individuals over 4 time points. Along with this response two body factorswere also measured at each of those time points. The researcher wants to know if the two body factors that weretracked over time affects the response. However, he also wants to take into consideration the random effects dueto the subjects in the study.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The key concept in Bayesian model is the likelihood function. The likelihood expresses our notion about we how”likely” the observed data is but itis a function of our parameters. Change the values of the parameters and the likelihood of the observed data changes.
It is obvious that a model with random effects due to the subjects should be used to model the data.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The Model
The binary variable will be modeled using probit regression i.e. if πit is the probability that the response for i th
individual at time t will be 1 then,
πit = φ(xTit β + zT
i θ)
The Bayesian model will use a latent factor to model the response. The latent factor structure maintains thelikelihood function as required by our model and also makes calculations simpler.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
The hierarchical structure in a Bayesian model also includes our prior belief about β, θ and ψ−1.
yit = 1, if γit > 0
= 0, if γit ≤ 0
γit ∼ N(xTit β + zT
i θ)
β ∼ MVN(β0,Σ0)
θ ∼ MVN(0, ψ−1I)
ψ ∼ Gamma(ε, ε)
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
Estimates obtained from the Gibb’s Sampler.
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
OutlineThe Mixed Effects Model Through Examples
The Mathematical ModelGeneralized Linear Mixed Models
The Respiratory Illness ExampleA Bayesian Hierarchical Model
References
http://cran.r-project.org/doc/contrib/Fox-Companion/appendix-mixed-models.pdfContemporary Statistical Models for the Plant and Soil Sciences by Oliver Schabenberger, Francis J. PierceSAS for mixed models by Ramon C. Littell
Dipayan Maiti Mixed Models and Hierarchical Models - A Short Overview
Top Related