Multilevel Models - Loginpsfaculty.ucdavis.edu/bsjjones/multilevelII.pdf · Today I Implementation...

Multilevel Models

Brad Jones1

1Department of Political ScienceUniversity of California, Davis

November 28, 2007

Jones Multilevel Modeling

Today

I Implementation and Interpretation

I Some extensions of the basic two-level model todiscrete-choice setting.


Summarizing Last Time

I Basic problems emerge because of clustering.

I Consider the simplest model again.

yi = β0j + εij . (1)

I Simple RANOVA Model

I The variable coefficient:

β0j = γ00 + u0j , (2)

I The reduced form model:

yi = γ00 + u0j + εij . (3)


Variance Components

I ∃ two sources of variation: level-1 and level-2

I The intraclass correlation:

ρ =σ2

u0

σ2u0

+ σ2ε

, (4)

I The usual assumptions:

εij ∼ N(0, σ2ε )

u0j ∼ N(0, σ2u0

),


Variance Components

I Var-Cov Matrix for some cluster:

A =

σ2

u0+ σ2

ε

σ2u0

σ2u0

+ σ2ε

σ2u0

σ2u0

σ2u0

+ σ2ε

......

. . .

σ2u0

. . . σ2u0

σ2u0

+ σ2ε .

(5)

I Implications?

I For some other cluster . . .

B =

σ2

u0+ σ2

ε

σ2u0

σ2u0

+ σ2ε

σ2u0

σ2u0

σ2u0

+ σ2ε

......

. . .

σ2u0

. . . σ2u0

σ2u0

+ σ2ε

. (6)


Variance Components

I If we had K level-2 units, the full covariance matrix would be:

V2 =

A0 B...

. . .. . .

0 . . . 0 K

(7)

I This is a block-diagonal matrix with 0s on the off-diagonal.


Variance Components

I Implications? This shows clearly how multilevel models handlenon-independent observations: there is non-zero covariancesfor level-1 observations nested within level-2 units.

I This covariance is assumed to be equal to the level-2 variance(i.e. σ2

u0).

I It is important to see that the level-2 variance term is also ameasure of intraclass covariance.

I Along the main diagonal, the variance in the model accountsfor both sources of variance.

I The zeroes in the matrix indicate that observations fromdifferent level-2 units are independent.


Variance Components

I Useful to consider the case when σ2u0

= 0.

I For level-2 unit A, the covariance matrix would be

A =

0 + σ2

ε

0 0 + σ2ε

0 0 0 + σ2ε

......

. . .

0 . . . 0 0 + σ2ε

. (8)

I The covariance structure would be similar for level-2 unit B.


Variance Components

I When there is no level-2 residual variance, the covariancematrix for the full data set reduces to

V =

σ2

ε

0 σ2ε

0 0 σ2ε

......

. . .

0 . . . 0 σ2ε

(9)

I This is the familiar variance-covariance matrix for thestandard linear model.

I Standard regression is therefore more restrictive.


Issues

I Simple model with two variable coefficients: slope, intercept.

β0j = γ00 + γ01Zj + u0j

β1j = γ10 + u1j . (10)

I The reduced-form model is given by

yi = (γ00 + γ01Zj + u0j) + (γ10 + u1j)Xij + εij .

I Rearranging, we obtain the “multilevel” model:

yi = γ00 + γ10Xij + γ01Zj + u1jXij + u0j + εij . (11)

I For this model, three variances and one covariance term areestimated.


Issues

I The level-2 variances are the σ2u.

and the level-1 variance is σ2ε .

I The covariance matrix for the random coefficients issummarized as

Σ =

(σ2

u0σu01

σu10 σ2u1

)(12)

I σu01 corresponds to the covariance between the randomintercept and slope in equation (??).

I Some authors convey matrix (??) as:

T =

(τ00 τ01

τ10 τ11

). (13)

I Both expressions are identical.


Issues

I The matrix in (??) is useful to help clarify what the level-2var-cov matrix actually looks like.

I The addition of random parameters adds more variances andcovariances to estimate.

I Suppose we estimated 1 random intercept and 2 randomslopes?

I The level-2 covariance matrix would have the form

Σ =

σ2u0

σu01 σu02

σu10 σ2u1

σu12

σu20 σu21 σ2u2

. (14)


Issues

I This is all obvious but what understand what you’re reallyasking of the data.

I Each time you add a random term to the level-2 model, you’rereally adding an additional variance term plus additionalcovariance terms (one for each k random term).

I Models with many random effects, apart from possiblyinducing computational problems, can quickly run intodegrees-of-freedom problems.

I The df problem here is a little different than the usualproblem of small n.


Issues

I Remember, with multilevel models, we are explicitly usingdata from both level-1 and level-2.

I Each time we ask a random level-2 coefficient to beestimated, we are requiring more and more information to beused from the set of level-2 observations.

I This is why even if you have many, many level-1 cases (i.e.“big n”) but very few level-2 cases, estimates of level-2random coefficients can become difficult.

I If we did an analysis of 4 U.S. cities and attempted toestimate a multilevel model with 2 random slopes and arandom intercept, we could not do it.

I Why? We need to estimate three variance and six covarianceparameters.

I That is, we are attempting to estimate seven random level-2parameters but we only have four unique level-2 units.

I It can’t be done . . . usually.


How do we do this?

I Variety of software packages.

I GLLAMM, Mixor/Mixreg, HLM, MLwin, R, . . . .

I All have tradeoffs.

I R is free.


lme4 and Linear Mixed Models

I Maintained by Douglas Bates

I Fairly easy to work with.

I Default estimation is RMLE for linear continuous model.


Dissecting R script from last week.

I Using DeBoef-Kellstedt data pedagogically.

I Minimal data requirements: y and x where x ∈ J.

I Repeated measures (time series) or cross-sectional nesting.

I Must be able to identify the higher-level unit (e.g. context)

I Model specification requires specifying fixed effects andrandom effects.

I Garden-variety regression only asks for fixed effects.



I RANOVA Model:

yi = γ00 + u0j + εij . (15)

I ranova<-lmer(indPCNC.BWNOMISS (1 | +monthyear))

I In lmer, random effects are in parentheses.

I The statement (1 | + monthyear) says: “let the constant(implied by a “1”) randomly vary over the level-2 units.” Thisis monthyear.


> ranova<-lmer(indPCNC.BWNOMISS ~ (1 | + monthyear))

>

> summary(ranova)

Linear mixed-effects model fit by REML Formula: indPCNC.BWNOMISS ~

(1 | monthyear)

AIC BIC logLik MLdeviance REMLdeviance

263386 263406 -131691 263376 263382

Random effects:

Groups Name Variance Std.Dev.

monthyear (Intercept) 0.047499 0.21794

Residual 0.365578 0.60463

number of obs: 143254, groups: monthyear, 226

Fixed effects:

Estimate Std. Error t value

(Intercept) 0.08573 0.01459 5.876



I The level-2 variance term is .047; the standard deviation is.218 (square root of the variance)

I The fixed effect is .09.

I Across “contextual units” the estimated standard deviation ofintercepts is .218.

I Useful to think what happens when this term approaches 0.

I Other information: we see we have 226 level-2 units and143,254 level-1 units.

I Data need not be balanced.


Random slope, random intercept

I Basic model:

yi = (γ00 + γ01zj + u0j) + (γ10 + u1j)xi + εi ,

I Rearranging terms gives the multilevel model:

yi = γ00 + γ10xi + γ01zj + u1jxi + u0j + εi . (16)

I The terms:γ00 corresponds to the intercept estimate.γ10 corresponds to the slope coefficient for xi .γ01 corresponds to the slope coefficient for zj .u1j corresponds to the disturbance term for the randomlyvarying slope coefficient γ10.u0j corresponds to the disturbance term for the randomintercept term.εi corresponds to the level-1 disturbance term.


Random slope, random intercept

I R code

I m2 <- lmer(indPCNC.BWNOMISS bottom.fourth +meancent.bad + + meancent.good + (1+bottom.fourth|monthyear))

I I’m saving the model in object m2.

I Random part given by (1+ bottom.fourth|monthyear)

I Says: “let the intercept (“1”) and the slope coefficient forbottom.fourth randomly vary across monthyear.”

I Again, monthyear is the contextual unit.


> m2 <- lmer(indPCNC.BWNOMISS ~ bottom.fourth + meancent.bad +

+ meancent.good + (1+ bottom.fourth|monthyear))

> summary(m2)


bottom.fourth + meancent.bad + meancent.good + (1 +

bottom.fourth | monthyear)


203632 203700 -101809 203577 203618

Random effects:

Groups Name Variance Std.Dev. Corr


bottom.fourth 0.007647 0.087447 -0.539

Residual 0.366571 0.605451


Fixed effects:


(Intercept) 0.1720628 0.0129923 13.243

bottom.fourth -0.2307735 0.0081826 -28.203

meancent.bad -0.0032392 0.0007307 -4.433

meancent.good 0.0002852 0.0005902 0.483



I The level-2 standard deviation for the intercept is .178; for theslope, it is .087; the correlation between the two is −.539(recall var-cov matrix from above).

I The fixed effects are .17 (intercept); −.23 (slope); plus thetwo contextual variables.


I The standard deviation of monthyear-slopes is .087.

I Again, it’s useful to think what happens when these termsapproaches 0.

I Note, group sizes drop (important to be aware of this whencomparing models).


Cross-level interactions

I We might posit these factors condition the relationship of xi

on yi .

I Leads to the consideration of a random coefficients modelwith cross-level interactions.

I Consider the simple model again:

yi = βj[i ] + β1j[i ]xi1 + εij .

I We condition xi1 on zj and rewrite the model to account forvariable parameters:

βj[i ] = γ00 + γ01zj + u0j

β1j[i ] = γ10 + γ11zj + u1j . (17)

I The second equation from (??) gives us the change in β1j[i ] asa function of covariate zj .


Cross-level interactions

I The reduced-form model is given by

yi = (γ00 + γ01zj + u0j) + (γ10 + γ11zj + u1j)xi + εi

I Rearranging terms give

yi = γ00 + γ10xi + γ01zj + γ11zjxi + u1jxi + u0j + εi . (18)

I γ00 is the intercept estimate.γ10 is the slope coefficient for the relationship between xi andyi when zj = 0.γ01 is the slope coefficient for the relationship between zj andyi when xi = 0.γ11 is the interaction between xi and zj .u1j is the disturbance term for the randomly varying slopecoefficient γ10.u0j is the disturbance term for the random intercept term.εi is the level-1 disturbance term.


Random slope, random intercept, cross-level interactions

I R code

I m3 <- lmer(indPCNC.BWNOMISS bottom.fourth +meancent.bad + + meancent.good +bottom.fourth:meancent.bad +bottom.fourth:meancent.good + (1+bottom.fourth|monthyear) )

I I’m saving the model in object m3.

I Random part given by (1+ bottom.fourth|monthyear)

I Says: “let the intercept (“1”) and the slope coefficient forbottom.fourth randomly vary across monthyear.”

I Cross-level interaction given by:bottom.fourth:meancent.bad +bottom.fourth:meancent.good


> m3 <- lmer(indPCNC.BWNOMISS ~ bottom.fourth + meancent.bad +

+ meancent.good + bottom.fourth:meancent.bad +

bottom.fourth:meancent.good + (1+ bottom.fourth|monthyear) + )

>

> summary(m3)


bottom.fourth + meancent.bad + meancent.good +

bottom.fourth:meancent.bad + bottom.fourth:meancent.good + (1 +

bottom.fourth | monthyear)


203660 203746 -101821 203572 203642

Random effects:

Groups Name Variance Std.Dev. Corr


bottom.fourth 0.0075197 0.086716 -0.536

Residual 0.3665666 0.605447


Fixed effects:


(Intercept) 0.1720295 0.0129756 13.258

bottom.fourth -0.2311378 0.0081415 -28.390

meancent.bad -0.0029870 0.0008267 -3.613

meancent.good -0.0002516 0.0006636 -0.379

bottom.fourth:meancent.bad -0.0003687 0.0004997 -0.738

bottom.fourth:meancent.good 0.0007401 0.0004112 1.800



I The level-2 standard deviation for the intercept is .176; for theslope, it is .087; the correlation between the two is −.536(recall var-cov matrix from above).

I The fixed effects are .17 (intercept); −.23 (slope); plus thetwo contextual variables.


I The standard deviation of monthyear-slopes is .087.

I Note that this model fits worse than previous model (AIC/BICshow this).

I May want to consider an alternative model.

I Eliminating bottom.fourth:meancent.bad produces “bestmodel.”


Context-Specific Estimates

I Context-specific estimates:

βj[i ] = ωjµβ + (1− ωj)y j , (19)

I Where

ωj = 1−σ2

β

σ2β + σ2

y/nj.

I Gelman and Hill (2007) refer to this expression as a “poolingfactor.”

I With “no pooling,” βj[i ] will equal the group mean; whenthere is complete pooling, the group-level coefficient isindistinguishable from the grand mean.

I That is, when ωj = 0, the “group” effect “shrinks” to thegrand mean.


Extracting these Estimates

I Our model:m3 <- lmer(indPCNC.BWNOMISS bottom.fourth +meancent.bad + meancent.good +bottom.fourth:meancent.bad +bottom.fourth:meancent.good + (1+bottom.fourth|monthyear) + (0+ bottom.fourth |meancent.good) + (0+ bottom.fourth |meancent.bad) )

I Three statements:coef(m2)fixef(m2)ranef(m2)

I The first recovers the context-specific slope and intercepts.

I The second recovers the fixed effects.

I The third gives the deviations off of the the fixed effects.


> coef(m3)

An object of class coef.lmer

[[1]]

(Intercept) bottom.fourth meancent.bad meancent.good . .

198101 0.038629302 -0.13100081 -0.002986706 -0.000251532

198102 -0.002688956 -0.17704378 -0.002986706 -0.000251532

198103 -0.069757109 -0.17780747 -0.002986706 -0.000251532

.

.

.

.

199608 0.395587218 -0.28091675 -0.002986706 -0.000251532

199609 0.322903090 -0.24719050 -0.002986706 -0.000251532

199610 0.339982550 -0.22567410 -0.002986706 -0.000251532

199611 0.307877625 -0.23153489 -0.002986706 -0.000251532

199612 0.344746186 -0.24538340 -0.002986706 -0.000251532


> fixef(m3)

(Intercept) bottom.fourth

0.1720212677 -0.2315523110

meancent.bad meancent.good

-0.0029867059 -0.0002515320

bottom.fourth:meancent.bad bottom.fourth:meancent.good

-0.0003610276 0.0007066620

(Intercept) bottom.fourth

198101 -0.1333919659 1.005515e-01

198102 -0.1747102233 5.450853e-02

198103 -0.2417783764 5.374485e-02

.

.

.

199608 0.2235659507 -4.936444e-02

199609 0.1508818218 -1.563819e-02

199610 0.1679612827 5.878210e-03

199611 0.1358563577 1.742285e-05

199612 0.1727249179 -1.383108e-02


Extracting these Estimates

I Fully interpreting the model would entail possibleconsideration of these effects.

I Plots a la Gelman and Hill are useful here.

I (I can’t quite get them to work!)


Multilevel Models - Loginpsfaculty.ucdavis.edu/bsjjones/multilevelII.pdf · Today I Implementation...

Documents

Transcript of Multilevel Models - Loginpsfaculty.ucdavis.edu/bsjjones/multilevelII.pdf · Today I Implementation...