Aqm 2008 Lecture Random
Transcript of Aqm 2008 Lecture Random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 1/121
Fixed and Random Effects
Jos Elkink
April, 2008
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 2/121
1 Introduction
2 Motivation
3 Fixed effects
4 Random effects
5
Random coefficients
6 Further information
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 3/121
Outline
1 Introduction
2 Motivation
3 Fixed effects
4 Random effects
5 Random coefficients
6 Further information
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 4/121
Four topics
Missing data March 27
Fixed & random effects April 3Time-series models April 10
Causation and inference April 17
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 5/121
Outline
1 Introduction
2 Motivation
3 Fixed effects
4 Random effects
5 Random coefficients
6 Further information
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 6/121
Motivations
Clustered sampling
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 7/121
Sampling strategies
Probability sampling:
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 8/121
Sampling strategies
Probability sampling:
Simple random sampling
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 9/121
Simple random sampling
The sampling here is a purely random
selection from the sampling frame, selected
without replacement.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 10/121
Simple random sampling
The sampling here is a purely random
selection from the sampling frame, selected
without replacement.Each subject from a population has the exact
same chance of being selected in the sample.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 11/121
Simple random sampling
The sampling here is a purely random
selection from the sampling frame, selected
without replacement.Each subject from a population has the exact
same chance of being selected in the sample.
The sample probability for each subject is the
same.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 12/121
Sampling strategies
Probability sampling:
Simple random samplingSystematic random sampling
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 13/121
Sampling strategies
Probability sampling:
Simple random samplingSystematic random sampling
Stratified sampling
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 14/121
Sampling strategies
Probability sampling:
Simple random samplingSystematic random sampling
Stratified sampling
Cluster sampling
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 15/121
Cluster sampling
To reduce costs, clusters are (randomly)sampled first, before lower levels are
clustered.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 16/121
Cluster sampling
To reduce costs, clusters are (randomly)sampled first, before lower levels are
clustered.
E.g. selecting schools before selecting
students, so that fewer schools need to be
visited.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 17/121
Cluster sampling
To reduce costs, clusters are (randomly)sampled first, before lower levels are
clustered.
E.g. selecting schools before selecting
students, so that fewer schools need to be
visited.Individual observations from a clustered
sample are not independent .
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 18/121
Motivations
Clustered sampling
Inherent structure
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 19/121
Examples
schools teachers
classes pupils
firms employees
countries political parties
doctors patients
subjects measurementsinterviewers respondents
judges suspects
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 20/121
Motivations
Clustered sampling
Inherent structure
Panel data
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 21/121
Motivations
Clustered sampling
Inherent structure
Panel data
Time-Series Cross-Section
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 22/121
Multilevel characteristics
Observations are not completely
independent
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 23/121
Multilevel characteristics
Observations are not completely
independent
Variance can be divided inbetween-group and within-group
variances
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 24/121
Multilevel characteristics
Observations are not completely
independent
Variance can be divided inbetween-group and within-group
variances
Variables can be measured at eithermicro- or marco-level, or both
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 25/121
Example
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 26/121
Overall mean
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 27/121
Group means
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 28/121
Between variation
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
s ope = − .
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 29/121
Group means
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 30/121
Within variation
−4 −2 0 2 4 6 8
− 1 .
5
− 1 . 0
− 0 .
5
0 .
0
0
. 5
1 .
0
x
y .
d e v
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 31/121
Outline1
Introduction2 Motivation
3 Fixed effects
4 Random effects
5 Random coefficients
6 Further information
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 32/121
Pooled model
When we simply run a regression using all
micro-level data, ignoring the multilevel
structure, we call this a pooled model.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 33/121
Pooled model
If we have some observations at the
macro-level, we are artificially increasing the
number of observations.
P l d d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 34/121
Pooled model
If we have some observations at the
macro-level, we are artificially increasing the
number of observations. Thus we will beoverconfident in our results.
P l d d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 35/121
Pooled model
If we have some observations at the
macro-level, we are artificially increasing the
number of observations. Thus we will beoverconfident in our results.
E.g. characteristics of judges in explainingthe severity of court rulings.
P l d d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 36/121
Pooled model
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
s ope = .
Fi d ff d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 37/121
Fixed effects model
With a fixed effects model we explain thewithin-group variation, removing the
between-group variation by:
Fi d ff d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 38/121
Fixed effects model
With a fixed effects model we explain thewithin-group variation, removing the
between-group variation by:
Adding dummy variables for each group
Fi d ff t d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 39/121
Fixed effects model
With a fixed effects model we explain thewithin-group variation, removing the
between-group variation by:
Adding dummy variables for each group
Subtracting the group means from all
variables
Fi d ff t d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 40/121
Fixed effects model
With a fixed effects model we explain thewithin-group variation, removing the
between-group variation by:
Adding dummy variables for each group
Subtracting the group means from all
variables
The two are equivalent.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 41/121
Fi d ff ts d l
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 42/121
Fixed effects model
In essence, we thus have different interceptsfor each group.
y i = β 0 + X i β + µ j [i ] + εi ,
whereby i denotes the individual unit, j the
group, and j [i ] the group of i .
Fixed effects model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 43/121
Fixed effects model
In essence, we thus have different interceptsfor each group.
y i = β 0 + X i β + µ j [i ] + εi ,
whereby i denotes the individual unit, j the
group, and j [i ] the group of i .
If the fixed effects model is the true model,
pooled estimates are biased and inconsistent.
Pooled model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 44/121
Pooled model
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
s ope = .
Fixed effects model (1)
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 45/121
Fixed effects model (1)
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
s ope = .
Fixed effects model (2)
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 46/121
Fixed effects model (2)
−6 −4 −2 0 2 4
− 1 .
5
− 1 .
0
− 0 .
5
0 .
0
0 .
5
1 .
0
x.dev
y .
d e v
s ope = .
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 47/121
Between effects model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 48/121
Between effects model
Another way of dealing with clustered data islooking at the between model:
¯ y j = β 0 + X j β + ε j
Typical mistake: conclusions aboutindividuals from aggregate data - ecological
fallacy.
Between effects model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 49/121
Between effects model
−4 −2 0 2 4 6 8
1
2
3
4
5
6
x
y
s ope = − .
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 50/121
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 51/121
Fixed effects in Stata
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 52/121
Fixed effects in Stata
xtreg grade aptitude age, i(school) fe
Fixed effects in Stata
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 53/121
Fixed effects in Stata
xtreg grade aptitude age, i(school) fe
Or, “manually”:
xi: reg grade aptitude age i.school
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 54/121
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 55/121
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 56/121
Group-level variables
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 57/121
Group level variables
Note that fixed effects models cannot deal
with group-level variables.
Group-level variables
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 58/121
Group level variables
Note that fixed effects models cannot deal
with group-level variables.
The effect would be perfect multicollinearity .
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 59/121
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 60/121
Group-level variables
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 61/121
p
In most cases with group-level variables,
however, a random effects or randomintercept model is more appropriate.
Outline
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 62/121
1 Introduction
2 Motivation
3
Fixed effects4 Random effects
5 Random coefficients
6 Further information
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 63/121
Random effects
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 64/121
For the random effects model we still have:
y i = β 0 + X i β + µ j [i ] + εi .
However, this time we assume µ j ∼ N (0, σ2µ).
By assuming that µ j comes from a normal
distribution, we have fewer parameters to
estimate (only one σ2µ instead of J µ’s).
Variance components
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 65/121
p
In the population, the variance of the
dependent variable can be split in
within-group and between-group variance:
σ
2
Y = σ
2
between + σ
2
within
Intraclass correlation
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 66/121
Aside: the proportion of the variance that is
accounted for by the group level is the
intraclass correlation.
ρintra =
σ2between
σ2between + σ2within
Variance estimators
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 67/121
σ2within = s 2within
Variance estimators
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 68/121
σ2within = s 2within
σ2between = s 2between −
s 2withinn ,
where
n = n −s 2n j
N n
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 69/121
When to use random effects?
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 70/121
When to use random effects?
A group effect is random if we can think
of the levels we observe in that group tobe samples from a larger population.
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 71/121
When to use random effects?
A group effect is random if we can think
of the levels we observe in that group tobe samples from a larger population.
When making out-of-sample inferences.
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 72/121
When to use random effects?
A group effect is random if we can think
of the levels we observe in that group tobe samples from a larger population.
When making out-of-sample inferences.
When there are group-level variables.
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 73/121
When to use random effects?
A group effect is random if we can think
of the levels we observe in that group tobe samples from a larger population.
When making out-of-sample inferences.
When there are group-level variables.When the sizes of groups are small.
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 74/121
When to use random effects?
Alternatively, one can primarily look at n j
and N :N small fixed effects
N not small, n j small random effects
n j larger not as importantBut this is only a preliminary quick judgment!
Fixed vs random
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 75/121
When to use random effects?
Gelman & Hill (2007): “Our advice (...) is toalways use multilevel modeling (’random
effects’).”
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 76/121
Random effects in R
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 77/121
library(arm)lmer(grade ~ aptitude + age + (1|school))
Random effects in Stata
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 78/121
xtreg grade aptitude age, i(school) re
xtreg grade aptitude age, i(school) re mle
School example
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 79/121
Note that we are talking about 3 schools -
this is too few groups to seriously consider arandom effects model!
School example
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 80/121
Linear mixed-effects model fit by REML
Random effects:
Groups Name Variance Std.Dev.
school (Intercept) 1.737 1.318
Residual 0.293 0.542number of obs: 60, groups: school, 3
Fixed effects:
Estimate Std. Error t value(Intercept) 3.0259 1.1360 2.66
aptitude 0.9216 0.0723 12.75
age 0.2020 0.0675 2.99
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 81/121
School example (fixed)
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 82/121
−2 −1 0 1 2
2
3
4
5
6
7
8
9
aptitude
g r a d e
School example (random)
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 83/121
−2 −1 0 1 2
2
3
4
5
6
7
8
9
aptitude
g r a d e
School example
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 84/121
Random-effects ML regression Number of obs = 60Group variable (i): school Number of groups = 3
Random effects u_i ~ Gaussian Obs per group: min = 20avg = 20.0
max = 20
LR chi2(2) = 83.55Log likelihood = -53.896431 Prob > chi2 = 0.0000
------------------------------------------------------------------------------grade | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------aptitude | .9210743 .0710091 12.97 0.000 .781899 1.06025
age | .2022943 .0662681 3.05 0.002 .0724112 .3321775_cons | 3.021863 1.034865 2.92 0.003 .993565 5.050161
-------------+----------------------------------------------------------------/sigma_u | 1.073727 .4438302 .4775795 2.414027/sigma_e | .5320764 .0498341 .4428439 .639289
rho | .8028508 .1342531 .4616776 .9640621------------------------------------------------------------------------------Likelihood-ratio test of sigma_u=0: chibar2(01)= 83.34 Prob>=chibar2 = 0.000
R-squared
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 85/121
In linear regression, a popular statistics is R 2
,which is the squared multiple correlation
coefficient
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 86/121
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 87/121
R-squared
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 88/121
Remember, variance of a multilevel model
has different components:
σ2Y = σ2
between + σ2within
R-squared: individuallevel
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 89/121
level
Estimated two models, one with and one
without explanatory variables (A and B ,
respectively).
Then,
R 2within = σ2
µ,A + σ2
ε,A
σ2µ,B + σ2
ε,B
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 90/121
Predicted random effects
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 91/121
With a fixed effects model, we have the
coefficients on the group dummies which we
can interpret as group-level predictors.
Predicted random effects
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 92/121
With a fixed effects model, we have the
coefficients on the group dummies which we
can interpret as group-level predictors.In a random effects model, we do not have
these predictions, as we only estimated σ2µ
and β 0.
Predicted random effects
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 93/121
The predicted group levels can be estimatedusing:
β 0, j = λ j ¯ y j + (1 − λ j )β 0
λ j =σ2µ
σ2µ + σ2
ε/n j ,
whereby ¯ y j is the mean on y of group j .
Predicted random effects
I R h i d fi d ff i h
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 94/121
In R, you get the estimated fixed effects with:
model.random <- lmer(y ~ x1 + x2 + (1|
fixef(model.random)
and the predicted random effects with:
ranef(model.random)
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 95/121
Random coefficients
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 96/121
In the random effects model, we assume thatgroup intercepts vary according to a normal
distribution.
Random coefficients
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 97/121
In the random effects model, we assume thatgroup intercepts vary according to a normal
distribution.
But what about the coefficients?
Random coefficients
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 98/121
In the random effects model, we assume thatgroup intercepts vary according to a normal
distribution.
But what about the coefficients?
I.e. what about group slopes that vary
following a normal distribution?
Random coefficients
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 99/121
y i = β 0 + X i β + X i γ j [i ] + µ j [i ] + εi
µ j ∼ N (0, σ2µ)
γ j ∼ N (0, σ2γ )
Random coefficients
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 100/121
y i = β 0 + X i β + X i γ j [i ] + µ j [i ] + εi
µ j ∼ N (0, σ2µ)
γ j ∼ N (0, σ2γ )
Note that a model with random coefficients,
but a constant intercept across groups rarely
makes sense, especially because of the often
arbitrary location if x = 0.
Random effects in R
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 101/121
library(arm)
lmer(grade ~ aptitude + age + (aptitude|school))
School example
Linear mixed-effects model fit by REML
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 102/121
Linear mixed effects model fit by REML
Random effects:
Groups Name Variance Std.Dev. Corr
school (Intercept) 1.74e+00 1.32e+00
aptitude 1.47e-10 1.21e-05 0.000
Residual 2.93e-01 5.42e-01
number of obs: 60, groups: school, 3
Fixed effects:
Estimate Std. Error t value(Intercept) 3.0259 1.1359 2.66
aptitude 0.9216 0.0723 12.75
age 0.2020 0.0675 2.99
School example (random)
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 103/121
−2 −1 0 1 2
2
3
4
5
6
7
8
9
aptitude
g r a d e
Example
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 104/121
−5 0 5 10
− 5
0
5
1 0
x
y
Pooled model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 105/121
−5 0 5 10
− 5
0
5
1 0
x
y
s ope = .
Fixed effects model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 106/121
−5 0 5 10
− 5
0
5
1 0
x
y
s ope = .
Random effects model
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 107/121
−5 0 5 10
− 5
0
5
1 0
x
y
s ope = .
Random coefficientsmodel
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 108/121
− 5
0
5
1 0
y
mean s ope = .
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 109/121
Outline1 Introduction
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 110/121
2 Motivation
3 Fixed effects
4 Random effects
5
Random coefficients6 Further information
Important other topics
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 111/121
Time-dependence within groups (nextweek)
Important other topics
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 112/121
Time-dependence within groups (nextweek)
Predictors on the random coefficients
Important other topics
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 113/121
Time-dependence within groups (nextweek)
Predictors on the random coefficients
Bayesian estimation
Important other topics
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 114/121
Time-dependence within groups (nextweek)
Predictors on the random coefficients
Bayesian estimationMore complex models dealing with panel
data structures
Important other topics
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 115/121
Time-dependence within groups (nextweek)
Predictors on the random coefficients
Bayesian estimationMore complex models dealing with panel
data structures
Extensions towards limited dependent
variables
Further information
A clear, relatively introductory textbook on
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 116/121
, y y
multilevel modeling is Snijders & Bosker
(1999), Multilevel analysis. An introduction
to basic and advanced multilevel modeling .
Further information
A clear, relatively introductory textbook on
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 117/121
, y y
multilevel modeling is Snijders & Bosker
(1999), Multilevel analysis. An introduction
to basic and advanced multilevel modeling .An excellent, modern book on multilevel
modeling, using primarily R and Bugs, is
Gelman & Hill (2007), Data analysis using regression and multilevel/hierarchical models .
Further information
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 118/121
Their websites are also interesting:
Snijders: http://stat.gamma.rug.nl/snijders/
Gelman: http://www.stat.columbia.edu/ gelman/
Further information
When using Stata, the
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 119/121
g
Longitudinal/panel-data reference manual is
of very high quality. The relevant chapters
for this lecture are in fact freely available assample chapters (xtreg and xtmixed) at
http://www.stata.com/bookstore/xt.html.
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 120/121
Further information
Two standard textbooks on panel data are
7/23/2019 Aqm 2008 Lecture Random
http://slidepdf.com/reader/full/aqm-2008-lecture-random 121/121
Baltagi (2005), Econometric analysis of
panel data (primarily for small N , large T )
and Hsiao (2003), Analysis of panel data
(primarily for large N , small T ). Both are
very technical in nature. Perhaps an easier
introduction is Wooldridge (2002),
Econometric analysis of cross-section and
panel-data.