7. Autocorrelation(Violation of Assumption #B3)
Assumption #B3:
• The error term ui is not autocorrelated, i.e.
Cov(ui, uj) = 0
for all i = 1, . . . , N and j = 1, . . . , N where i 6= j
Where do we typically find autocorrelation:
• Time-series data sets(less frequently in cross-sectional data sets)
169
Outlook:
• Strong similarities between heteroskedasticity and autocor-relation with respect to consequences and estimation proce-dures (GLS, FGLS estimators)
Example: [I]
• Estimation of a price-revenue function (monthly data)
• Variables:
yi = monthly revenue quantity (in 1000 pieces)
xi = selling price (in euros)
170
Month Obs. Price Revenue Month Obs. Price Revenue 01:2002 1 24.2 1590 01:2003 13 32.2 1700 02:2002 2 25.5 1630 02:2003 14 32.4 1450 03:2002 3 26.8 1570 03:2003 15 33.2 1450 04:2002 4 26.4 1960 04:2003 16 34.0 1450 05:2002 5 25.2 2150 05:2003 17 33.7 1000 06:2002 6 24.4 2450 06:2003 18 32.8 1080 07:2002 7 26.2 2770 07:2003 19 31.3 1270 08:2002 8 26.1 2400 08:2003 20 30.9 1520 09:2002 9 27.4 2200 09:2003 21 30.0 1820 10:2002 10 28.4 1270 10:2003 22 28.3 1660 11:2002 11 29.8 1250 11:2003 23 27.5 1500 12:2002 12 31.3 1500 12:2003 24 26.8 1410
Dependent Variable: REVENUE Method: Least Squares Date: 11/17/04 Time: 13:50 Sample: 2002:01 2003:12 Included observations: 24
Variable Coefficient Std. Error t-Statistic Prob. C 4262.118 686.0876 6.212207 0.0000
PRICE -89.58094 23.56704 -3.801111 0.0010R-squared 0.396408 Mean dependent var 1668.750 Adjusted R-squared 0.368972 S.D. dependent var 445.9802 S.E. of regression 354.2746 Akaike info criterion 14.65768 Sum squared resid 2761231. Schwarz criterion 14.75585 Log likelihood -173.8921 F-statistic 14.44845 Durbin-Watson stat 0.760994 Prob(F-statistic) 0.000978
Example: [II]
• Linear model with a single regressor:
yi = α + β · xi + ui (i = 1, . . . ,24)
• Price increase of 1 euro decreases the monthly revenue by90000 pieces
172
500
1000
1500
2000
2500
3000
24 26 28 30 32 34 36
P r i c e
Revenu
e
Evidently:
• Line connecting the residuals rarely crosses the regressionline
−→ we frequently find that
positive ui−1-values are followed by positive ui-values
negative ui−1-values are followed by negative ui-values
−→ Cov(ui−1, ui) 6= 0
(violation of Assumption #B3)
Question:
• Impact on estimation and testing procedures
174
7.1 Consequences
Note:
• We assume an explicit pattern of autocorrelation(alternative patterns are not considered here)
Definition 7.1: (AR(1) process)
Let u1, . . . , uN be the error terms of the linear regression model.Furthermore, let ρ ∈ R be a (constant) parameter and let e1, . . . ,eN denote additional error terms that satisfy all B-assumptions(#B1–#B4). If
ui = ρui−1 + ei, (i = 2, . . . , N)
we say that the error term ui follows a first-order autoregressiveprocess (in symbols: ui ∼ AR(1)).
175
Remarks:
• An AR(1) process regresses ui on its predecessor value ui−1plus the new random shock ei
• For ρ = −1 or ρ = 1 we have so-called random walks(important stochastic processes)
• For |ρ| > 1 processes become explosive
−→ in this lecture: −1 < ρ < 1
Now:
• Expected values, (co)variances, correlation coefficients of anAR(1) process
176
Theorem 7.2: (Moments of an AR(1) process)
Let the error term ui (i = 1, . . . , N) follow an AR(1) processaccording to Definition 7.1 where −1 < ρ < 1. Furthermore, letVar(ei) ≡ σ2
e denote the constant variance of all ei. We thenhave for all (admissible) i = 1, . . . , N :
E(ui) = 0,
Var(ui) =σ2
e1− ρ2 ≡ σ2,
Cov(ui, ui−τ) = ρτ σ2e
1− ρ2 = ρτσ2 6= 0,
Corr(ui, ui−τ) = ρτ .
(Proof: class)
177
Obviously:
• If the error term ui follows an AR(1) process with −1 < ρ < 1,then the Assumptions #B1, #B2 are satisfied whereas #B3is violated
Now:
• Autocorrelation in matrix notation(ui’s follow an AR(1) process)
Notation:
u =
u1u2...
uN
, u−1 =
u0u1...
uN−1
, e =
e1e2...
eN
178
Matrix representation: [I]
• Linear regression model
y = Xβ + u
with AR(1) error terms (−1 < ρ < 1)
u = ρu−1 + e
−→ Theorem 7.2 yields Cov(u)
179
Matrix representation: [II]
• Due to σ2 ≡ σ2e /(1− ρ2) we obtain
Cov(u) =
σ2 Cov(u1, u2) · · · Cov(u1, uN)Cov(u2, u1) σ2 · · · Cov(u2, uN)
... ... · · · ...Cov(uN , u1) Cov(uN , u2) · · · σ2
=
σ2 ρσ2 · · · ρN−1σ2
ρσ2 σ2 · · · ρN−2σ2... ... · · · ...
ρN−1σ2 ρN−2σ2 · · · σ2
= σ2Ω
180
Matrix representation: [II]
where
Ω =
1 ρ · · · ρN−1
ρ 1 · · · ρN−2... ... · · · ...
ρN−1 ρN−2 · · · 1
181
Question:
• Is there any transformation of the autocorrelated model sothat
the parameter vector β remains unchanged
autocorrelation vanishes
the transformed model
y∗ = X∗β + u∗
satisfies all #A-, #B-, #C- assumptions?
(cf. Section 6, Slide 111)
182
Hope:
• If yes, then the OLS estimator of the transformed model (theGLS estimator) would be BLUE(cf. Section 6, Slides 123–126)
Result:
• In analogy to the line of argument given on Slides 117–121under heteroskedasticity the following result obtains:
there exists a regular matrix P so that the transformedmodel
Py = PXβ + Pu
satisfies all #A-, #B-, #C-assumptions
183
Form of P in the autocorrelated model: [I]
• P has to satisfy the following equations:
P′P = Ω−1 and PΩP′ = IN(see Slides 117, 120)
• First, the inverse of Ω from Slide 181 is given by
Ω−1 =1
1− ρ2
1 −ρ 0 · · · 0 0−ρ 1 + ρ2 −ρ · · · 0 00 −ρ 1 + ρ2 · · · 0 0... ... ... ... ... ...0 0 0 ... 1 + ρ2 −ρ0 0 0 ... −ρ 1
(check it)
184
Form of P in the autocorrelated model: [II]
• The form of P is given by
P =1
√
1− ρ2
√
1− ρ2 0 0 · · · 0 0−ρ 1 0 · · · 0 00 −ρ 1 · · · 0 0... ... ... ... ... ...0 0 0 · · · 1 00 0 0 · · · −ρ 1
185
Form of P in the autocorrelated model: [III]
−→ transformed model:
y∗ = X∗β + u∗
where
y∗ = Py =
√
1− ρ2y1y2 − ρy1
...yN − ρyN−1
, u∗ = Pu =
√
1− ρ2u1e2...
eN
X∗ = PX =
√
1− ρ2√
1− ρ2x11 · · ·√
1− ρ2xK11− ρ x12 − ρx11 · · · xK2 − ρxK1... ... ... ...1− ρ x1N − ρx1(N−1) · · · xKN − ρxK(N−1)
186
Remarks:
• The transformed model
y∗ = X∗β + u∗
satisfies all #A-, #B-, #C-assumptions
• The parameter vector β remains unchanged
−→ consequences of autocorrelation parallel those of hetero-skedasticity
187
Consequences of autocorrelation: [I]
• The OLS estimator
β =(
X′X)−1
X′y
is still unbiased, but no longer BLUE(cf. Theorem 6.1, Slide 109)
• The covariance matrix of the OLS estimator is given by
Cov(
β)
= σ2(
X′X)−1
X′ΩX(
X′X)−1
• The GLS estimator
βGLS
=[
X∗′X∗]−1
X∗′y∗ =[
X′Ω−1X]−1
X′Ω−1y
is BLUE
188
Consequences of autocorrelation: [II]
• Its covariance matrix is given by
Cov(
βGLS
)
= σ2[
X′Ω−1X]−1
(cf. Theorem 6.3, Slide 123)
• Unbiased estimator of σ2:
σ2 =u∗′u∗
N −K − 1=
(Pu)′PuN −K − 1
189
Impact of neglecting autocorrelation: [I]
• OLS estimator of β
β =(
X′X)−1
X′y
is unbiased, but inefficient
• The estimator σ2(X′X)−1 of the covariance matrix Cov(
β)
is biased
• The estimator
σ2 =u′u
N −K − 1of the error-term variance is biased
190
Impact of neglecting autocorrelation: [II]
−→ test statistics are based on biased estimators
−→ hypothesis tests are likely to be unreliable(t-, F -tests)
191
7.2 Diagnostics
Graphical analysis: [I]
• First, estimation of the model
y = Xβ + u
by OLS, i.e.
β =(
X′X)−1
X′y
−→ calculation of the residuals
u = y−Xβ
192
Graphical analysis: [II]
• Plot of the residuals versus time
slow swings around zero −→ positive autocorrelationfast swings around zero −→ negative autocorrelation
• Scatterplot of ui−1 versus ui
positive slope −→ positive autocorrelationnegative slope −→ negative autocorrelation
Example: [I]
• Price-revenue function on Slides 170–172
193
Dependent Variable: Revenue Method: Least Squares Date: 11/17/04 Time: 13:50 Sample: 2002:01 2003:12 Included observations: 24
Variable Coefficient Std. Error t-Statistic Prob. C 4262.118 686.0876 6.212207 0.0000
PRICE -89.58094 23.56704 -3.801111 0.0010R-squared 0.396408 Mean dependent var 1668.750 Adjusted R-squared 0.368972 S.D. dependent var 445.9802 S.E. of regression 354.2746 Akaike info criterion 14.65768 Sum squared resid 2761231. Schwarz criterion 14.75585 Log likelihood -173.8921 F-statistic 14.44845 Durbin-Watson stat 0.760994 Prob(F-statistic) 0.000978
OBS RESID(-1) RESID 1 -504.2594 2 -504.2594 -347.8042 3 -347.8042 -291.3490 4 -291.3490 62.81861 5 62.81861 145.3215 6 145.3215 373.6567 7 373.6567 854.9024 8 854.9024 475.9443 9 475.9443 392.3995
10 392.3995 -448.0195 11 -448.0195 -342.6062 12 -342.6062 41.76520 13 41.76520 322.3880 14 322.3880 90.30423 15 90.30423 161.9690 16 161.9690 233.6337 17 233.6337 -243.2406 18 -243.2406 -243.8634 19 -243.8634 -188.2348 20 -188.2348 25.93283 21 25.93283 245.3100 22 245.3100 -66.97761 23 -66.97761 -298.6424 24 -298.6424 -451.3490
-600
-400
-200
0
200
400
600
800
1000
02:01 02:04 02:07 02:10 03:01 03:04 03:07 03:10
i = 1, . . ., 24
Res
iduals
1000
-1000
-500
0
500
-1000 -500 0 500 1000
iu∧
1−∧
iu
Obviously:
• Positive dependence between ui−1 and ui
−→ indication of positive autocorrelation
−→ a conceivable specification of an AR(1) error-term processcould be
ui = ρui−1 + ei
with 0 < ρ < 1
196
Now:
• Use the pair of residuals (ui−1, ui) to estimate ρ
• Model
ui = ρui−1 + e∗i (i = 2, . . . , N)
• OLS estimator of ρ:
ρ =
N∑
i=2
(
ui−1 − u) (
ui − u)
N∑
i=2
(
ui−1 − u)2
=
N∑
i=2ui−1ui
N∑
i=2u2
i−1
197
Remarks:
• For the sum of the residuals computed via OLS we alwayshave
N∑
i=1ui = 0 and thus u =
1N
N∑
i=1ui = 0
(cf. Von Auer, 2007, p. 57)
• Since for i = 1 there is no residual ui−1 = u0, we only havei = 2, . . . , N observations
198
OLS estimate of ρ in the price-revenue example:
ρ = 0.579310
Question:
• Is ρ significantly different from zero?
−→ Durbin-Watson test for autocorrelation
199
Dependent Variable: RESID Method: Least Squares Date: 11/20/04 Time: 19:15 Sample(adjusted): 2002:02 2003:12 Included observations: 23 after adjusting endpoints
Variable Coefficient Std. Error t-Statistic Prob. RESID(-1) 0.579310 0.171176 3.384285 0.0027
21.92432 336.8232 14.10481
0.339455 Mean dependent var 0.339455 S.D. dependent var 273.7493 Akaike info criterion 1648652. Schwarz criterion 14.15418
R-squared Adjusted R-squared S.E. of regression Sum squared resid Log likelihood -161.2053 Durbin-Watson stat 1.574718
Durbin-Watson test: [I]
• Most popular test for autocorrelation(due to Durbin & Watson, 1950, 1951)
• Tests for both, positive (ρ > 0) and negative autocorrelation(ρ < 0)
• Test statistic:
calculate the residuals ui via OLS
test statistic:
DW =
N∑
i=2
(
ui − ui−1)2
/
N∑
i=1u2
i
200
Durbin-Watson test: [II]
• Relation to the OLS estimator ρ from Slide 197:
DW ≈ 2(1− ρ)
• Properties:
Since −1 < ρ < 1 it follows that 0 < DW < 4
no autocorrelation: ρ ≈ 0 ⇒ DW ≈ 2
positive autocorrelation: ρ ≈ 1 ⇒ DW ≈ 0
negative autocorrelation: ρ ≈ −1 ⇒ DW ≈ 4
201
Durbin-Watson test: [III]
• Test for positive autocorrelation:
hypotheses:
H0 : ρ ≤ 0 versus H1 : ρ > 0
distribution of DW under H0 depends on
– sampling size (N)
– number of exogenous regressors (K)
– specific values of the regressors x1i, . . . , xKi
−→ exact calculation by econometric software
202
Durbin-Watson test: [IV]
distribution under H0 of DW has lower and upper bounds
−→ exact critical values at the α-level (dα) have lower andupper bounds (i.e. dL
α ≤ dα ≤ dUα)
(for α = 0.05 see Von Auer, 2007, p. 402)
explicit decision rule:
– reject H0 : ρ ≤ 0 if DW < dLα
– do not reject H0 : ρ ≤ 0 if DW > dUα
– no decision if dLα ≤ DW ≤ dU
α
203
Durbin-Watson test: [V]
• Test for negative autocorrelation:
hypotheses:
H0 : ρ ≥ 0 versus H1 : ρ < 0
explicit decision rule:
– Reject H0 : ρ ≥ 0 if DW > 4− dLα
– Do not reject H0 : ρ ≥ 0 if DW < 4− dUα
– No decision if 4− dUα ≤ DW ≤ 4− dL
α
204
Example:
• Estimation of the price-revenue function (Slides 170, 171)
• Test for positive autocorrelation at the 5% level:
H0 : ρ ≤ 0 versus H1 : ρ > 0
• We have
N = 24, K = 1,DW = 0.760994, dL0.05 = 1.27, dU
0.05 = 1.45
and thus
DW = 0.760994 < 1.27 = dL0.05
−→ reject H0 at the 5%-level
205
Drawbacks of the Durbin-Watson test:
• Frequently there is no decision(e.g. if DW ∈ [dL
α, dUα]) when testing for pos. autocorrelation)
• DW-Test is unreliable if predecessor values like yi−1, yi−2, . . .are used as regressors(so-called lag-models)
• DW-test only tests for AR(1)-autocorrelation
206
7.3 Feasible Estimation Procedures
Now:
• Estimation of the autocorrelated model
y = Xβ + u
with AR(1) error-terms
u = ρu−1 + e (−1 < ρ < 1)
207
Problem:
• From the data set (X,y) we do not have direct knowledgeabout the autocorrelation parameter ρ
−→ FGLS estimation
Two feasible estimation procedures:
• GLS approach (Hildreth & Lu)
• FGLS approach (Cochrane & Orcutt)
208
1. Method by Hildreth & Lu: [I]
• Search algorithm
• Consider the GLS estimator
βGLS
=[
X′Ω−1X]−1
X′Ω−1y
where
Ω−1 =1
1− ρ2
1 −ρ 0 · · · 0 0−ρ 1 + ρ2 −ρ · · · 0 00 −ρ 1 + ρ2 · · · 0 0... ... ... ... ... ...0 0 0 ... 1 + ρ2 −ρ0 0 0 ... −ρ 1
(cf. Slides 184, 188)
209
1. Method by Hildreth & Lu: [II]
• Perform GLS estimation for distinct ρ-values (−1 < ρ < 1)
−→ compute the sum of squared residuals u∗′u∗ for each esti-mation
• Find the ρ-value with the minimal sum of squared residuals
−→ GLS estimator of β associated with this ρ-value is calledHildreth-Lu estimator
210
Example:
• Data on price-revenue function
−→ Hildreth-Lu estimates:
αHL = 3162.48, βHL = −53.51
211
ρ 0.60 0.68 0.69 0.70 0.80
u* 'u*∧ ∧
1756.358 1739.757 1739.748 1740.212 1772.438 ∧α 3420.42 3192.98 3162.48 3131.62 2817.53 ∧β −61.97 −54.50 −53.51 −52.50 −42.40
2. Method by Cochrane & Orcutt: [I]
• Iterative multi-step procedure
Procedure:
1. OLS estimation of the model
y = Xβ + u
2. Save the residuals
u = y−Xβ
212
2. Method by Cochrane & Orcutt: [II]
3. Consider the resgression
u = ρu−1 + e∗
and estimate ρ by the OLS estimator
ρ =
N∑
i=2ui−1ui
N∑
i=2u2
i−1
4. Use ρ to apply the FGLS estimator
βFGLS
=[
X′ Ω−1
X]−1
X′ Ω−1
y
213
2. Method by Cochrane & Orcutt: [III]
5. Improvement due to iteration:
• Compute the ”new” residuals
u(2) = y−XβFGLS
• Re-estimate ρ as in Step #3
• Find the ”new” FGLS estimator βFGLS(2)
• Repeat Step #5 until the FGLS estimator of β does notexhibit any further (substantial) change
214
Example: [I]
• Consider the price-revenue example
−→ Cochrane-Orcutt estimates:
αCO = 3264.88, βCO = −56.86
(possibly further iterations)
215
Estimate Iteration #1 Iteration #2 Iteration #3∧ρ
OLS estimate
0.58 0.64 0.66∧α 4262.12 3473.74 3310.85 3264.88∧β −89.58 −63.73 −58.37 −56.86
Example: [II]
• Contrasting both estimation results:
216
Parameter OLS estimate Hildreth-Lu Cochrane-Orcutt 4262.12262.12 262.12 89.58 53.51 56.86
Top Related