Advanced Econometrics - univie.ac.at

Univariate time series Multivariate time series Panels

Advanced EconometricsBased on the textbook by Verbeek:A Guide to Modern Econometrics

Robert M. [email protected]

University of Vienna

and

Institute for Advanced Studies Vienna

April 18, 2013

Advanced Econometrics University of Vienna and Institute for Advanced Studies Vienna


Outline

Univariate time seriesThe basicsGeneral ARMA processesUnit rootsChoosing a modelARCH

Multivariate time series

Panels



The basics

Time-series analysis: the idea

Time-series analysis searches data for dynamic structures that maybe useful in predicting the future. Besides forecasts, the structures(statistical models) may also reveal features of further interest.

The origin of the observed variable plays a minor role in theanalysis. Identical methods can be used on economic and onbiological data. Time-series analysis is not a branch of economics,but of statistics.



The basics

White noise

Modern time-series analysis assumes that observed sequences ofdata (time series) are realizations of stochastic processes. Astochastic process (εt) is called white noise iff

1. Eεt ≡ 0 for all t;

2. varεt ≡ σ2ε < ∞ for all t;

3. Eεtεt−j = 0 for all j 6= 0.

White noise is a process with no linear dynamic structure. If dataare white noise, this provides a poor prospect for analysis.However, white noise is an important building block in moreinteresting models.



The basics

The first-order autoregressive process

Assume the generating law

Yt = δ + θYt−1 + εt ,

with δ, θ ∈ R and (εt) white noise. Then, the process (Yt) is calleda first-order autoregressive process or AR(1) process.



The basics

Removing the mean from an AR(1) process

Assume |θ| < 1. Then, is it possible to obtain an AR(1) processwith time-constant mean EYt = EYt−1 = µ? It appears to be so,as this condition and

EYt = E(δ + θYt−1 + εt)

= δ + θE(Yt−1) + 0

yield µ = δ

1−θ. With the definition yt = Yt − µ, one may also write

yt = θyt−1 + εt

for the centered variable yt .



The basics

The variance of an AR(1) process

Assume |θ| < 1. Is it possible to obtain an AR(1) process withtime-constant variance varYt = varyt = σ2

Y ? It appears to be so,as this condition and

varyt = var(θyt−1 + εt)

= θ2var(yt−1) + σ2ε

yield σ2Y = σ2

ε

1−θ2. This uses the property that the noise term εt is

uncorrelated with the past observation yt−1, which is a reasonableassumption, to some researchers part of the definition of theAR(1).



The basics

Autocovariances

The second moments

cov(Yt ,Yt−j) = E(ytyt−j)

are an important characteristic of the joint distribution. If they aretime-constant (independent of t but dependent on j), they arecalled autocovariances γj . Note that γ0 = varYt and thatγj = γ−j .



The basics

Autocovariances for simple processesFor white noise εt , clearly

γj = 0, j 6= 0

and γ0 = σ2ε . For the AR(1) process,

γ1 = E(ytyt−1) = E{yt−1(θyt−1 + εt)}= θEy2t−1 = θγ0,

and, by similar substitution, generally

γk = θkγ0 = θkσ2ε

1− θ2.

If all second and first moments are time-constant, autocovariancesdecrease geometrically.



The basics

Stationarity

A process (Yt) is called covariance-stationary iff

EYt = µ ∀t,varYt = σ2

Y ∀t,cov(Yt ,Yt−k) = γk ∀t, k .

In short, we will use ‘stationary’ for ‘covariance-stationary’. In someapplications, it may be of interest that the entire distribution istime-constant, not only the first two moments (strict stationarity).

Clearly, white noise is stationary.



The basics

Is the AR(1) process stationary?

1. If θ = ±1, there is no solution to the variance condition, theAR(1) can never be stationary;

2. If |θ| > 1, solutions would violate the condition that errors areuncorrelated with past observations. There are stationary‘solutions’ that are unreasonable (time runs backward).Started from given values, the process ‘explodes’;

3. If |θ| < 1, the AR(1) can be stationary. It becomes stationaryif started from a given value and kept running for an infinitetime span. It is stationary if started from the correctdistribution. Sloppily, many researchers call such a processstationary, others call it stable.



The basics

The first-order moving-average processThe process defined by

Yt = µ+ εt + αεt−1

is called the first-order moving-average process or MA(1). Themean EYt = µ is time-constant, such that yt = Yt − µ has meanzero. The variance

varYt = σ2ε + α2σ2

ε = (1 + α2)σ2ε

is time-constant. The first-order autocovariance

γ1 = E(ytyt−1) = E(εt + αεt−1)(εt−1 + αεt−2)

= ασ2ε

is time-constant.



The basics

Finite linear dependence in MA processes

The second-order autocovariance

γ2 = E(ytyt−2) = E(εt + αεt−1)(εt−2 + αεt−3)

is zero, and similarly for all γk with k ≥ 2. The MA(1) process isstationary, and it is finite dependent in linear terms: observationsat time distance greater one are linearly unrelated. MA models‘forget fast’, they have very short ‘memory’.

Traded wisdom is that AR models are more often found ineconomic data than MA models.



The basics

The autocorrelation function

Rather than the autocovariances γk , most researchers prefer thenormalized autocorrelations

ρk =γkγ0

,

which by definition are bounded: −1 ≤ ρk ≤ 1. Because ofρk = ρ−k , they can be visualized as a function of the non-negativeintegers: the autocorrelation function or ACF.

The empirical ACF or sample ACF is sometimes called thecorrelogram. Others use the term ‘correlogram’ for a visualsummary of the empirical ACF and the empirical PACF.



The basics

The ACF of simple processes

◮ The ACF of white noise is

ρk =

{

1 , k = 0,0 , k > 0.

◮ The ACF of the stationary AR(1) process is

ρk = θk , k ∈ N.

◮ The ACF of the MA(1) process is

ρ0 = 1, ρ1 =α

1 + α2, ρk = 0, k ≥ 2.



General ARMA processes


A moving-average process of order q or MA(q) is defined by

yt = εt + α1εt−1 + . . .+ αqεt−q.

Similarly, an autoregressive process of order p or AR(p) is definedvia

yt = θ1yt−1 + . . .+ θpyt−p + εt .

An amalgam of the two is the ARMA(p, q) process

yt = θ1yt−1 + . . .+ θpyt−p + εt + α1εt−1 + . . .+ αqεt−q.




Infinite-order MA processes

The stable AR(1) model can be subjected to repeated substitution

yt = θyt−1 + εt = θ(θyt−2 + εt−1) + εt

= θkyt−k + εt + θεt−1 + . . . + θk−1εt−k+1

=

∞∑

j=0

θjεt−j ,

as the term depending on yt−k disappears in the limit. This is anMA(∞) process, sensibly defined only if (as here) the coefficientsconverge to 0 fast enough. All stable AR processes can berepresented as such MA(∞) processes.




The lag operatorAn operator is a function defined on a set or on its power set withthe image in the same set. The lag operator L is defined by

Lyt = yt−1

and operates on processes, observations etc. It is often mainly anotational device. Its powers are well defined by

L0yt = yt , L−1yt = yt+1, Lkyt = yt−k ,

and there are also lag polynomials in L:

(1− θ1L− θ2L2 − . . .− θpL

p)yt = yt − θ1yt−1 − . . .− θpyt−p = εt

writes the AR(p) model. In short, we may write

θ(L)yt = εt .




ARMA in lag polynomials

Usingθ(L) = 1− θ1L− θ2L

2 − . . .− θpLp

andα(L) = 1 + α1L+ α2L

2 + . . . + αqLq ,

the ARMA(p, q) process can be written compactly as

θ(L)yt = α(L)εt .

This is short, and it also admits many simple manipulations.




Inverting lag polynomialsIn general, the inverse of a polynomial is not a polynomial. Undercertain conditions, it can be written as a convergent power series:

θ(z) = 1− θz ∴ θ−1(z) =

∞∑

j=0

θjz j

converges for |z | ≤ 1 if |θ| < 1. Thus,

(1− θL)yt = εt ∴ yt = (1− θL)−1εt

yields the MA(∞) representation of an AR(1) process. Undercertain conditions, the expressions

yt = θ−1(L)α(L)εt , α−1(L)θ(L)yt = εt

work for general ARMA processes.




Characteristic polynomialsO.c.s. that the inverses of lag polynomials exist iff all ‘roots’(zeros) of the corresponding characteristic polynomials are largerthan one in modulus:

θ(L) = 1− θ1L− θ2L2 − . . .− θpL

p

has the corresponding characteristic polynomial

θ(z) = 1− θ1z − θ2z2 − . . . − θpz

p,

and this condition means that all ζ with θ(ζ) = 0 must have theproperty |ζ| > 1. Likewise, the MA lag polynomial has thecorresponding characteristic polynomial

α(z) = 1 + α1z + . . .+ αqzq.




Characteristic polynomials for p = 1

For the AR(1) process, the characteristic polynomial is

θ(z) = 1− θz ,

and its only root is larger one iff |θ| < 1. Then, the MA(∞)representation exists, and the AR(1) is stable. Generally, roots arelarge when coefficients are small.

For the MA(1) process, the characteristic polynomial is

α(z) = 1 + αz ,

and its only root is larger one iff |α| < 1. Then, the process has anAR(∞) representation, which may be convenient for prediction.




Canceling or common roots

Suppose that, in the representation of an ARMA process

yt = θ−1(L)α(L)εt , α−1(L)θ(L)yt = εt ,

some of the roots in the characteristic polynomials θ(z) and α(z)coincide. The fundamental theorem of algebra supports therepresentation (ζj , j = 1, . . . , p are the roots)

θ(z) = (1− ζ−11 z)(1− ζ−1

2 z) . . . (1− ζ−1p z),

with maybe some complex conjugates, and similarly for α(z). Thefactors in the expressions θ−1(L)α(L) cancel, and the ARMA(p, q)process is equivalent to an ARMA(p − 1, q − 1) process.




An example for common roots

Consider the ARMA(2,1) model

yt = yt−1−0.25yt−2+εt−0.5εt−1 ∴ (1−L+0.25L2)yt = (1−0.5L)εt .

Because of 1− z + 0.25z2 = (1− 0.5z)2, there is a common rootof z = 2, and the defined process is really the AR(1) model

yt = 0.5yt−1 + εt .

Common roots should be avoided. Representations becomenon-unique, and attempts to fit an ARMA(2,1) model to AR(1)data imply numerical problems.



Unit roots

The random walk

The AR(1) process with θ = 1

yt = yt−1 + εt

is called the random walk. It is not stationary. The root of itscharacteristic polynomial 1− z is 1. Its first-order difference

∆yt = yt − yt−1 = εt

is stationary. O.c.s. that this property is shared by all ARMAprocesses with exactly one root of 1 in their AR polynomial:non-stationary but first difference stationary. Such processes arecalled first-order integrated or I(1).



Unit roots

I(1) processes and the lag operator

Suppose that, in the ARMA model

θ(L)yt = α(L)εt ,

the polynomial θ(z) has exactly one root of 1 and all other rootsare nice (modulus greater one). Then, we may write

θ(z) = (1− z)θ∗(z) ∴ θ∗(L)(1 − L)yt = α(L)εt .

A valid ARMA representation exists for ∆yt = y∗t with θ∗(z) andα(z) both invertible, and yt is definitely I(1).



Unit roots

Higher-order integration

If the AR polynomial θ(z) has exactly two roots of 1 and all otherroots are nice, the transform

∆2yt = (1− L)2yt = yt − 2yt−1 + yt−2

will be stationary but yt and ∆yt will not be stationary. Theprocess (yt) is said to be second-order integrated or I(2).

Box and Jenkins suggested the notation ARIMA(p, d , q) for aprocess with d roots of one and all other roots nice. Usually, onlyd ∈ {0, 1, 2} occurs with economic variables.



Unit roots

Testing for a unit root in AR(1)

Apparently, testing for unit roots is of interest: I(1) processes arenot stationary, ‘all shocks have a permanent effect’, and takingdifferences is recommended.

Dickey and Fuller considered testing H0 : θ = 1 in the model

Yt = δ + θYt−1 + εt ,

using the simple t–statistic

DF =θ − 1

s.e.(θ),

where the denominator is just taken from the usual regressionoutput.



Unit roots

Dickey-Fuller distribution

O.c.s. that the statistic DF defines a valid test, but that DF isneither t–distributed nor normally distributed under H0. Thedistribution is non-standard and was tabulated first in 1976.

If a linear time trend is added to the regression

∆Yt = δ + (θ − 1)Yt−1 + γt + ut ,

the distribution of DF is again different. Versions with mean onlyand with trend are often called DF–µ and DF–τ .



Unit roots

Why include a linear trend?

For the regression model

∆Yt = δ + (θ − 1)Yt−1 + ut ,

H0 : θ = 1 defines the random walk with drift, HA : θ ∈ (−1, 1) astable AR(1).

In the regression model

∆Yt = δ + (θ − 1)Yt−1 + γt + ut ,

H0 : θ = 1 admits a random walk with quadratic trendsuperimposed, HA : θ ∈ (−1, 1) a trend-stationary AR(1), i.e. aprocess that becomes stationary after removing a trend. Thissecond problem is maybe more relevant for trending data.



Unit roots

Unit-root testing in an AR(2) model

Consider the AR(2) model

Yt = δ + θ1Yt−1 + θ2Yt−2 + εt ,

which can be re-written by some algebraic manipulation as

∆Yt = δ + (θ1 + θ2 − 1)Yt−1 − θ2∆Yt−1 + εt .

There will be a unit root in θ(z) iff the coefficient θ1 + θ2 − 1 is 0.The t–statistic on the coefficient of Yt−1 has the same propertiesas the DF in an AR(1) model: ‘augmented’ Dickey-Fuller test,really just a DF test.



Unit roots

Unit-root testing in an AR(p) model

Testing for a unit root in the AR(p) model can be conducted bythe t–statistic on π in the regression

∆Yt = δ[+γt] + πYt−1 + c1∆Yt−1 + . . . + cp−1∆Yt−p+1 + ut .

Significance points will be identical to the DF tests in the AR(1)model. O.c.s. that π = θ1 + . . .+ θp − 1 is 0 for an I(1) process.

◮ If the DF test rejects, Yt may be stable, stationary,trend-stationary.

◮ If the DF test does not reject, Yt may be I(1).



Choosing a model

Order selection: the idea

How determine the orders p and q in an ARMA model? Howdetermine the augmentation order in a DF test application?

The problem is that methods that choose a parameter from a finiteor countable set are less developed than hypothesis testing andestimation on a continuum. Fixes:

◮ Visual tools (require individual skills)

◮ Sequences of hypothesis tests (significance levels incorrect)

◮ Information criteria



Choosing a model

The autocorrelation function

In MA(q) models, the ACF values ρk are non-zero for k ≤ q andzero for k > q. The sample ACF can be used to determine q.

Under conditions, o.c.s. that, for k > q, i.e. for those ρk that arereally zero, √

T (ρk − ρk) → N (0, υk),

whereυk = 1 + 2ρ21 + . . .+ 2ρ2q .

This property can be used for testing H0 : ρk = 0 and for drawingconfidence bands, plugging in estimates for ρ1, . . . , ρq .



Choosing a model

The partial autocorrelation function

For AR and ARMA models, the sample ACF plots are not helpful.Rather, it is suggested to fit AR(p) models with increasing p:

Yt = δ + θ1.1Yt−1 + ut ,

Yt = δ + θ1.2Yt−1 + θ2.2Yt−2 + ut ,

Yt = δ + θ1.3Yt−1 + θ2.3Yt−2 + θ3.3Yt−3 + ut ,

and note down the last coefficient estimate θ1.1, θ2.2, θ3.3, . . ..Their population counterparts should be non-zero for k = p andzero for k > p. O.c.s. that

√T θk.k → N (0, 1), k > p.



Choosing a model

The classical visual oder determination

◮ If the ACF decays smoothly and the PACF cuts off at lag p,try an AR(p) model;

◮ If the PACF decays smoothly and the ACF cuts off at lag q,try an MA(q) model;

◮ If both ACF and PACF decay smoothly, this may be anARMA(p, q) model, but you do not know p nor q;

◮ If the ACF decays very slowly, the variable may correspond toan I(1) or even I(2) process, and you may wish to takedifferences.



Choosing a model

Information criteria: the idea

Information criteria are penalized likelihoods: more complexmodels have a larger likelihood, and penalizing complexityeventually leads to a reasonable choice. By convention,information criteria rely on the negative log-likelihood, are minimalat the optimum and are often negative.

Information criteria are not an alternative to likelihood-ratio testsor at odds with them. Under conditions, they are equivalent fornested comparisons. Many comparisons are non-nested (such asARMA(2,1) versus ARMA(1,3)), and direct hypothesis testscannot be used.



Choosing a model

The information criterion AIC

Akaike has suggested to penalize complexity linearly:

AIC = log σ2 + 2p + q + 1

T,

with σ2 an estimate for the error variance. The variance termrepresents minus the log-likelihood. The count of parameters (herep + q + 1) varies among authors (mean, variance).



Choosing a model

The information criterion BIC

Schwarz suggested a simplified version of a BIC that had beenintroduced by Akaike and penalizes complexity by a logarithmicfunction of the sample size:

BIC = log σ2 +p + q + 1

TlogT .

O.c.s. that minimizing BIC leads to the true p and q as T → ∞.By contrast, minimizing AIC optimizes the prediction properties ofthe selected model and may be preferable in smaller samples.



ARCH

ARCH models: the ideaFinancial time series, such as stock prices and exchange rates,often show little predictability in their means beyond their levels,their log-differences are white noise. However, the followingfeatures are found:

◮ Heavy tails: there are far more unusually large and smallobservations than for Gaussian data. There is substantialexcess kurtosis.

◮ Volatility clustering: episodes with large and small variationfollow each other. Large changes in the level are oftensucceeded by more large moves, with their direction beingunpredictable.

The ARCH (autoregressive conditional heteroskedasticity) modelby Engle (1982) captures these features successfully.



ARCH

The ARCH(1) model

The simplest ARCH model is the ARCH(1) model

σ2t = E(ε2t |It−1) = + αε2t−1,

where (εt) is a white-noise process, It−1 denotes an informationset containing all εs , s ≤ t − 1, and > 0, α ≥ 0.

σ2t is a local variance and represents volatility.



ARCH

Stationary ARCH(1)

If an ARCH(1) process (εt) is stationary, its variance must betime-constant. Taking expectations yields

σ2ε = Eσ2

t = + αE(ε2t−1)

= + ασ2ε ,

and henceσ2ε =

1− α.

Indeed, o.c.s. that stationary ARCH(1) processes exist iffα ∈ [0, 1). Whereas local volatility changes over time, theunconditional variance is time-constant.



ARCH

Higher-order ARCH models

The modelσ2t = + α1ε

2t−1 + . . . + αpε

2t−p

defines an ARCH(p) process. Like the ARCH(1) process, it can beeasily generated from a purely random series νt iteratively by

εt = σtνt = νt

√

√

√

√ +

p∑

j=1

αjε2t−j .

Stationarity would require α1 + . . . + αp < 1.



ARCH

GARCH models

The GARCH (generalized ARCH) model by Bollerslev letsvolatility depend also on its own past. For example, theGARCH(1,1) reads

σ2t = + αε2t−1 + βσ2

t−1,

with α, β ≥ 0 and α = 0 ⇒ β = 0. This is a very popular modeland fits some financial time series surprisingly well. Stationarityrequires α+ β < 1. Then, it is easy to show thatσ2 = /(1− α− β).

It is straightforward to define GARCH(p, q) models with largerorders.



ARCH

Integrated GARCH

Often, in estimated GARCH(1,1) models α+ β ≈ 1. Forα+ β = 1, the name is IGARCH (integrated GARCH). IGARCHprocesses may even be stationary, with infinite variance. Here, oneneeds the alternative definition of strict stationarity (time-constantdistribution) rather than covariance stationarity (covariances donot exist).

Generally, ARCH processes have large or infinite kurtosis, thoughinfinite variance may be extreme and is usually not supported inempirical finance.



ARCH

Exponential GARCH model

The EGARCH (exponential GARCH) model by Daniel Nelson

is the most popular nonlinear ARCH–type model:

log σ2t = + β log σ2

t−1 + γεt−1

σt−1+ α

|εt−1|σt−1

,

with γ 6= 0 implying asymmetric reaction. EGARCH needs nopositivity constraints on parameters. Often, the model fits wellempirically.


Advanced Econometrics - univie.ac.at

Documents

Transcript of Advanced Econometrics - univie.ac.at