Time Series Econometrics Lecture Notes - econ.boun.edu.tr Note-Tim… · Time Series Econometrics...

Time Series Econometrics Lecture Notes

Burak Saltoglu

May 2017

Contents

1 Introduction 71.1 Linear Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.1.1 Why do we study time series in econometrics . . . . . . . . . . . . . . 81.2 Objectives of time series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.2 Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.3 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.4 Policy and control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.3 Distributed Lag Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.1 Autoregressive Models . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.2 ARDL Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.3 Granger Causality Test . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4 Linear Time Series Models: y(t) . . . . . . . . . . . . . . . . . . . . . . . . 111.4.1 Stochastic Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.4.2 Times Series And White Noise . . . . . . . . . . . . . . . . . . . . . . 121.4.3 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.4.4 Strict Sense Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . 121.4.5 Wide Sense Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . 121.4.6 Basic ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.4.7 Lag Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.4.8 AR vs MA Representation Wold Decomposition . . . . . . . . . . . . 131.4.9 Autocorrelations and Autocovariance Functions . . . . . . . . . . . . 141.4.10 Sample Counterpart of AutoCovariance Function . . . . . . . . . . . 141.4.11 Partial Autocorrelation Function . . . . . . . . . . . . . . . . . . . . 141.4.12 Linear Time Series-AR . . . . . . . . . . . . . . . . . . . . . . . . . . 151.4.13 AR(1) Application on Turkish Growth Rate . . . . . . . . . . . . . . 211.4.14 general interpretation of time series and autocorrelation graphs . . . 231.4.15 stationarity in the graphs . . . . . . . . . . . . . . . . . . . . . . . . 241.4.16 Linear Time Series Models:MA(k) . . . . . . . . . . . . . . . . . . . . 241.4.17 MA(1) Correlogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251.4.18 An MA(1) Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251.4.19 Variance Covariance MA(2) . . . . . . . . . . . . . . . . . . . . . . . 251.4.20 Moving Average MA(k) Process . . . . . . . . . . . . . . . . . . . . . 261.4.21 ARMA Models: ARMA(1,1) . . . . . . . . . . . . . . . . . . . . . . . 27

3

CONTENTS CONTENTS

1.4.22 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . 281.4.23 Likelihood function for ARMA(1,1) process . . . . . . . . . . . . . . 28

1.5 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291.5.1 Two Model Selection Criteria . . . . . . . . . . . . . . . . . . . . . . 291.5.2 Characterization of Time Series . . . . . . . . . . . . . . . . . . . . . 301.5.3 Correlogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301.5.4 Sample Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . 301.5.5 Correlogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301.5.6 Test for Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . 311.5.7 Box-Pierce Q Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 321.5.8 Ljung-Box Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 321.5.9 Are Residuals Clean? . . . . . . . . . . . . . . . . . . . . . . . . . . . 331.5.10 Are Residuals GAUSSIAN? . . . . . . . . . . . . . . . . . . . . . . . 331.5.11 Optimal ARMA order level choice: Turkish GDP Growth . . . . . . . 341.5.12 Box-Jenkins Approach to Time Series . . . . . . . . . . . . . . . . . . 34

1.6 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351.6.1 Introduction to Forecasting . . . . . . . . . . . . . . . . . . . . . . . 351.6.2 In Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361.6.3 Mean Square Prediction Error Method (MSPE) . . . . . . . . . . . . 361.6.4 A Forecasting Example for AR(1) . . . . . . . . . . . . . . . . . . . . 361.6.5 Forecasting Performance . . . . . . . . . . . . . . . . . . . . . . . . . 381.6.6 Forecast Performance Evaluation . . . . . . . . . . . . . . . . . . . . 391.6.7 Forecast Combinations . . . . . . . . . . . . . . . . . . . . . . . . . . 391.6.8 Forecasting Combination Example . . . . . . . . . . . . . . . . . . . 401.6.9 Using Regression for Forecast Combinations . . . . . . . . . . . . . . 40

1.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2 Testing for Stationarity and Unit Roots 412.0.1 Spurious Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.0.2 Example Spurious Regression . . . . . . . . . . . . . . . . . . . . . . 422.0.3 Examples: Gozalo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.0.4 Unit Roots: Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . 442.0.5 Some Time Series Models: Random Walk Model . . . . . . . . . . . . 442.0.6 Why a Formal Test is Necessary? . . . . . . . . . . . . . . . . . . . . 462.0.7 How Instructive to Use ACF? . . . . . . . . . . . . . . . . . . . . . . 472.0.8 Testing for Unit Roots: Dickey Fuller . . . . . . . . . . . . . . . . . 472.0.9 Dickey Fuller Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482.0.10 Dickey-Fuller F-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502.0.11 ADF: Augemented Dickey Fuller Test . . . . . . . . . . . . . . . . . . 50

2.1 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3 COINTEGRATION 573.0.1 Money Demand Stability . . . . . . . . . . . . . . . . . . . . . . . . . 593.0.2 Testing for Cointegration Engle – Granger Residual-Based Tests Econo-

metrica, 1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4

CONTENTS CONTENTS

3.0.3 Residual Based Cointegration test: Dickey Fuller test . . . . . . . . . 60

3.0.4 Examples of Cointegration: Brent Wti Regression . . . . . . . . . . . 60

3.0.5 Example of ECM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.0.6 Error Correction Term . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.0.7 Use of Cointegration in Economic and Finance . . . . . . . . . . . . 63

3.0.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4 Multiple Equation Systems 65

4.1 Seemingly Unrelated Regression Model . . . . . . . . . . . . . . . . . . . . . 65

4.1.1 Recursive Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.1.2 Structural and Reduced Forms . . . . . . . . . . . . . . . . . . . . . . 66

4.1.3 Some Simultaneous Equation Models . . . . . . . . . . . . . . . . . . 67

4.1.4 Keynesian Income Function . . . . . . . . . . . . . . . . . . . . . . . 68

4.1.5 Inconsistency of OLS . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

4.1.6 Underidentification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

4.1.7 Test for Simultaneity Problem . . . . . . . . . . . . . . . . . . . . . 71

4.1.8 Simultaneous Equations in Matrix Form . . . . . . . . . . . . . . . . 71

5 Vector Autoregression VAR 73

5.0.1 Why VAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.0.2 VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.0.3 An Example VAR Models: 1 month 12 months TRY Interest ratesmonthly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.0.4 Hypotesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.1 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

6 Asymptotic Distribution Theory 79

6.1 Why large sample ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6.1.1 Asymptotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6.2 Convergence in Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

6.2.1 Mean Square Convergence . . . . . . . . . . . . . . . . . . . . . . . . 80

6.2.2 Convergence in Probability . . . . . . . . . . . . . . . . . . . . . . . . 80

6.2.3 Almost sure convergence . . . . . . . . . . . . . . . . . . . . . . . . . 80

6.3 Law of large numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.3.1 Convergence in Functions . . . . . . . . . . . . . . . . . . . . . . . . 81

6.3.2 Rules for Probability Limits . . . . . . . . . . . . . . . . . . . . . . . 82

6.4 Convergence in Distribution : Limiting Distributions . . . . . . . . . . . . . 82

6.4.1 Example Limiting Probability Distributions . . . . . . . . . . . . . . 82

6.4.2 Rules for Limiting Distributions . . . . . . . . . . . . . . . . . . . . . 82

6.5 Central Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

6.5.1 Example: distirbution of sample mean . . . . . . . . . . . . . . . . . 83

6.5.2 A simple monte carlo for CLT . . . . . . . . . . . . . . . . . . . . . . 84

5

CONTENTS CONTENTS

7 references 87

6

Chapter 1

Introduction

We will discuss various topics in time series this course. We will mainly discuss linear timeseries methods. We will go over mainly five critical topics first general introduction on thelinear time series. Then we will discuss non-stationary time series methods. A related topicto this will be co-integration and unit root testing. Finally, we will go over the VectorAutoregression (VAR) methodology. There are various topics such as time varying volatiltymodelling and nonlinear time seris methods that are left for more advanced courses. Thereis no single textbook that we will rely on. We will use, Vance Martin, Stan Hurn andDavid Harris, 2013, Econometric Modellig with Time Series. However, there aremany other relevant textbooks such as Hamilton (1994), Enders (2014),Chafield (2003) andDiebold(2016) among others. We will use various softwares mainly R or Matlab. Ability incoding is very useful to understand time series methods. The recent industrial popularity inbig data, machine learning, deep learning are all very related time series econometrics. Thiscourse might be helpful for a student who wants to learn these topics too. In this section wewill begin by linear time series methods.

1.1 Linear Time Series

One of the most popular analytical tools in econometrics/statistics is the linear time seriesmodels. We can basically summarize what we will be doing as follows.First, what is timeseries? A time series consists of a set of variable y, that takes values on a equally spacedinterval of time. Time subscript t is used to denote a variable that is observed in a a givensample frequency. Initially, low frequency time series studied. Yearly data, then quarterlydata in macroeconomics was useful. But as the computer technology and financial transactionbecame more and more complex now financial econometricians are even working with timefrequency of milisecond intervals.1. We will be mainly focusing on relatilvey lower frequencyaspects of the time series.

11 milisecond is 0.001 seconds

7

1.2. OBJECTIVES OF TIME SERIES CHAPTER 1. INTRODUCTION

1.1.1 Why do we study time series in econometrics

Main aim of time series analysis is to describe main the time series characteristics of variousmacroeconomic or financial variables.Historically, Yule(1927), Slutsky(1937) and Wold (1938)were the main pioneers of using econometrics in regression analysis. Econometricians mainlywere analyzing long and short term cyclical decomposition of macro time series. Lateron more is done in multiple regression and simultaneous regression. Research made byOperation research group was more focused on smoothing and forecasting time series. Boxand Jenkins(1976) was the main methodology adapted by all economists and engineers. Sincethen there are still an important developments in the field of time series. In a univariatecontext, an economist tries to understand whether there is any seasonal pattern. Or toanalyze whether a macro variable show a trend. More importantly, if there is a predictablepattern that we observe in a macro variable we try to forecast the future realization of thesevariables. Forecasting is a very important outlet for policy makers and the business.

Why do we trust time series?

There are various reasons why we see some predictable patterns in various time series.Onereason why there are patterns in time series reliance on Psychological Reasons. Peopledo not change their habits immediately.People’s consumer’s choice also show some type ofmomentum. For instance, once they start to demand housing they tend follow each otherand house prices may go monotonically for a long period of time. Businessmen and otheragents may also behave in a manner which makes the production follow a certain pattern.So time series distinguishes and analyzes these trends. There may be

1.2 Objectives of time series

Time series analysis has four main objectives.

1.2.1 Description

First step involves to plot the data and to obtain descriptive measures of the time series.Decomposing the cyclical and seasonal components of a given time series is also conductedin the first round.

1.2.2 Explanation

Fitting the most appropriate specification is done at this stage. Choosing the lag order andlinear model is done at this stage.

1.2.3 Prediction

Given the observed macro or financial data,one may usually want to predict the future valuesof these series. This is the predictive process where the unknown future values.

8

CHAPTER 1. INTRODUCTION 1.3. DISTRIBUTED LAG MODELS

1.2.4 Policy and control

Once the mathematical relationship between the input and output is found then to achieve atargeted output the level of input is set. It is usually useful for engineering but also relevantfor policy analysis in macroeconomics.

1.3 Distributed Lag Models

Distribute lag models rely on the philosophy that economic changes can be distributed overa number of time period. In other words, a series of lagged explanatory variables account forthe time adjustment process. In the distributed lag (DL) model we have not only currentvalue of the explanatory variable but also its past value(s). With DL models, the effect ofa shock in the explanatory variable lasts more. We can estimate DL models (in principal)with OLS. Because the lags of X are also non-stochastic. Error terms assumed to hold all thenecessary OLS assumptions such as normality, homoscedasticity and no serial correlation.Since the model is linear and attains all relevant assumptions these models can be estimatedthrough OLS. Usually,there should be an economic relationship exists between x and y.

DL(0) : yt = β0 + β1xt + ut (1.1)

DL(1) : yt = β0 + β1xt + β2xt−1 + ut (1.2)

DL(q) : yt = β0 +

q∑i=0

βixt−i + ut (1.3)

1.3.1 Autoregressive Models

AR(0) : yt = β0 + ut (1.4)

AR(1) : yt = β0 + α1yt−1 + ut (1.5)

AR(p) : yt = β0 +

p∑i=1

αiyt−i + ut (1.6)

In the Autoregressive (AR) models, the past value(s) of the dependent variables becomes anexplanatory variable. We can not esitmate an autoregressive model with OLS because:

1. Presence of stochastic explanatory variables

2. Posibility of serial correlation

1.3.2 ARDL Models

In the ARDL models, we have both AR and DL part in one regression.

ARDL(p, q) : yt = β0 +

p∑i=1

αiyt−i +

q∑j=0

βixt−i + ut (1.7)

9

1.3. DISTRIBUTED LAG MODELS CHAPTER 1. INTRODUCTION

1.3.3 Granger Causality Test

A common problem in economics is to determine changes in one variable are a cause ofchanges in another variable. For instance, do changes in the money supply cause changes inGNP or are GNP and money suppy are determined endogenously. Granger has suggestedthe following methodology to solve this issue. Let us consider the relation between GNP andmoney supply. A regression analysis can show us the relation between these two. But ourregression analysis can not say the direction of the relation.

The granger causality test examines the causality between series, the direction of therelation. We can test whether GNP causes money supply to increase or a monetary expansionlead GNP to rise, under conditions defined by Granger. First we can write the unrestrictedregression between GNP and money supply.

GNPt =m∑i=1

αiMt−i +m∑j=1

βjGNPt−j + ut (1.8)

Then we can run the restricted regression

GNPt =m∑j=1

βjGNPt−j + ut (1.9)

If Money supply has no contribution in determining GNP then we can set up the followinghypothesis α1 = α2 = ... = 0 If beta’s are significant then we can say that Money suppyGranger causes GNP. Similarly, to test whether GNP causes Money Suppy we run thefollowing set of regressions. First, the unrestricted regression is run

Mt =m∑i=1

λiGNPt−i +m∑j=1

δjMt−j + ut (1.10)

Then we run the restrictions

Mt =m∑j=1

δjMt−j + ut (1.11)

If the GNP has no contribution on determining Money supply one should expect λ1 = λ2 =.. = 0.If we reject the null of all coefficients are zero then we can say that GNP causesMoney supply. On the other hand if we also conclude that M causes GNP simultaneouslythen we talk of bi-variate causation.Both Money Suppy and GNP causes each other. It is anindication of endogeniety.

Steps in Granger Casuality

In order to formally test for testing M (Granger) causes GNP;

1. Regress GNP on all lagged GNP, obtain RSS1

2. Regress GNP on all lagged GNP and all lagged M, obtain RSS2

10

CHAPTER 1. INTRODUCTION 1.4. LINEAR TIME SERIES MODELS: Y(T)

3. The null is all α’s are zero.

4. Test statistics;

F =(RSS1 −RSS2)/m

RSS2/(n− k)

where m is the number of lags, k is the number of parameters in step-2. df(m,n-k).

Figure 1.1: Granger Causality Test

As can be seen from the above EVIEWS output, there is not enough information toclaim that there is Granger causality.

1.4 Linear Time Series Models: y(t)

Time series analysis is useful when the economic relationship is difficult to set. Even if thereare explanatory variables to express y, it is not possible to forecast y(t).

1.4.1 Stochastic Process

Any time series data can be thought of as being generated by a stochastic process. Astochastic process is said to be stationary if its mean and variance are constant over time .The value of covariance between two time periods depends only on the distance or lag betweenthe two time periods and not on the actual time at which the covariance is computed.

11

1.4. LINEAR TIME SERIES MODELS: Y(T) CHAPTER 1. INTRODUCTION

1.4.2 Times Series And White Noise

Suppose we have observed a sample of size T of some random variable Yt : y1, y2, y3, ....yT.This set of T numbers is only one possible outcome of the stochastic process. For example:yt∞t=−∞ = ..., y−1, y0, y1, ..., yT , yT+1, ...yt∞t=−∞ would still be viewed as a single realization from a time series process.

1.4.3 White Noise

A process is said to be white noise if it follows the following properties.

1. εt∞t=−∞ = ..., ε−1, ε0, ε1, ..., εT , εT+1, ...

2. E[εt

]= 0

3. E[ε2t

]= σ2

4. E[εtετ

]= 0 for t 6= τ

If a time series is time invariant with respect to changes in time, then the process can beestimated with fixed coefficients.

1.4.4 Strict Sense Stationarity

If f(y1, ..., yT

)represents the joint probability density of yt is said to be strictly stationarity

iff(yt, ..., yt+k

)= f

(yt+m, ..., yt+k+m

)1.4.5 Wide Sense Stationarity

yt is said to be wide sense stationary if the mean, variance and covariance of a time seriesare stationary.

µy = E[yt]

E[yt] = E[yt+m]

σ2y = E[(yt − µy)2] = E[(yt+m − µy)2]

Covariance of the series must be stationary

γk = cov(yt, yt+k) = E[(yt − µy)(yt+k − µy)

]cov(yt, yt+k) = cov(yt+m, yt+k+m)

Strict sense stationarity implies wide sense stationarity but the reverse is not true.Implication of stationarity: inference we obtain from a non-stationary series is mislead-

ing and wrong.

12


1.4.6 Basic ARMA Models

AR : yt = φ1yt−1 + δ + εt (1.12)

MA : yt = εt + θ1εt−1 (1.13)

AR(p) : yt = φ1yt−1 + φ2yt−2 + ....+ φpyt−p + δ + εt (1.14)

MA(q) : yt = εt + θ1εt−1 + θ2εt−2 + ....+ θqεt−q (1.15)

1.4.7 Lag Operators

Lyt = yt−1

L2 = LLyt = yt−2

or in general;Lj = yt−j

L−j = yt+j

or we can use lag polynomials;

a(L) = (a0L0 + a1L

1 + a2L2)

AR : (1− φ1L)yt = δ + εt (1.16)

MA : yt = (1 + θ1L)εt (1.17)

AR(p) : (1− φ1L1 − φ2L

2 − ...− φpLp)yt = δ + εt (1.18)

MA(q) : yt = (1 + θ1L1 + θ2L

2 + ...+ θqLq)εt (1.19)

1.4.8 AR vs MA Representation Wold Decomposition

yt = φ1yt−1 + εt

yt = φ1(φ1yt−2 + εt−1) + εt = φ21yt−2 + φ1εt−1 + εt

yt = φ1(φ21yt−3 + φ1εt−2 + εt−1) + εt = (φ3

1yt−3 + φ21εt−2 + φεt−1) + εt

yt = φk1yt−k + φk−11 yt−k+1 + ...+ φ2

1εt−2 + φεt−1 + εt

if |φ1| < 1 so thatlimk→∞

φk1yt−k = 0

then;

yt =∞∑j=0

φj1εt−j

So AR(1) can be represented as MA(∞)

13


1.4.9 Autocorrelations and Autocovariance Functions

yt =∞∑j=0

φjεt−j

var(yt) = E

[(∞∑j=0

φjεt−j

)2]

var(yt) =∞∑j=0

φ2jE[ε2t−j]

Note that;∑∞

j=0 φ2j = 1 + φ2 + ... = 1

1−φ2

var(yt) =1

1− φ2E[ε2

t−j] =1

1− φ2σ2

Autocorrelation:

γj = cov(yt, yt−j)

cov(yt, yt−j) = E[(yt − E[yt])(yt−j − E[yt−j])

]γ0 = cov(yt, yt) = var(yt)

Correlation of yt and yt−j given as

ρj =cov(yt, yt−j)

var(yt)=γjγ0

1.4.10 Sample Counterpart of AutoCovariance Function

γ0 =1

n− 1

T∑t=1

(yt − y)2 = σ2

γk =T∑t=1

(yt − y)(yt−k − y) k = 1, 2, ..

Because of stationarity:

γk = γ−k

ρk =γkγ0

1.4.11 Partial Autocorrelation Function

The PACF of a time series is a function of its ACF and is a useful tool for determining theorder p of an AR model. A simple,yet effective way to introduce PACF is to consider thefollowing AR models in consecutive orders:

14


rt = φ0,1 + φ1,1rt−1 + e1t,

rt = φ0,2 + φ1,2rt−1 + φ2,2rt−2 + e2t,

rt = φ0,3 + φ1,3rt−1 + φ2,3rt−2 + φ3,3rt−3 + e3t,

rt = φ0,4 + φ1,4rt−1 + φ2,4rt−2 + φ3,4rt−3 + φ4,4rt−4 + e4t,

.

.

.

where φ0,j, φi,j and ejt are, respectively, the constant term, the coefficient of rt−i, and theerror term of an AR(j) model. These models are in the form of a mulitple linear regressionand can be estimated by the least squares method. As a matter of fact, they are arrangedin a sequential order that enables us to apply the idea of partial F test in multiple linearregression analysis. The estimate φ1,1 of the first equation is called the lag-1 sample PACF

of rt.The estimate φ2,2 of the second equation is called the lag-2 sample PACF of rt.The

estimate φ3,3 of the third equation is called the lag-3 sample PACF of rt, and so on.

From the definition, the lag-2 sample PACF φ2,2 shows the added contribution of rt−2 tort over the AR(1) model rt = φ0+φ1rt−1+e1t. The lag-3 PACF shows the added contributionof rt−3 to rt over an AR(2) model, and so on. Therefore, for an AR(p) mode, the lag-p sample

PACF should not be zero, but φj,j should be close to zero for all j > p. We make use of thisproperty to determine the order p.

Measures the correlation between an observation k periods ago and the current obser-vation, after controling for intermediate lags. For The first lags pacf and acf are equal.

1.4.12 Linear Time Series-AR

yt = φ1yt−1 + δ + εt |φ1| ≥ 1 non-stationarity condition

E[yt] = µ =δ

1− φ1

|φ1| ≤ 1 stationarity condition

Let δ = 0 then,γ0 = E[(yt − µ)2] = E[(φ1yt−1 + εt)

2]

γ0 = E[φ21y

2t−1 + 2φ1yt−1εt + ε2

t ]

From stationarity, note that E[y2t−1] = γ0;

γ0 = φ21γ0 + σ2

ε

γ0 =σ2ε

1− φ21

You can also see the above result directly from the equation, yt = φ1yt−1 + δ + εt bytaking the variance of both sides.

15


γ1 = E[(yt − µy)(yt−1 − µy)]

γ1 = E[(φ1yt−1 + εt)(yt−1)]

γ1 = φ1γ0

γ1 =φ1σ

2ε

1− φ21

, γ0 =σ2ε

1− φ21

ρ1 =γ1

γ0

=

φ1σ2ε

1−φ21σ2ε

1−φ21

= φ1

For j = 2;

γ2 = E[(yt)(yt−2)]

γ2 = E[(φ1yt−1 + εt)(yt−2)]

γ2 = E[(φ1(φ1yt−2 + εt−1) + εt)(yt−2)] = E[(φ21yt−2 + φ1εt−1 + εt)(yt−2)]

γ2 = φ21σ

2y = φ2

1γ0

ρ2 =φ2

1γ0

γ0

,

So if you have a data which is generated by an AR(1) process, it is correlogram willdiminish slowly (if it is stationary).

ρ1 =γkγ0

= φ1

ρ2 = φ21

.

.

ρk = φk1

16


Figure 1.3: AR(1) Process with φ1 = 0.95

Figure 1.2: Random Walk (No Drift)

17




18



Figure 1.7: AR(1) Process with φ1 = 0.05 (Weak Predictable Part)

19


Figure 1.8: Turkish GDP Growth

Figure 1.9: Turkish GDP

Figure 1.10: US GDP: 1947-2017 (Quarterly)

20


Figure 1.11: Turkish GDP quarterly: Autocorrelations

1.4.13 AR(1) Application on Turkish Growth Rate

Turkish GDP Estimate SE t-stat

Constant 0.93 0.46 2.05AR(1) 0.80 0.07 11.99

Variance 11.86 1.87 6.33

Mean 4.6635

S. Deviation 5.6062Skewness -1.3031Kurtosis 4.3208

E[yt] =δ

1− φ2=

0.93

1− 0.82= 4.562

var(yt) =1

1− φ2σ2

21


Figure 1.12: Turkish Inflation

Figure 1.13: White Noise

22


Figure 1.14: Correlograms with φ = 0.9 and φ = −0.8

Linear Time Series Models:AR(p)

Autoregressive :

yt =k∑i=1

φiyt−i + δ + εt AR(k)

Expected value of Y:

E[Yt] =k∑i=1

φiµ+ δ

µ =k∑i=1

φiµ+ δ

µ =δ

1−∑k

i=1 φi

1.4.14 general interpretation of time series and autocorrelationgraphs

As can be seen in various simulated graphs in previous chapters we notice that, correlogramof stationary AR models tells a lot about the structure of the time series. First of all,white noise model produces time series realizations which are very difficult to predict. Theyproduce rather erratic and unpredictable pattern. (see figure 1.13). For the AR process withφ1 = 0.05 shows similar pattern to a white noise process (see figure 1.7). The predictive

23


power of with higher AR coefficients exhibit a totally different pattern. in the AR model thehgiher the persistence. For instance when φ1 = 0.9 (see figure 1.14) for an AR process we seemore predictable patterns in the original time series. In addition, we notice the correlogramregarding to this process has a slowly decaying pattern unlike when φ1 = 0.5.

1.4.15 stationarity in the graphs

As can be seen in the graph 1.2 random walk depicts a very different picture than otherstationary AR processes. For instance, in a typical random walk realization, time series doesnot have a stable mean or variance. In general, random walk models are known with theirunpredictable patterns. In some macro variables we notice random walk type features.

1.4.16 Linear Time Series Models:MA(k)

Moving average:

yt = µ+ εt + θ1εt−1 MA(1)

Note that in some texts yt = µ+ εt − θ1εt−1 is also considered as an MA process.The term ‘moving average’ comes from the fact that y is constructed from a weighted

sum of the two most recent error terms.

var(yt) = E[(yt − µ)(yt − µ)] = γ0 = E[(µ+ εt + θ1εt−1 − µ)(µ+ εt + θ1εt−1 − µ)]

By using white noise property,var(yt) = σ2

ε + θ21σ

2ε

var(yt) = σ2ε(1 + θ2

1)

Covariance:

γ1 = E[(yt − µ)(yt−1 − µ)] = E[(µ+ εt + θ1εt−1 − µ)(µ+ εt−1 + θ1εt−2 − µ))]

E[(yt − µ)(yt−1 − µ)] = θ1σ2ε

Higher covariances, j=2;

γ2 = E[(yt − µ)(yt−2 − µ)] = E[(µ+ εt + θ1εt−1 − µ)(µ+ εt−2 + θ1εt−3 − µ))]⇒ 0

γj = 0 j = 2, 3, ...

correlation(yt, yt−j) =cov(yt, yt−j)

var(yt)=γjγ0

=γj√γ0√γ0

= ρj

ρ1 =θ1σ

2

(1 + θ21)σ2

=θ1

(1 + θ21)

ρ2 = 0

.

.

ρk = 0

24


1.4.17 MA(1) Correlogram

ρ1 =γ1

γ0

ρ2 = 0

.

.

ρk = 0

ρk =γkγ0

=

θ1

(1+θ21), for k = 1

0, for k > 1

since

γk = E[(εt + θ1εt−1)(εt−k + θ1εt−k−1)] = 0 for k > 1

So if you have a data that is generated by MA(1) its correlogram will decline to zeroquickly(after one lag).

1.4.18 An MA(1) Example

yt = µ+ εt + 0.5εt−1 MA(1)

ρ1 =θ1

(1 + θ21)

=0.5

(1 + 0.52)= 0.4

.

.

ρk = 0

One major implication is the MA(1) process has a memory of only one Lag. i.e. MA(1)process forgets immediately after one term or only remembers just one previous realization.

1.4.19 Variance Covariance MA(2)

γ0 = E[(εt + θ1εt−1 + θ2εt−2)2] = E[(ε2t + θ2

1ε2t−1 + θ2

2ε2t−2)]

γ0 = σ2ε + θ2

1σ2ε + θ2

2σ2ε = σ2

ε(1 + θ21 + θ2

2)

For j = 1;γ1 = cov(yt, yt−1) = E[(yt − µ)(yt−1 − µ)] since E[yt] = µ

γ1 = E[(εt + θ1εt−1 + θ2εt−2)(εt−1 + θ1εt−2 + θ2εt−3)

]γ1 = θ1E[ε2

t−1] + θ1θ2E[ε2t−2]

γ1 = θ1σ2ε + θ1θ2σ

2ε

γ1 = σ2ε(θ1 + θ1θ2)

25


For j = 2 ;

cov(yt, yt−2) = E[(yt − µ)(yt−2 − µ)] = E[(εt + θ1εt−1 + θ2εt−2)(εt−2 + θ1εt−3 + θ2εt−4)

]γ2 = θ2σ

2ε

For j = 3;

γ3 = 0

Summary;

γ0 = σ2ε + θ2

1σ2ε + θ2

2σ2ε = σ2

ε(1 + θ21 + θ2

2)

γ1 = θ1σ2ε + θ1θ2σ

2ε

γ2 = θ2σ2ε

γ3 = 0

Correlations;

ρ1 =γ1

γ0

=θ1 + θ1θ2

1 + θ21 + θ2

2

ρ2 =γ2

γ0

=θ2

1 + θ21 + θ2

2

ρk = 0 k>2

1.4.20 Moving Average MA(k) Process

yt = µ+ εt +k∑i=1

θiεt−i MA(k)

• Error term is white noise.

• MA(k) has k+2 parameters.

• Variance of y;

var(yt) = γ0 = var(µ+ εt + θ1εt−1 + θ2εt−2 + ...+ θkεt−k)

var(yt) = σ2ε + θ2

1σ2ε + θ2

2σ2ε + ...+ θ2

kσ2ε

var(yt) = σ2ε(1 + θ2

1 + θ22 + ...+ θ2

k)

Homework: Derive the autocorrelation function for MA(3),..MA(k).

26


1.4.21 ARMA Models: ARMA(1,1)

yt = φ1yt−1 + εt + θ1εt−1

γ0 = E[y2t ] = E

[φ2

1y2t−1 + 2φ1yt−1εt + 2φ1θ1yt−1εt−1 + ε2

t + 2θ1εtεt−1 + θ21ε

2t−1

]γ0 = φ2

1γ0 + σ2ε + θ2

1σ2ε + 2φ1θ1σ

2ε

γ0 =σ2ε + θ2

1σ2ε + 2φ1θ1σ

2ε

(1− φ21)

γ0 =σ2ε(1 + θ2

1 + 2φ1θ1)

(1− φ21)

For j = 1;

γ1 = E[yt−1(φ1yt−1 + εt + θ1εt−1)]

γ1 = E[yt−1(φ1yt−1) + yt−1εt + θ1yt−1εt−1]

γ1 = E[φ1y2t−1 + yt−1(εt + θ1εt−1)]

γ1 = E[φ1y2t−1 + (φ1yt−2 + εt−1 + θ1εt−2)(εt + θ1εt−1)]

Using white noise property,

γ1 = φ1γ0 + θ1σ2ε

For j = 2;

γ2 = E[yt−2(φ1yt−1 + εt + θ1εt−1)]

γ2 = φ1E[yt−2yt−1] since

[yt−2 = (φ1yt−3 + εt−2 + θ1εt−3)] no correlation between yt−2, εt and εt−1

γ2 = φ1γ1

..

..

γk = φ1γk−1

ρ1 =γ1

γ0

So ARMA will have either oscilating or exponential decay depending on φ1 or θ1, but will bestationary and short memory(though we won’t cover long memory models in this course).

27


1.4.22 Maximum Likelihood Estimationε1

ε2

.

.εt

∼ N(0;σ2ε)

f(εt) =1√2πexp

(− 1

2.ε2t

σ2

)Then if we use, independent and identical distribution assumptions,

f(εT )xf(εT−1)...xf(ε0) = f(εT , εT−1...ε0)

L(θ; y) =1√

2πσ2.exp

(− 1

2.ε2

1

σ2

)1√

2πσ2.exp

(− 1

2.ε2

2

σ2

)...

1√2πσ2

.exp

(− 1

2.ε2T

σ2

)

L(θ; r) = f(εT , εT−1, ...., ε1, ε0) =T∏t=1

f(εt|Ωt−1)

fε(ε1, ...εT ) = (2πσ2ε)−T

2 exp

− 1

2σ2ε

T∑t=1

ε2t

Deriving the likelihood function:

L(θ; y) =1√

(2πσ2)(2πσ2)..(2πσ2)e

(− 1

2

∑Ttε2tσ2

)

Estimation AR(1);Since T and other parameters are constant we can ignore them in optimization and use

:

ln(L(θ)) = l(θ) = −1

2

T∑t=1

ln(σ2)− 1

2

T∑t=1

ε2t

σ2

εt = yt − θ1yt−1

θ = (θ1, σ2)′

1.4.23 Likelihood function for ARMA(1,1) process

yt = φ1yt−1 + εt + θ1εt−1

εt = yt − φ1yt−1 − θ1εt−1

ln(L(θ)) = l(θ) = −1

2

T∑t=1

ln(σ2)− 1

2

T∑t=1

[yt − φ1yt−1 − θ1εt−1

σ

]2

θ = (φ1, θ1, σ2)′

28

CHAPTER 1. INTRODUCTION 1.5. MODEL SELECTION

1.5 Model Selection

There are various steps to be followed in time series modelling. First step is to use graphicalinterpretation of the data. Graphical representation helps us in various aspects

1. To summarize and reveal patterns in the data:to distinguish between the linear andnonlinear features of the data can be seen from the data.

2. To identify anomalies in the data

3. Graphics enables us to present a big amount of data in a small space.

4. Multiple comparisons: being able to compare different piece of the data both withinand between various series.

to summarize

• How well does it fit the data?

• Adding additional lags for p and q will reduce the SSR.

• Adding new variables decrease the degrees of freedom

• In addition, adding new variables decreases the forecasting performance of the fittedmodel.

• Parsimonious model: optimizes this trade-off.

1.5.1 Two Model Selection Criteria

• Akaike Information Criterion

• Schwartz Bayesian Criterion

• AIC: k is the number of parameters estimated. If intercept term is allowed: k =(p+ q + 1) else k = p+ q.

• T: number of observations

AIC = T ln(SSR) + 2k

SBC = T ln(SSR) + kln(T )

Choose the lag order which minimizes the AIC or SBC.

AIC may be biased towards selecting overparametrized model wheras SBC is asymptot-ically consistent.

29

1.5. MODEL SELECTION CHAPTER 1. INTRODUCTION

1.5.2 Characterization of Time Series

• Visual inspection

• Autocorrelation order selection

• Test for significance

– Barlett (individual)

– Box Ljung (joint)

1.5.3 Correlogram

One simple test of stationarity is based on autocorrelation function (ACF). ACF at lag k is;

ρk =E[(yt − µy)(yt+k − µy)

]√E[(yt − µy)2]E[(yt+k − µy)2]

ρk =cov(yt, yt+k)

σytσyt+k

Under stationarity, ρk =cov(yt, yt+k)

σ2y

ρk =γkγ0

1.5.4 Sample Autocorrelation

ρk =

∑T−kt=1 (yt − y)(yt+k − y)∑T

t=1(yt − y)2

1.5.5 Correlogram

−1 < ρk < 1

If we plot ρk against k, the graph is called as correlogram. As an example let us lookat the correlogram of Turkey’s GDP.

30


Figure 1.15: Correlogram of Turkey’s GDP

1.5.6 Test for Autocorrelation

Barlett Test: to test for ρk = 0

H0 : ρk = 0

H1 : ρk 6= 0

ρk ∼ N(0,1√T

)

Figure 1.16: Turkish Monthly Interest Rates

31


Figure 1.17: ISE30 Return Correlation

1.5.7 Box-Pierce Q Statistics

To test the joint hypothesis that all the autocorrelation coefficients are simultaneously zero,one can use the Q statistics.

Q = Tm∑k=1

ρ2k where m= lag length, T=sample size

Q ∼asy χ2m

1.5.8 Ljung-Box Statistics

It is variant of Q statistics as;

LB = T (T + 2)m∑k=1

( ρk2

n− k

)

LB ∼asy χ2m

32


Figure 1.18: Box-Pierce Q Statistics

1.5.9 Are Residuals Clean?

Figure 1.19: Graph of residuals from an AR(1)

1.5.10 Are Residuals GAUSSIAN?

Figure 1.20

33


1.5.11 Optimal ARMA order level choice: Turkish GDP Growth

ARMA order to minimize BIC . ARMA(1,1),ARMA(1,2),. . . ARMA(4,4)

(p,q) 1 2 3 41 388.3247 387.5649 381.3381 383.45512 386.1168 381.3223 385.5452 384.76643 390.0506 385.5218 385.4699 377.45324 388.1361 386.6513 389.7934 381.5608

ARMA(3,4) minimizes the BIC. What is the maximum lag order to start with? No clearrule but not more than the 10% of the whole sample should be left out. i.e. With 100observations a maximum of 10 lag order is more or less the maximum AR level..

1.5.12 Box-Jenkins Approach to Time Series

Figure 1.21: Box-Jenkins Approach

34

CHAPTER 1. INTRODUCTION 1.6. FORECASTING

1.6 Forecasting

Figure 1.22: Forecasting

1.6.1 Introduction to Forecasting

yt = φ0 + φ1yt−1 + εt

But if we want to project future realizations of y we may use

yT+1 = φ0 + φ1yT + εT+1

ET [yT+1] = φ0 + φ1yT

Formally;

ET [yT+h] = E(yT+h|yT , ..., εT , ..., ε1)

ET [yT+2] = φo0ppkp0 + ET [φ1yT+1]

ET [yT+2] = φ0 + φ1(φ0 + φ1yT ) = φ0 + φ1φ0 + φ21yT

2 step ahead forecasts;

ET [yT+2] = φ0 + φ1(φ0 + φ1yT ) = φ0 + φ1φ0 + φ21yT

3 step ahead forecasts;

ET [yT+3] = φ0 + φ1(φ0 + φ1φ0 + φ21yT ) = φ0 + φ1φ0 + φ2

1φ0 + φ31yT

h step ahead forecasts;

ET [yT+h] = φ0(1 + φ1 + φ21 + ...+ φh−1

1 ) + φh1yT

35

1.6. FORECASTING CHAPTER 1. INTRODUCTION

1.6.2 In Practice

yt = φ0 + φ1yt−1

But if we want to project the future realizations of y we may use

ET [yT+1] = φ0 + φ1yT

If we can consistently estimate the order via AIC then one can forecast the future values ofy.

There are alternative measures to conduct forecast accuracy.

1.6.3 Mean Square Prediction Error Method (MSPE)

Choose model with the lowest MSPE. If there are observations in the holdback periods, theMSPE for Model 1 is defined as:

MSPE =1

R

T+R∑t=T+1

e2t

where e is the prediction error(i.e. et = yT+1 − yT+1)

RMSPE =

√√√√ 1

R

T+R∑t=T+1

e2t

1.6.4 A Forecasting Example for AR(1)

Suppose we are given

t = 1, 2, .., 150

T = 151, ..., 160

R = 10

yt = 0.9yt−1 + εt

Figure 1.23: AR(1) series with φ1 = 0.9

36


Figure 1.24

yt = φ1yt−1+εt for convenience we dropped the intercept after estimating we found: φ1 = 0.9

Suppose we want to forecast t = T + 1, T + 2, .., T +R

T = 150 : y150 = −7.16

ET [yT+1] = φ1yT

y150+1 = 0.9× (−7.16) = −6.45

y150+1 = −6.26 actual

e150 = 0.18 forecast error

2-step ahead forecast:

ET [y150+2] = φ1(φ1yT ) = φ21yT = 0.81× (−7.16)

and so forth.

Figure 1.25: Forecast of AR(1) Model

37

1.6. FORECASTING CHAPTER 1. INTRODUCTION

1.6.5 Forecasting Performance

Figure 1.26

MSPE =1

R

T+R∑t=T+1

e2t

where e is the prediction error(i.e. et = yT+1 − yT+1)

RMSPE =

√√√√ 1

R

T+R∑t=T+1

e2t

Figure 1.27: AR(1) Forecast

38


Figure 1.28: Forecast Error and Error Square

1.6.6 Forecast Performance Evaluation

If model A has less RMSE than Model B, then Model A is said to have a better forecastingpower. Recently many papers are out to test whether A is better than B in terms of predictionpower, but looking to RMSE is a good starting point.

Seminal papers: Diebold and Mariano (1995) White (2000), Reality Check Econometricapaper..and many more recently

1.6.7 Forecast Combinations

Assume that there are 2 competing forecast models: a and b

yT+1 = wyaT+1 + (1− w)ybT+1

In addition, the forecast errors also has the same linear combination;

εT+1 = wεaT+1 + (1− w)εbT+1

σ2T+1 = w2

aσ2a,T+1 + (1− wa)2σ2

b,T+1 Assuming no correlation between model a and b

∂σ2T+1

∂wa= 2waσ

2a,T+1 − 2(1− wa)σ2

b,T+1

2waσ2a,T+1 − 2(1− wa)σ2

b,T+1 = 0

2waσ2a,T+1 + 2waσ

2b,T+1 − 2σ2

b,T+1 = 0

wa(σ2a,T+1 + σ2

b,T+1) = σ2b,T+1

w∗a =σ2b,T+1

(σ2a,T+1 + σ2

b,T+1)

39

1.7. SUMMARY CHAPTER 1. INTRODUCTION

So, if model a has greater prediction error than b we give more weights to a.

yT+1 =

[σ2b,T+1

(σ2a,T+1 + σ2

b,T+1)

]yaT+1 +

[σ2a,T+1

(σ2a,T+1 + σ2

b,T+1)

]ybT+1

1.6.8 Forecasting Combination Example

Voting Behavior: Suppose company A forecasts the vote for party X: 40%, B forecasts 50%.past survey performances:

σ2a = 0.3

σ2b = 0.2

yT+1 =

[20

(20 + 30)

]yaT+1 +

[30

(20 + 30)

]ybT+1

yT+1 = 0.40× 40 + 0.6× 50 = 46%

1.6.9 Using Regression for Forecast Combinations

Run the following regression and then do the forecasts on the basis of estimated coefficients

yT+1 = β0 + βayaT+1 + βby

bT+1 + εT+1

1.7 Summary

• Find the AR, MA order via autocovariances, correlogram plots

• Use, AIC, SBC to choose orders

• Check LB stats

• Run a regression

• Do forecasting (use RMSE or MSE) to choose the best out-of-sample forecasting model.

40

Chapter 2

Testing for Stationarity and UnitRoots

Outline

• What is unit roots?

• Why is it important?

• Spurious regression

• Test for unit roots

• Dickey Fuller

• Augmented Dickey Fuller tests

• Stationarity and random walk

• Can we test via ACF or Box Ljung?

• Why a formal test is necessary?

• Source: W Enders Chapter 4, chapter 6

2.0.1 Spurious Regression

Regressions involving time series data include the possibility of obtaining spurious or dubiousresults signals the spurious regression. Two variables carrying the same trend makes twoseries to move together this does not mean that there is a genuine or natural relationship.

If both yt and xt are non-stationary,

yt = β1xt + εt

might display rather high R2 high t-stats.One of OLS assumptions was the stationarity of these series, we will call such regression

as spurious regression (Newbold and Granger (1974)).

41

CHAPTER 2. TESTING FOR STATIONARITY AND UNIT ROOTS

Figure 2.1: Clive Granger

Figure 2.2: Robert Engle

The least squares estimates are not consistent and regular tests and inference do nothold. As rule of thumb (Granger and Newbold,1974)

R2 > dw

2.0.2 Example Spurious Regression

Two simulated RW:Arl.xls

Xt = Xt−1 + ut ut ∼ N(0, 1)

Yt = Yt−1 + εt ε ∼ N(0, 1)

ut and εt are independent.

42


Figure 2.3: Random Walks

Figure 2.4: Yt = βXt + ut

2.0.3 Examples: Gozalo

1. Egyptian infant mortality rate (Y), 1971-1990, annual data on Gross Aggregate Incomeof American farmers (I) and Total Honduran Money Supply (M):

Yt = 179.9(16.63)

+−2.952(−2.32)

I − −4.26(−0.0439)

M

R2 = 0.918 D/W = 0.4752 F = 95.17 CORR. = 0.8858, −0.9113, −0.9445

2. Total crime rates in US (Y) 1971-1991, annual data, on life expactancy of SouthAfrica(X),

Yt = −24569(−6.03)

+ 628.9(9.04)

X

R2 = 0.811 D/W = 0.5061 F = 81.72 CORR. = 0.9008

43


2.0.4 Unit Roots: Stationarity

yt = β1yt−1 + εt

|β1| < 1⇒ AR(1)

β1 = 1⇒ Unit Roots!

2.0.5 Some Time Series Models: Random Walk Model

yt = yt−1 + εt εt ∼ i.i.d. N(0, σ2ε)

Where error term(εt) follows the white noise property with the following properties:

1. E[εt] = E[εt|εt−1, εt−2...] = E[εt|All information at t-1] = 0

2. E[εtεt−j] = cov(εt, εt−j) = 0

3. var(εt) = var(εt|εt−1, εt−2...) = var(εt|All information at t-1) = σ2ε

Now let us look at the dynamics of such a model;

yt = yt−1 + εt

if y0 = 0

y1 = ε1

y2 = y1 + ε2 ⇒ y2 = ε1 + ε2

y3 = y2 + ε3 ⇒ y3 = ε1 + ε2 + ε3

.

.

.

yN = ε1 + ε2 + ε3 + ...+ εN =N∑t=1

εt

σ2(yt) = E[ε21 + ....ε2

N ] = σ2 + ...+ σ2 = Nσ2

limN→∞

σ2(yt)→∞

44


Implications of Random Walk:

• Variance of yt diverges to infinity as N tends to infinity

• Usefulness of point forecast yt+1 diminishes as N increases

• Unconditional variance of yt is unbounded.

• Shocks in a random walk model does not decay over time.

• So shocks will have a permanent effect on the y series.

Figure 2.5: Random Walk with No Drift

Figure 2.6: Random Walk: BIST 30 Index

45


Figure 2.7: Random Walk:ISE Percentage Returns

Figure 2.8: Turkish Export and Imports (in USD mio)

2.0.6 Why a Formal Test is Necessary?


To test for β1 = 1 through t-test is not feasible since var(yt)→∞ so that the standardt-test is not applicable

For instance, daily brent oil series given below graph shows non-stationarity time series.

46


Figure 2.9: Brent Oil Historical Data

2.0.7 How Instructive to Use ACF?

Figure 2.10: Correlogram of daili Brent Oil

Does Crude Oil data follow random walk? (or does it contain unit root)?Neither Graph nor autocovariance functions can be formal proof of the existence of

random walk series.How about standard t-test?

2.0.8 Testing for Unit Roots: Dickey Fuller

We estimated the daily crude oil data with the following spesification


Estimated regression model is;yt = 1.000335yt−1

SE=0.000305

47


t− test statistic = 3277

But it would not be appropriate to use this information to reject the null of unit root.This t-test is not appropriate under the null of a unit–root. Dickey and Fuller (1979,1981)developed a formal test for unit roots. Hypothesis tests based on non-stationary variablescannot be analytically evaluated. But non-standard test statistics can be obtained via MonteCarlo.

2.0.9 Dickey Fuller Test

yt = β1yt−1 + ut : Pure Random Walk Model

yt = β0 + β1yt−1 + ut : Random Walk with Drift

yt = β0 + β1yt−1 + β2t+ ut : Random Walk with Drift and Time trend

These are three versions of the Dickey-Fuller (DF) unit root tests. The null hypothesisfor all versions is same whether β1 is one or not.

Now If we subtract yt−1 from each side

yt − yt−1 = β1yt−1 − yt−1 + ut

yt − yt−1 = β0 + β1yt−1 − yt−1 + ut

yt − yt−1 = β0 + β1yt−1 − yt−1 + β2t+ ut

The test involves to estimate any of the below specifications:

∆yt = γyt−1 + ut

∆yt = β0 + γyt−1 + ut

∆yt = β0 + β2t+ γyt−1 + ut

So we will run and test the slope to be significant or not So the test statistic is the sameas conventional t-test.

∆yt = β0 + γyt−1 where γ = β1 − 1

Hence;

H0 : γ = 0

48


Figure 2.11: Running DF regression

Figure 2.12: Testing DF in E-views

49


Figure 2.13: DF E-views

Figure 2.14: Testing for DF for other specifications: RW with trend

2.0.10 Dickey-Fuller F-test

∆yt = β0 + γyt−1 + ut H0 : β0 = γ = 0

∆yt = β0 + β2t+ γyt−1 + ut H0 : β0 = β2 = γ = 0

Now of course the test statistic is distributed under F test which can be found in DickeyFuller tables. They are calculated under conventional F tests.

2.0.11 ADF: Augemented Dickey Fuller Test

∆yt = β0 + γyt−1 + ut

Granger points out that if the above equation has serial correlation then the test can have nomeaning. He suggested that the lags of ∆yt’s should be used to remove the serial correlation.

50


This augmented test is known as Augmented Dickey Fuller test(ADF).

∆yt = β0 + γyt−1 +

p∑i=1

αi∆yt−i

∆yt = β1 + β2t+ γyt−1 +m∑i=1

αi∆yt−i + ut

With Dickey-Fuller (ADF) test we can handle with the autocorrelation problem. Them, number of lags included, should be big enough so that the error term is not seriallycorrelated. The null hypothesis is again the same. Let us consider GDP example again

Figure 2.15: Augmented Dickey Fuller Test


For above figure, at 99% confidence level, we can not reject the null.

51



For above figure, at 99% confidence level, we reject the null. This time we “augmented”the regression to handle with serial correlation. Note that, because GDP is not stationary atlevel and stationary at first difference,it is called integrated order one, I(1). Then a stationaryserie is I(0).

∆yt = β1 + β2t+ γyt−1 +

p∑i=1

αi∆yt−i + ut

In order to handle the autocorrelation problem, Augmented Dickey-Fuller (ADF) testis proposed. The p, number of lags included, should be big enough so that the error term isnot serially correlated. So in practice we use either SBC or AIC to clean the residuals. Thenull hypothesis is again the same.

∆yt = γyt−1 +

p∑i=1

αi∆yt−i + εt H0 : γ = 0

∆yt = δ + γyt−1 +

p∑i=1

αi∆yt−i + εt H0 : δ = γ = 0

∆yt = δ + γyt−1 + φt+

p∑i=1

αi∆yt−i + εt H0 : φ = δ = γ = 0

Example: Daily Brent Oil

52


Figure 2.18: Augmented Dickey Fuller Test Daily Brent Oil

We can not reject the null of unit root. So Crude levels may behave like RW.

Figure 2.19: Correlogram of Interest Rates

Figure 2.20: Short and Long rates: Trl30 and 360

53

2.1. QUESTIONS CHAPTER 2. TESTING FOR STATIONARITY AND UNIT ROOTS

I(1) and I(0) Series

If a series is stationary it is said to be I(0) series. If a series is not stationary but its firstdifference is stationary it is called to be difference stationary or I(1). Next section willinvestigate the stationarity behaviour of more than one time series known as co-integration.

2.1 Questions

1. Descriptive ANALYSIS OF TIME SERIES (STYLIZED FACTS)

a In the following data set you are given US, and Turkish GDP growth rates andinflation data.

b In addition, download both the GDP and ınflation for

i. EU as one country

ii. Germany

iii. South Korea, Soth Africa,Brasil, India and China

c GDP growth data are quarterly (like the one used in my data set Turkish and US,GDP growth series quarterly),

d Inflation data is monthly observed (so use monthly US, EU inflation rates (source:(ECB or Eurostat, OECD or any other source. (Year on Year)

e Similar credit rating countries such as (Brasil, South Africa, India) can constituteone group which is comparable with Turkey.

f Find the sample mean, sample variance, skewness, and excess kurtosis estimateson these data compare your findings on different years and US Turkey comparison.

g For Turkey divide the sample into two (after April 2001 and before).

h Sharp Ratio is given as E(X)/Variance(X) is a critical performance measure. Lookat the Turkish and US GDP and briefly comment on it.

i Calculate autocovariance and autocorrelations (autocorrelation between y(t) andy(t-i), i=1,...50, for quarterly and monthly series seperately). (use the Matlabcode attached or translate it into R or use Stata, EVIEWS)

j Fit an AR(1) for all of the four series (Brasil, South Africa, India and Turkey)and compare your results.

k Compare the uncoditional mean of the Turkish GDP and inflation series.

l Compare the uncoditional mean of the Turkish GDP and inflation series.

m Compare the Turkish inflation series with respect to others (i.e try to answerfor similarities and differences between emerging market countries and developedmarkets).

2. AR(1) simulation (Given the following model):

yt = φ0 + φ1yt−1 + εt

54

CHAPTER 2. TESTING FOR STATIONARITY AND UNIT ROOTS 2.1. QUESTIONS

Simulate the AR(1) process when φ1 = 1, 0.95, 0.85, 0.5, 0.25, 0.10

for two cases with or without drift term (i.e. )

a for two cases with or without drift term (i.e. φ0 = 0, 1

b a. What are the differences and similarities of the time series behaviour of theseAR series?

c What can be said about the unconditional mean of each series. Compare yoursimulated AR(1) sample average and the theoretical unconditional means of AR(1)model. (as we did in our lab session)

d Draw autocovariances for each of the series you generate.

e Estimate the Turkish GDP as an AR(1) model and use the coefficients for simu-lation. Compare the actual GDP and simulated AR(1).

3. By using the above series: estimate the AR(1”

,p) MA(1”q) and find the most suitable

ARMA(p,q) combination. By using Akaike Information Criterion (AIC) and BayesianInformation Criterion.

4. On the basis of your optimal lag order choice above do a forecasting exercise for GDPand Inflation for Turkey and US.

5. Testing with unit roots:

a Conduct the unit root tests (ADF) for the 5 of the (GDP or Inflation) data youhave used in PS1.

b State in a table which series are I(0) and I(1) or I(2) if any.

c If you find any of your series I(1) then conduct an ARIMA forecast.

6. 1. AR(1) simulation (Given the following model):

yt = φ0 + φ1yt−1 + εt

Simulate the AR(1) process when φ1 = 1, 0.99, 0.95, 0.90, 0.5

Note: In this question you need to simulate a total of 3× 5 = 15 time series.

a Without drift φ0 = 0

b With a drift φ0 = 1

c With drift and time trend

yt = φ0 + φ1yt−1 + φ2t+ εt

d Then conduct ADF tests each of these three specifications.

e In a table summarize your findings .

7. By using TUIK (or any other data source like TCMB) download total money supply,GDP (levels), exports and imports: (Use TL as the currency of GDP. )

55

2.1. QUESTIONS CHAPTER 2. TESTING FOR STATIONARITY AND UNIT ROOTS

a Test the existence of cointegration between money supply and gdp

b Test the existence of cointegration between exports and imports

c If there is co-integration relationship in the above case test the existence of errorcorrection mechanism.

d Clearly comment on the speed of adjustment coefficient.

56

Chapter 3

COINTEGRATION

Economic theory, implies equilibrium relationships between the levels of time series variablesthat are best described as being I(1). Similarly, arbitrage arguments imply that the I(1)prices of certain financial time series are linked. (two stocks, two emerging market bondsetc). If two (or more) series are themselves non-stationary (I(1)), but a linear combinationof them is stationary (I(0)) then these series are said to be co-integrated. Examples:

• Inflation and interest rates,

• Exchange Rates and inflation rates,

• Money Demand: inflation, interest rates, income

Figure 3.1: Consumption and Income Logs

57

CHAPTER 3. COINTEGRATION

Figure 3.2: Brent vs WTI

Figure 3.3: Crude oil Futures

58


Figure 3.4: Usd treasury 2 year vs 30 years

3.0.1 Money Demand Stability

mdt = β0 + β1rt + β2yt + β3inft + εt r: interest rates, y: income, infl: inflation.

Each series in the above eqn may be nonstationary (i.e. I(1)) but the money demandrelationship may be stationary. All of the above series may wander around individuallybut as an equilibrium relationship MD is stable. Or even though the series themselves maybe non-stationary, they will move closely together over time and their difference will bestationary. Consider the m time series variables y1,t, y2,t, . . . ym,t known to be non-stationary,ie. suppose

yi,t = I(1), i = 1, 2, 3, ...,m

Then, yt = (y1,t, y2,t, . . . , ym,t)′ are said to form one or more cointegrating relations if

there are linear combinations ofyi,t’s that are I (0)Where, r denotes the number of cointegrating vectors.

3.0.2 Testing for Cointegration Engle – Granger Residual-BasedTests Econometrica, 1987

Step1 Run an OLS regression of y1,t (say) on the rest of the variables: namely y2,t, y3,t, . . . ym,tand save the residual from this regression .

y1,t =m∑i=2

βiyi,t + ut

Dickey Fuller Test

59


ut = β1ut−1 + εt

Dickey-Fuller unit root tests.

3.0.3 Residual Based Cointegration test: Dickey Fuller test

∆ut = δ + γut−1 where γ = β1 − 1

Hence

H0 : γ = 0

Therefore, testing for co-integration yields to test whether the residuals from a combinationof I(1) series are I(0). If u is an I(0) then we conclude. Even the individual data series areI(1) their linear combination might be I(0). This means that there is an equilibrium vectorand if the variables divert from equilibrium they will converge there at a later date. If theresiduals appear to be I(1) then there does not exist any co-integration relationship implyingthat the inference obtained from these variables are not reliable.

Higher Order Integration

Higher order integration if two series are I(2) may be they might have an I(1) relation-ship.

3.0.4 Examples of Cointegration: Brent Wti Regression

Null Hypothesis: RESID01 has a unit root

Exogenous: Constant

Lag Length: 0 (Automatic - based on SIC, maxlag=12)

t-statistic Prob*Augmented Dickey-Fuller test statistic -4.226414 0.0009

Test Critical Values 1% level -3.4875505% level -2.88650910% level -2.580163

So we reject the null.

3.0.5 Example of ECM

The following is the ECM that can be formed,

∆yt = α + β∆xt − λ(ut−1)

λ is the speed of adjustment towards equilibrium.

λ < 0 is expected since error can be correction it is expected to lie between 0 and 1

ut−1: is the equilibrium error

60


Estimation of ECM

ECM: λ: speed of adjustment coefficient

ut−1 = (yt−1 − βxt−1) equilibrium error

If the system hits a random shock the λ will push the system back to equilibrium. Thesign and the magnitude of the λ will be the main determinants of ECM. It is negative andthe size shows the speed with which error corrects.

3.0.6 Error Correction Term

The error correction term tells us the speed with which our model returns to equilibrium fora given exogenous shock. It should have a negative sign, indicating a move back towardsequilibrium, a positive sign indicates movement away from equilibrium. The coefficientshould lie between 0 and 1, 0 suggesting no adjustment one time period later, 1 indicatesfull adjustment.

An Example Are Turkish interest rates with different maturities (1 month versus 12months) co-integrated ?

Step 1: Test for I(1) for each series.

Step 2: Test whether two of these series move together in the long-run. If yes thenset up an Error Correction Mechanism.

Figure 3.5: TRLGOV30 TRLGOV360

Figure 3.6

61


Figure 3.7

So both of these series are non-stationary, i.e I(1). Now we test whether there exists alinear combination of these two series which is stationary.

r360t = βr30

t + ut

Both r360t and r30

t are I(1) and test is I(0)

Run another ADF on

∆ut = δ + γut−1 + εt

Test for Cointegration

Figure 3.8: Test for co-integration

Estimate the ECM

r360t = βr30

t + ut

Both r360t and r30

t are I(1) and εt is I(0). Then we have an equilibrium relationship whichcan be given as ECM:

∆r360t = α + λ(r360

t−1 − βr30t−1) + εt

∆r360t = −0.0032− 0.099874(r360

t−1 − 1.1r30t−1)

62


Figure 3.9: Residual Actual Fitted

Figure 3.10: ECM Regression

3.0.7 Use of Cointegration in Economic and Finance

• Purchasing Power Parity: FX rate differences between two countries is equal to inflationdifferences. Big Mac etc. . .

• Uncovered Interest Rate Parity: Exchange rate can be determined with the interestrate differentials

• Interest Rate Expectations: Long and short rate of interests should be moving together.

• Consumption Income

• HEDGE FUNDS! (ECM can be used to make money!)

3.0.8 Conclusion

• Test for co-integration via ADF is easy but might have problems when the relationshipis more than 2-dimensional (Johansen is more suitable)

• Nonlinear co-integration, Near unit roots, structural breaks are also important.

• But stationarity and long run relationship of macro time series should be investigatedin detail.

63


64

Chapter 4

Multiple Equation Systems

Outline:

• Simultaneous Equations

• structural versus reduced for models

• Inconsistency of OLS

• Simultaneous Equations in Matrix Form

• VAR Models

4.1 Seemingly Unrelated Regression Model

y1,t = β1,0 + β1,1xt + u1,t

y2,t = β2,0 + β1,2xt + u1,t

ut = (u1,t, u2,t)T

ut : iidN

[(00

)(σ2

1 σ1,2

σ2,1 σ22

)]X’s are exogenous and disturbances are comptemporenously correlated.

y1,t = β1,0 + β1,1x1,t + β1,2y2,t + u1,t

y2,t = β2,0 + β2,1x2,t + β2,2y1,t + u2,t

ut = (u1,t, u2,t)T

ut : iidN

[(00

)(σ2

1 σ1,2

σ2,1 σ22

)]Here both endogenous variables are incorporated into the systems (y1 and y2).

65

4.1. SEEMINGLY UNRELATED REGRESSION MODELCHAPTER 4. MULTIPLE EQUATION SYSTEMS

4.1.1 Recursive Systems

y1,t = β1,3x1,t + u1,t

y2,t = β2,2y1,t + β2, 3x2,t + u2,t

y3,t = β3,1y1,t + β3,2y2,t + β3, 3x3,t + u3,t

ut : iidN

[000

σ21 0 0

0 σ22 0

0 0 σ23

]

4.1.2 Structural and Reduced Forms

y1,t − β1y2,t = u1,t

−β2y1,t + y2,t − α = u2,t

This model is known as the structural model which can be represented as

Byt + Axt = ut

yt =

(y1,t

y2,t

), B =

(1 −β1

−β2 1

), A =

(0−α

), ut =

(u1,t

u2,t

)Reduced Form

In order to express everything in terms of y:

yt = −B−1Axt +B−1ut

yt = Πxt + vt

Π = −B −−1A

vt = B−1ut

Why simultaneous equations matter? it matters because it avoids the endogeneity.So we can avoid inconsistent estimators. In addition, in the past they were very useful tomacroeconomic forecasting. But after 1980’s their forecasting power turned out to be ratherweak.

66

CHAPTER 4. MULTIPLE EQUATION SYSTEMS4.1. SEEMINGLY UNRELATED REGRESSION MODEL

4.1.3 Some Simultaneous Equation Models

Demand and Supply Model:

Qdt = α0 + α1Pt + u1,t α1 < 0

Qst = β0 + β1Pt + u2,t β1 > 0

Qdt = Qs

t

Figure 4.1: Some Simultaneous Equation Models


67



4.1.4 Keynesian Income Function

Yt = Ct + It

Ct = β0 + β1Yt + ut


4.1.5 Inconsistency of OLS

OLS may not be applied to estimate a single equation embedded in a system of simultaneousequations if one or more of explanatory variables are correlated with the disturbance termin that equation Result: the estimators thus obtained are inconsistent.

Let us consider the previous model.

68


By substituting C

Yt = β0 + β1Yt + It + ut

Yt =β0

1− β1

+It

1− β1

+ut

1− β1

E[Yt] =β0

1− β1

+It

1− β1

cov(Yt, ut) = E[(Yt − E[Yt])(ut − E[ut])

]cov(Yt, ut) = E[

u2t

1− β1

]

Since Y − E[Yt] = ut1−β1 and ut − E[ut] = ut

cov(Yt, ut) =σ2

1− β1

β1 =

∑nt=1(Ct − C)(Yt − Y )∑n

t=1(Yt − Y )2

β1 =

∑nt=1Ctyt∑nt=1 y

2t

β1 =

∑nt=1(β0 + β1Yt + ut)yt∑n

t=1 y2t

β1 =β0

∑nt=1 yt∑n

t=1 y2t

+β1

∑nt=1 Ytyt∑nt=1 y

2t

+

∑nt=1 utyt∑nt=1 y

2t

β1 = β1 +

∑nt=1 utyt∑nt=1 y

2t

E[β1] = β1 + E

[∑nt=1 utyt∑nt=1 y

2t

]We cannot evaluate it via E(.) operator since expectation operator is a linear one. But

we also know that u and Y are not independent.

plim(β1) = plim(β1) + plim

(∑nt=1 utyt∑nt=1 y

2t

)

plim(β1) = plim(β1) + plim

((∑n

t=1 utyt/n)

(∑n

t=1 y2t /n)

)

69


plim(β1) = β1 +σ2/1− β1

σ2Y

plim(β1) = β1 +1

1− β1

σ2

σ2Y

So inconsistent.

cov(Yt, ut) =σ2

1− β1

By substituting C we obtained;

Yt =β0

1− β1

+It

1− β1

+ut

1− β1

A reduced for equation is one that expresses an endogenous variable solely in terms ofpredetermined variables and stochastic disturbance.

So if we re-write this as;

Yt = Π0 + Π1It + wt

where Π0 = β01−β1 ; Π1 = 1

1−β1 ; wt = ut1−β1

This is reduced form equation for Y. By applying same way we can derive reduced formfor C too.

4.1.6 Underidentification

Consider the Demand & Supply model

Qst = β0 + β1Pt + u2,t β1 > 0

Qdt = α0 + α1Pt + u1,t α1 < 0

Qdt = Qs

t

α0 + α1Pt + u1,t = β0 + β1Pt + u2,t

Reduced forms;

(α1 − β1)Pt = β0 − α0 + u2t − u1t

Pt =β0 − α0

α1 − β1

+u2,t − u1,t

α1 − β1

Pt = Π0 + vt

70


Qt = α0 + α1

(β0 − α0

α1 − β1

+u2,t − u1,t

α1 − β1

)+ u1t

Qt =α1β0 − α1α0 + (α1 − β1)α0

α1 − β1

+α1u2t − α1u1t + (α1 − β1)u1t

α1 − β1

Qt =α1β0 − α0β1

α1 − β1

+α1u2t − β1u1t

α1 − β1

Qt = Π1 + wt

Now we have 2 reduced form parameters which include all four structural parameters.So we have 2 equations and 4 unknowns Then there is no unique solution If we regressreduced forms what we would have is only the mean values of price and quantity, nothingmore! We can not identify the demand or supply function.

4.1.7 Test for Simultaneity Problem

Hausman Specification Problem

Demand: Q = α0 + α1P + α2I + α3R + u1

Supply: Q = β0 + β1P + u2

Estimation of Simultaneous Equations

Figure 4.5: Estimation of Simultaneous Equations

4.1.8 Simultaneous Equations in Matrix Form

Full Information Maximum Likelihood(FIML) Estimation:We have:

Γyt + Cxt = ut

71


or The likelihood function is given as

yt = Πxt + vt vt ∼ iidN(0,Ω) where Ω = Γ−1Σ(Γ−1)

L = (2π)−n|Ω|−n/2exp

[− 1/2

T∑t=1

(yt − Πxt)TΩ−1(yt − Πxt)

]Consistent estimates are available with FIML, however FIML is very sensitive to correct

specification of the system.

72

Chapter 5

Vector Autoregression VAR

In 1980’s proposed by Christopher Sims is an econometric model used to capture the evolutionand the interdependencies among multiple economic time series. Generalizes the univariateAR models. All the variables in the VAR system are treated symmetrically . (by own lagsand the lags of all the other variables in the model)

VAR models as a theory-free method to estimate economic relationships. They consitutean alternative to the ”identification restrictions” in structural models .

5.0.1 Why VAR

Figure 5.1: Christoffer Sims, from Princeton (nobel prize winner 2011) First VAR paper in1980

5.0.2 VAR Models

In Vector Autoregression specification, all variables are regressed on their and others laggedvalues.For example a simple VAR model is

y1t = m1 + a11y1,t−1 + a12y2,t−1 + ε1t

y2t = m2 + a21y1,t−1 + a22y2,t−1 + ε2t

or (y1t

y2t

)=

(m1

m2

)+

(a11 a12

a21 a22

)(y1,t−1

y2,t−1

)+

(ε1t

ε2t

)73

CHAPTER 5. VECTOR AUTOREGRESSION VAR

Which is called VAR(1) model with dimension 2

yt = m+ Ayt−1 + εt

Generally VAR(p) model with k dimension is

yt = m+ A1yt−1 + A2yt−2 + ...+ Apyt−p + εt

Where each Aiis a k × k matrix of coefficients, m and εt is the k × 1 vectors.Furthermore,E[εt] = 0 for all t and E[εtε

Ts ] = ω for t = s

E[εtε′s] = 0 for t 6= s → No serial correlation but there can be contemporaneous corre-

lations.

5.0.3 An Example VAR Models: 1 month 12 months TRY Interestrates monthly

DUPELICATEDGenerally VAR(p) model with k dimension is

yt = m+ A1yt−1 + A2yt−2 + εt

where each Ai is a k × k matrix of coefficients, m and εt is the k × 1 vectors.Furthermore,E[εt] = 0 for all t and E[εtε

′s] = ω for t = s

E[εtε′s] = 0 for t 6= s → No serial correlation but there can be contemporaneous corre-

lations.

Figure 5.2: Regression Output

A1 =

(0.78 −0.581.25 0.06

)A2 =

(0.06 0.50−0.28 −0.03

)Akaike Information Criterion : −4.089038 Schwarz Criterion : −3.914965

74

CHAPTER 5. VECTOR AUTOREGRESSION VAR

5.0.4 Hypotesis Testing

To test whether a VAR with a lag order 8 is preferred to a log order 10.

(T − c)(log|∑r

| − log|∑u

|) ∼ χ2 df: # of restrictions

T : number of observationsc: number of parameters estimated in each equation of unresticted systemlog|

∑u | log of the determinant of

∑u

Impulse Response Functions

Suppose we want to see the reaction of our simple initial VAR(1) model to a shock, sayε1 = [1, 0]

′and the rest is 0 where,

A =

(0.4 0.10.2 0.5

)y0 = 0

y1 =

(10

)→ y2 = Ay1 + εt =

(0.4 0.10.2 0.5

)(10

)+

(00

)=

(0.40.2

)→ y3 = Ay2 =

(0.4 0.10.2 0.5

)(0.40.2

)+

(00

)=

(0.180.18

)

Figure 5.3: Response to Cholesky One S.D. Innovations ± 2 S.E.

75

5.1. QUESTIONS CHAPTER 5. VECTOR AUTOREGRESSION VAR

5.1 Questions

1. Descriptive ANALYSIS OF TIME SERIES (STYLIZED FACTS)

a In the following data set you are given US, and Turkish GDP growth rates andinflation data.

b In addition, download both the GDP and ınflation for

i. EU as one country

ii. Germany

iii. South Korea, Soth Africa,Brasil, India and China

c GDP growth data are quarterly (like the one used in my data set Turkish and US,GDP growth series quarterly),

d Inflation data is monthly observed (so use monthly US, EU inflation rates (source:(ECB or Eurostat, OECD or any other source. (Year on Year)

e Similar credit rating countries such as (Brasil, South Africa, India) can constituteone group which is comparable with Turkey.

f Find the sample mean, sample variance, skewness, and excess kurtosis estimateson these data compare your findings on different years and US Turkey comparison.

g For Turkey divide the sample into two (after April 2001 and before).

h Sharp Ratio is given as E(X)/Variance(X) is a critical performance measure. Lookat the Turkish and US GDP and briefly comment on it.

i Calculate autocovariance and autocorrelations (autocorrelation between y(t) andy(t-i), i=1,...50, for quarterly and monthly series seperately). (use the Matlabcode attached or translate it into R or use Stata, EVIEWS)

j Fit an AR(1) for all of the four series (Brasil, South Africa, India and Turkey)and compare your results.

k Compare the uncoditional mean of the Turkish GDP and inflation series.

l Compare the uncoditional mean of the Turkish GDP and inflation series.

m Compare the Turkish inflation series with respect to others (i.e try to answerfor similarities and differences between emerging market countries and developedmarkets).

2. AR(1) simulation (Given the following model):

yt = φ0 + φ1yt−1 + εt

Simulate the AR(1) process when φ1 = 1, 0.95, 0.85, 0.5, 0.25, 0.10

for two cases with or without drift term (i.e. )

a for two cases with or without drift term (i.e. φ0 = 0, 1

76

CHAPTER 5. VECTOR AUTOREGRESSION VAR 5.1. QUESTIONS

b a. What are the differences and similarities of the time series behaviour of theseAR series?

c What can be said about the unconditional mean of each series. Compare yoursimulated AR(1) sample average and the theoretical unconditional means of AR(1)model. (as we did in our lab session)

d Draw autocovariances for each of the series you generate.

e Estimate the Turkish GDP as an AR(1) model and use the coefficients for simu-lation. Compare the actual GDP and simulated AR(1).

3. By using the above series: estimate the AR(1”

,p) MA(1”q) and find the most suitable

ARMA(p,q) combination. By using Akaike Information Criterion (AIC) and BayesianInformation Criterion.

4. On the basis of your optimal lag order choice above do a forecasting exercise for GDPand Inflation for Turkey and US.

5. Testing with unit roots:

a Conduct the unit root tests (ADF) for the 5 of the (GDP or Inflation) data youhave used in PS1.

b State in a table which series are I(0) and I(1) or I(2) if any.

c If you find any of your series I(1) then conduct an ARIMA forecast.

6. 1. AR(1) simulation (Given the following model):

yt = φ0 + φ1yt−1 + εt

Simulate the AR(1) process when φ1 = 1, 0.99, 0.95, 0.90, 0.5

Note: In this question you need to simulate a total of 3× 5 = 15 time series.

a Without drift φ0 = 0

b With a drift φ0 = 1

c With drift and time trend

yt = φ0 + φ1yt−1 + φ2t+ εt

d Then conduct ADF tests each of these three specifications.

e In a table summarize your findings .

7. By using TUIK (or any other data source like TCMB) download total money supply,GDP (levels), exports and imports: (Use TL as the currency of GDP. )

a Test the existence of cointegration between money supply and gdp

b Test the existence of cointegration between exports and imports

c If there is co-integration relationship in the above case test the existence of errorcorrection mechanism.

d Clearly comment on the speed of adjustment coefficient.

77

5.1. QUESTIONS CHAPTER 5. VECTOR AUTOREGRESSION VAR

78

Chapter 6

Asymptotic Distribution Theory

Econometric theory very much involves with what happens to the parameter uncertanintywhen we can observe a very large data set. We will discuss the following concepts.

• Convergence in Probability

• Laws of Large Numbers

• Convergence of Functions

• Convergence in Distribution: Limiting Distributions

• Central Limit Theorems

• Asymtotic Distributions

6.1 Why large sample ?

• We have studied the finite (exact) distribution of OLS estimator and its associatedtests.

• If the regressors are endogeneous (i.e. X and u are correlated) we won’t be able tohandle the estimation.

• Rather than making assumptions on a sample of a given size, large sample theorymakes assumptions on the stochastic processes that generates the sample.

6.1.1 Asymptotics

• What happens to a rv or a distribution as n tends to infinity.

• What is the approximate distribution under these limiting conditions.

• What is the rate of convergence to a limit.

• 2 Critical Laws of statistics are studied under large sample theory.

79

6.2. CONVERGENCE IN PROBABILITYCHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY

– Law of Large Numbers

– Central Limit Theorem

6.2 Convergence in Probability

Definition : Let xn be a sequence random variable where n is sample size, the randomvariable xn converges in probability to a constant c if

limn→∞

Prob(|xn − c| > ε) = 0

for any fositive ε

• the values that the x may take that are not close to c become increasingly unlikely asn increases.

• If xn converges to c, then we say, plimxn = c

• All the mass of the probability distribution concentrates around c.

6.2.1 Mean Square Convergence

Definition mean square convergence:A stronger condition than convergence in probability is mean square convergence.A random sequence xn is said to converge in mean square to c which can be formulated

as

limn→∞

E[(xn − c)2] = 0

xnms−→ c

6.2.2 Convergence in Probability

Definition :An estimator θn of a parameter θ is a consistent estimator iff

plimθn = θ

6.2.3 Almost sure convergence

The random variable xn converges almost surely to the constant c if and only if

limn→∞

P (|xi − c| > ε) for all i ≥ n) = 0 for ε > 0

It can be written asxn

AS−→ c

80

CHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY6.3. LAW OF LARGE NUMBERS

Intuitively : once the sequence xn gets ccloser to c it stays that way.

Almost sure convergence:alternative definition

The random variable xn is said to converge ’almost surely’ to c if and only if

prob( limn→∞

xn = c) = 1

6.3 Law of large numbers

Law of large numbers is one of the key concepts in probability theory and statistics. Twomain versions are strong law of large numbers and weak law of large numbers.

Weak Law of large numbers : Based on convergence in probability

Strong form of large numbers : Based on Almost sure convergence

Laws of Large Numbers

1. Weak Law of large Numbers (Khinchine)

if xi, i = 2, 3, .., n is a random i.i.d. sample from a distribution with finite meanE(xi) =µ

plimxn = µ

Remarks:

a No finite variance assumption

b Requires i.i.d. sampling

2. Strong Law of Large Numbers (Kolmogorov)

if xi, i = 2, 3, .., n is a random i.i.d. random variables such that E(xi) < µ <∞(finitemean) and var(xi) = σ2

i <∞

then xn − µnAS−→ 0 Remarks:

• It is stronger because of in AS convergence iid’ness is not required

• But with iid”ness we will obtain a convergence to a constant mean

6.3.1 Convergence in Functions

Theorem (Slutsky : For a continious function, g(xn)

plim(g(xn) = g(plim(xn)

Using Slutsky theorem, we can write some rules of plim.

81

6.4. CONVERGENCE IN DISTRIBUTION : LIMITING DISTRIBUTIONSCHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY

6.3.2 Rules for Probability Limits

1. For plimx = c and plimy = d

• plim(x+ y) = c+ d

• plimxy = cd

• plim(x/y) = c/d

2. For matrices X and Y with plimX = A and plimY = B

• plimX−1 = A−1

• plimXY = AB

6.4 Convergence in Distribution : Limiting Distribu-

tions

Definition:

xn with cdf Fn(X) converges in distribution to a random variable with cdf F (x) if

limn→∞

|Fn(x)− F (x)| = 0 for all continuity points of F (x)

then F(x) is the limitin distribution of xn and can be shown as

xnd−→ x

6.4.1 Example Limiting Probability Distributions

Student’s t Distribution:

Given that Z ∼ N(0, 1) and X ∼ χ2n

t =Z√X/n

has student’s t distribution with n degrees of freedom.

Properties:

• It is symmetric, positive everywhere and leptokurtic(has fatter tails than normal dist.)

• Only parameter is n, degrees of freedom.

• E(t) = 0 and V ar(t) = n/(n− 2)→ n→∞⇒ t→ N(0, 1)

6.4.2 Rules for Limiting Distributions

:

82

CHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY6.5. CENTRAL LIMIT THEOREMS

1. If xnd−→ x and plimyn + c then;

xnynd−→ cx

xn + ynd−→ c+ x

xn/ynd−→ c/x

2. As a corrolary to Slutsky theorem, if xnd−→ x and g(x) is a continous function

g(xn)d−→ g(x)

3. If yn has a limiting distribution and plim(xn − yn) = 0 then xn has the same limitingdistibution with yn

6.5 Central Limit Theorems

Lindberge-Levy Central Limit Theorem CLT states that any sampling distribution from anydistribution would converge to normal distribution.

The mean of a sufficiently large number of independent random variables, each withfinite mean and variance, will be approximately normally distributed.

The sample mean xn for an iid sequence has a degenerate probability distributions asn→ µ.

For this purpose we would like to investigate√n(xn − µ)

xn = 1/n∑

xi

E[√n(xn − µ)2] = n/n2var(

∑xi) = 1/nvar(

∑xi) = σ2

Therefore in contrast to xn,√n(xn)− µ) may converge to nondegenerate distribution.

√n(x− µ)

d−→ N(0, σ2)

6.5.1 Example: distirbution of sample mean

X1, X2, .., XN can be viewed as N variables each having the same distribution mean andvariance.

X1 : Stands for all possible values that can be obtained by the first variable drawn fromX,

X2 : Second variable, and so on..

X = 1/NN∑i

Xi = 1/N(X1 +X2 + ...+XN)

83

6.5. CENTRAL LIMIT THEOREMSCHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY

So X’s are said to be identically and independently distributed (iid) random variables.Derive the distribution of

E[X] = 1/NEN∑i

Xi = 1/NE(X1 +X2 + ..+XN)

E[X] = 1/N [E(X1) + E(X2) + ..+ E(XN)]

sinde (X1, X2, .., XN) are all iid meaning to have the same mean distribution mean andvariance.

1/N [µ+ µ+ ...+ µ] = µ

This means the sample mean of X has is equal to population mean µ.

Central Limit Idea

σ2X = var(X) = var[1/N

N∑i

Xi] = 1/N2var(N∑i

Xi)

since X1, X2, ..XN are all iid meaning to have the same variance and no covariances.

σ2X = 1/N2var(

N∑i

Xi)

σ2X = 1/N2[var(X1) + ..+ var(XN)] = 1/N2[σ2 + ...+ sigma2]

σ2X = σ2/N

So X ∼ N(µ, σ2) is the sampling distribution

[X ∼ N(µ, σ2/n)]

6.5.2 A simple monte carlo for CLT

see : clt.m MATLAB code

LetX1, X2. . . Xnare N(0, 1),

AsE(Xbar) = mu

n→∞Stdev(X)→ 0

84

CHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY6.5. CENTRAL LIMIT THEOREMS

Figure 6.1: Logistic Function

Figure 6.2: X = (x1)x1 ∼ N(0, 1), E(X = −0.0182) and σ = 0.98

Figure 6.3: CLT when n = 5, E(X = 0.01) and σ = 0.47

85

6.5. CENTRAL LIMIT THEOREMSCHAPTER 6. ASYMPTOTIC DISTRIBUTION THEORY

Figure 6.4: CLT when n = 1000, E(X = 0.002) and σ = 0.031

Figure 6.5: Stdev of sample average disappears

86

Chapter 7

references

Chatfield, C (1992),The Analysis of Time Series, Chapman and HallEnders W (2014), Applied Econometric Time Series, fourth edition, WileyDiebold, F (2007), Elements of Forecasting, Thompson South Western , Greene Hayashi

Hamilton J (1994), Time Series Methods,Princeton UniversityHarvey, A (1993), Time Series Models,Harvester WheatsheafMills, T(1990), Time Series Techniques for Economists,Cambridge University Press.Pindyck R and Rubinfeld D (1998), Econometric Models and Economic Forecasts, Mc

Graw Hill, 4th edition.Van Diyck, Nonlinear Time Series,Tsay, R (2002),Analysis of Financial Time Series,Wiley.

87

Time Series Econometrics Lecture Notes - econ.boun.edu.tr Note-Tim… · Time Series Econometrics...

Documents

Transcript of Time Series Econometrics Lecture Notes - econ.boun.edu.tr Note-Tim… · Time Series Econometrics...