STAD57 Time Series

Analysis

Lecture 21

1

GARCH Model Extensions

There are many variations on GARCH models

geared towards different data characteristics

E.g. Integrated / Seasonal / Fractionally Integrated

GARCH model, Exponential GARCH model, etc

One common extension is to model the

conditional mean of a series with ARMA, and

its conditional variance with GARCH

E.g. ARMA with GARCH(r,s) errors

2 2 2 2

0 1 1

, ~ (0,1)( ) ( ) , where

iid

t t t t

p qt t

t j t j k t kj k

Z NB X B Z

Z

Example

Fit ARMA(1,2) w/ GARCH(3,2) errors in R

3

Estimate Std. Error t value Pr(>|t|)

mu -1.244e-04 6.574e-05 -1.893 0.058365 .

ar1 7.513e-01 8.771e-02 8.565 < 2e-16 ***

ma1 -7.223e-01 9.055e-02 -7.977 1.55e-15 ***

ma2 -8.961e-02 3.135e-02 -2.859 0.004255 **

omega 3.388e-06 1.373e-06 2.467 0.013626 *

alpha1 1.000e-08 2.645e-02 0.000 1.000000

alpha2 6.892e-02 3.126e-02 2.205 0.027480 *

alpha3 1.362e-01 3.704e-02 3.678 0.000235 ***

beta1 1.082e-01 1.637e-01 0.661 0.508556

beta2 6.601e-01 1.542e-01 4.281 1.86e-05 ***

AIC BIC SIC HQIC

-6.566089 -6.516973 -6.566287 -6.547421

garchFit(~arma(1,2) + garch(3,2),spr)

model specification

Example

4

Forecasts for ARMA(1,2) w/ GARCH(3,2) errors

Time Series Analysis: Time &

Frequency Domain

In general, there are two complementary

ways to analyze time series:

Time Domain Analysis

Describing how process evolves in time

Frequency Domain Analysis

Describing process as (random) combination of

periodic functions with different frequencies

Two ways are equivalent for stationary series,

but each is more appropriate for different tasks

5

Time Domain

In time domain, goal is to model evolution of

series in time, as function of its past

Already seen SARIMA models, for which function

F() is linear. More generally, can extended

method to nonlinear functions

Best suited for modeling auto-/cross-

correlations, and especially for forecasting

Commonly used for statistical, business, financial,

and economic applications 6

1 1, , , , ,t t t p t t qX F X X W W

Frequency Domain

In frequency domain, goal is to model series

as random composition of periodic functions

with different frequencies

Best suited for modeling series with strong

periodic behavior over several frequencies

E.g. radio signals, vibrations, sounds etc.

Commonly used in engineering & physical sciences 7

( ), is RV

where: is fn w/ freq.

k

t k kkk k

UX U f t

f

Frequency Domain

Purpose of frequency domain analysis is to

decompose process (a.k.a. signal) into its

constituent frequencies

Given , want to find frequencies

ωk (of functions fk) and the magnitude of their

associated RV’s Uk

E.g. Voice recognition application: want to map

speech (sound signals) into words, by analyzing

each syllable’s characteristic frequencies

8

( )t k kkX U f t

Example

Speech signal (syllable aaa…hhh sampled at

10,000 points/sec)

9 0 200 400 600 800 1000

01

00

02

00

03

00

04

00

0

Example

Note: complicated ACF/PACF, would require very

large SARIMA model in time domain analysis 10

0 50 100 150 200 250 300

-0.5

0.0

0.5

1.0

0 50 100 150 200 250 300

-0.5

0.0

0.5

Pa

rtia

l A

CF

ACF PACF

Periodic Signals

The building blocks of frequency domain are

the trigonometric functions (sines & cosines)

Periodic functions in given by:

Look at trigonometric fn’s because any periodic fn

can be written as linear combination of sines &

cosines w/ different frequencies (Fourier series) 11

( )

( )

1 2

cos( ) cos( ) cos( ) sin( ) sin( )

( ) cos(2 )

cos(2 ) sin(2 ), for

k k k

k k k k

a b a b a b

f t U t

U t U t

( )k kkU f t

1

2

cos( )

sin( )

k k

k k

U U

U U

Periodic Signals

For function define:

ω = frequency (# cycles per unit time)

1/ω = period (# unit times per cycle)

U = amplitude (max height of cosine)

φ = phase (start of cosine function)

In terms of , we have:

amplitude:

phase:

12

cos(2 )U t

1 2cos(2 ) sin(2 )U t U t 2 2

1 2U U U

1

2 1tan /U U

1

2

cos( )

sin( )from

U U

U U

0.0 0.5 1.0

-2-1

01

2

Example

13

t

( ) 2 cos(2 2 ) 2 sin(2 2 )f t t t

phase φ

amplitude period

Periodic Series

Consider process defined as:

Process is a sum of fixed periodic functions at

frequencies ωk, with random independent weights

The variances σk2 control the magnitude of the

weights Uk1 & Uk2 at frequencies ωk

14

1 21cos(2 ) sin(2 )

q

t k k k kkX U t U t

1 2

2

1 2 1 2[ ] [ ] 0 & [ ] [ ]

where, at frequency , RV's , are independent

with

k k k

k k k k k

U U

E U E U Var U Var U

Example

Series with q=1, ω1=.06, σk2=1

15 0 20 40 60 80 100

-0.5

0.0

0.5

Example

Series with q=2,

16

2

1 1

22 2

.06 1,

.1 2

0 20 40 60 80 100

-2-1

01

2

t

0 20 40 60 80 100

-3-1

13

Xt (sum)

ω2, σ22

ω1, σ12

0 20 40 60 80 100

-2-1

01

2

t

0 20 40 60 80 100

-4-2

02

Example

Series with q=3,

17

2

11

2

2 2

23 3

1.06

.1 , 2

.4 .5

ω2, σ2

2

ω1, σ12

Xt (sum)

ω3, σ32

Autocovariance

For

with independent 0-mean & σk2-var Uk1,Uk2,

the autocovariance function is given by:

Proof:

18

1 21cos(2 ) sin(2 )

q

t k k k kkX U t U t

2 2

1 1( ) cos(2 ) (0)

q q

k k kk kh h

19

Example

Series with q=3,

Find theoretical ACF of series

20

2

11

2

2 2

23 3

1.06

.1 , 2

.4 .5

0 10 20 30 40

-0.5

0.0

0.5

1.0

Series X

0 10 20 30 40

-0.5

0.0

0.5

1.0

AC

F/A

CF

[1]

0 20 40 60 80 100

-4-2

02

Example (cont’d)

21

Series:

Sample

ACF

Theoretical

ACF

Periodic Series

Can actually represent any 0-mean stationary

process as superposition of (possibly infinite)

trigonometric functions with different ωk & σk2

E.g. To generate White Noise {Wt}, t=1,…,n, with

Var[Wt]=σw2, use q=n random components with:

Obviously:

Can also show that

22

22& , 1, ,w

k k

kk n

n n

2 2

1(0)

n

k wk

( ) 0, 1, , 1h h n

Example

23

0 50 100 150 200

-2-1

01

23

2 1& , 1, ,k k

kk n

n n

generated

WN(0,1)

series

(n=200)

0 5 10 15 20

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

5 10 15 20

-0.1

5-0

.05

0.0

50

.15

Pa

rtia

l A

CF

ACF PACF

Scaled Periodogram

Given series data (X1,…,Xn), how can we find

frequency domain representation of {Xt}?

Don’t know ωk or σk2 beforehand

Idea: look at large set of frequencies ωj and

try to estimate their corresponding σj2

Since we only have data for t=1,…,n, look at

periods of 1/ωj=1,…,n → ωj=1/n,…,n/n=1

For each frequency ωj=j/n, j=1,…,n, estimate Uj1,

Uj2 using regression, and σj2 by their squares’ sum

24

Scaled Periodogram

For fixed j in (1,…,n), we have

Regression model becomes:

This is like bivariate regression without intercept

From orthogonality of cos & sin, estimates are

given by correlations of Xt with cos & sin:

25

/j j n

1 2( / )cos(2 / ) ( / )sin(2 / )tX j n tj n j n tj n

11 12

1

12 12

1

cos(2 / ) 2ˆ ( / ) cos(2 / )cos (2 / )

sin(2 / ) 2ˆ ( / ) sin(2 / )sin (2 / )

n

ntttn t

t

n

ntttn t

t

X tj nj n X tj n

ntj n

X tj nj n X tj n

ntj n

0, / 2,j n n

Scaled Periodogram

For j=n/2:

For j=0,n:

Scaled Periodogram, P(j/n), is defined as:

P(j/n) measures magnitude of sin/cos coefficients

Can be seen as estimate of 2σj2 at freq. ωj=j/n

P(j/n) is symmetric around P(1/2)

26

1 21

ˆ ˆ(1/ 2) ( 1) / , & (1/ 2) 0n t

ttX n

1 1 2 21

ˆ ˆ ˆ ˆ(0) (1) / , & (0) (1) 0n

ttX n

2 2

1 2ˆ ˆ( / ) ( / ) ( / ), 1, ,P j n j n j n j n

/ 1 / , 0,1, ,P j n P j n j n

Periodogram

There is an easy way to calculate P(j/n)

without running all n/2 regressions, using the

Discrete Fourier Transform (DFT):

d(j/n) is complex-valued weighted average of {Xt}

27

( )

1

1 1

/ exp 2 / /

cos 2 / sin 2 / /

n

tt

n n

t tt t

d j n X i tj n n

X tj n i X tj n n

( )exp cos( ) sin( )ix x i x

Periodogram

Periodogram, I(j/n), is defined as squared

modulus of DFT coefficients:

Can calculate P(j/n) as:

Fast Fourier Transform (FFT) algorithm offers

highly efficient way of calculating d(j/n) →

I(j/n) → P(j/n) 28

2

2 2

1 1

( / ) ( / )

1 1cos 2 / sin 2 /

n n

t tt t

I j n d j n

X tj n X tj nn n

4

/ /P j n I j nn

Example

Series with q=3,

29

2

11

2

2 2

23 3

1.06

.1 , 2

.4 .5

Scaled

Periodogram

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4

Example

To calculate P(j/n) in R, use fft() function

30

> n=length(X)

> P = abs( 2 * fft(X) / n )^2

> Fr = (0:(n-1))/n

> plot(Fr, P, type='o')

series length

Fast Fourier transform

Scaled periodogram P(j/n)

corresponding frequencies (ωj=j/n)

Example

P(j/n) for speech signal

31 0 2000 4000 6000 8000 10000

02

00

00

40

00

06

00

00

Frequncy in Hz (cycles/sec)

human voice frequency

ranges ~ 300-3500Hz

Example

White Noise series

32 0.0 0.2 0.4 0.6 0.8 1.0

0.0

00

0.0

05

0.0

10

0.0

15

0.0

20

P(j/n) almost

uniform across

all frequencies