The Spectral Representation of Stationary Time Series.
-
Upload
marina-madewell -
Category
Documents
-
view
235 -
download
3
Transcript of The Spectral Representation of Stationary Time Series.
The Spectral Representation of Stationary Time Series
Stationary time series satisfy the properties:
1. Constant mean (E(xt) = )
2. Constant variance (Var(xt) = 2)
3. Correlation between two observations (xt, xt + h) dependent only on the distance h.
These properties ensure the periodic nature of a stationary time series
-15
-10
-5
0
5
10
15
0 20 40 60 80 100
and X1, X1, … , Xk and Y1, Y2, … , Yk are independent independent random variables with
sincos1
k
iiiiit tYtXx
222 and iii YEXE 0 ii YEXE
where 1, 2, … k are k values in (0,)
Recall
is a stationary Time series
With this time series
2
1
2. cosk
i ii
h h
and
1
3. cos0
k
i ii
hh w h
k
jj
iiw
1
2
2
where
1. 0tE x We can give it a non-zero mean, , by adding to the equation
We now try to extend this example to a wider class of time series which turns out to be the complete set of weakly stationary time series. In this case the collection of frequencies may even vary over a continuous range of frequencies [0,].
The Riemann integral
10
0
limb n
i i ix
ia
g x dx g c x x
The Riemann-Stiltjes integral
10
0
limb n
i i ix
ia
g x dF x g c F x F x
If F is continuous with derivative f then:
b b
a a
g x dF x g x f x dx If F is is a step function with jumps pi at xi then:
b
i iia
g x dF x p g x
First, we are going to develop the concept of integration with respect to a stochastic process.
Let {U(): [0,]} denote a stochastic process with mean 0 and independent increments; that is
E{[U(2) - U(1)][U(4) - U(3)]} = 0 for
0 ≤ 1 < 2 ≤ 3 < 4 ≤ .and
E[U() ] = 0 for 0 ≤ ≤ .
In addition let
G() =E[U2() ] for 0 ≤ ≤ and assume G(0) = 0.
It is easy to show that G() is monotonically non decreasing. i.e. G(1) ≤ G(2) for 1 < 2 .
Now let us define:
analogous to the Riemann-Stieltjes integral
0
).()( dUg
0
( ) ( ).g dF
Let 0 = 0 < 1 < 2 < ... < n = be any partition of the interval. Let .
Let idenote any value in the interval [i-1,i]Consider:
Suppose that and
there exists a random variable V such that
* 1max iin
n
iiiin UUgV
11
* )]()()[(
0lim n
n
0lim 2
VVE nn
Then V is denoted by:
0
).()( dUg
Properties:
0).()(0
dUgE
2
2
0 0
( ) ( ). ( ) ( ).E g dU g dG
0
21 ).()()( dGgg
0
2
0
1 ).()().()( dUgdUgE
1.
2.
3.
The Spectral Representation of Stationary Time Series
Let {X(): [0,]} and {Y(): l [0,]} denote a uncorrelated stochastic process with mean 0 and independent increments. Also let
F() =E[X2() ] =E[Y2() ] for 0 ≤ ≤ and F(0) = 0.
Now define the time series {xt : t T}as follows:
00
)()sin()()cos( dYtdXtxt
Then
00
)()sin()()cos( dYtdXtExE t
00
)()sin()()cos( dYtEdXtE
0
Also
00
)()sin()()cos( dYhtdXhtE
tht xxEh
00
)()sin()()cos( dYtdXt
0
00
)()cos()()cos( dXtdXhtE
00
)()sin()()sin( dYtdYhtE
00
)()sin()()cos( dYtdXhtE
00
)()cos()()sin( dXtdYhtE
0
)()cos()cos( dFtht
00)()sin()sin(0
dFtht
0
)cos()cos( tht
)()sin()sin( dFtht
0
)()cos( dFh
Thus the time series {xt : t T} defined as follows:
00
)()sin()()cos( dYtdXtxt
is a stationary time series with:
0 txE
0
)()cos( and dFhh
F() is called the spectral distribution function:
If f() = Fˊ() is called then is called the spectral density function:
00
)cos()()cos( dfhdFhh
Note
00
)(0 dfdFxVar t
The spectral distribution function, F(), and spectral density function, f() describe how the variance of xt is distributed over the frequencies in the interval [0,]
00
)cos( dfedfhh hi
The autocovariance function, (h), can be computed from the spectral density function, f(), as follows:
)sin()cos( iei
Also the spectral density function, f(), can be computed from the autocovariance function, (h), as follows:
1
1 10 cos( )
2 h
f h h
00
02
h
hh
Example: Let {ut : t T} be identically distributed and uncorrelated with mean zero (a white noise series). Thus
and
1
1 10 cos( )
2 h
f h h
if 2
2
Graph:
frequency
f( )
Example:
Suppose X1, X1, … , Xk and Y1, Y2, … , Yk are independent independent random variables with
sincos1
k
iiiiit tYtXx
222 and iii YEXE 0 ii YEXE
Let 1, 2, … k denote k values in (0,)
Then
k
iii hh
1
2 cos
If we define {X(): [0,]} and{Y(): [0,]}
ii i
ii
i YYXX::
)( and )(by
)( )()( 2
:
2
:
2
YEXVarXEFii i
ii
i
Note: X() and Y() are “random” step functions and F() is a step function.
k
iii h
1
2 cos
0
)()cos( dFhh
00
)()sin()()cos(then dYtdXtxt
sincos1
k
iiiii tYtX
2
1
Note: 0k
ii
0
)()cos( dFhh
0
)()cos( dfh
)()( and fF
Another important comment
In the case when F() is continuous
then
dfhdfhh s)cos()cos(0
in this case
0
)cos(1
02
1
hs hhf
Sometimes the spectral density function, f(), is extended to the interval [-,] and is assumed symmetric about 0 (i.e. fs() = fs (-) = f ()/2 )
It can be shown that
0
)cos(2
01
andh
hhf
Hence
From now on we will use the symmetric spectral density function and let it be denoted by, f().
dfhh )cos(
0
)cos(1
02
1 and
h
hhf
Linear Filters
sstst xay
Let {xt : t T} be any time series and suppose that the time series {yt : t T} is constructed as follows: :
The time series {yt : t T} is said to be constructed from {xt : t T} by means of a Linear Filter.
input xtoutput yt
Linear Filter
as
thty yyEh
Let x(h) denote the autocovariance function of {xt : t T} and y(h) the autocovariance function of {yt : t T}. Assume also that E[xt] = E[yt] = 0.Then: :
kktk
sshts xaxaE
s kktshtks xxaaE
s kktshtks xxEaa
s k
xks kshaa
s k
xks dfkshaa
cos
s k
xkshi
ks dfeaa
Re
dfeaa x
s k
kshiksRe
dfeaae x
s k
ksiks
hiRe
dfeaeae
k
kik
s
sis
hiRe
dfeae x
s
sis
hi
2
Re
dfeae xs
sis
hi
2
Re
dfeah xs
sis
2
cos
Hence
dfAhh xy
2cos
functionTransfereaAs
sis the
where
hy
of the linear filter
dfeah xs
sis
2
cos
Note:
dfAh x
2cos
x
s
sisxy feafAf
22
hence
hy
dfh ycos
Spectral density functionMoving Average Time series of order q, MA(q)
qtqtttt uuuux 2211
Let 0 =1, 1, 2, … q denote q + 1 numbers.
Let {ut|t T} denote a white noise time series with variance 2.
Let {xt|t T} denote a MA(q) time series with = 0.
Note: {xt|t T} is obtained from {ut|t T} by a linear filter.
Now 2
2
uf
u
q
s
sisux fefAf
2
0
2
Hence
2
0
2
2
q
s
sise
Example: q = 1
21
0
2
2
s
sisx ef
2
1
2
12
ie
ii ee 11
2
112
ii ee
121
2
12
cos212 1
21
2
Example: q = 2
22
0
2
2
s
sisx ef
2221
2
12
ii ee
2
212
21
2
112
iiii eeee
ii ee
21122
21
2
12 22
2ii ee
2cos2cos212 2211
22
21
2
3.02.01.00.00.0
0.1
0.2
0.3
0.4
0.5
0.6
3.02.01.00.00.0
0.1
0.2
0.3
0.4
0.5
0.6
3.02.01.00.00.0
0.1
0.2
0.3
0.4
Spectral Density fuction of an MA(2) Series
1 = 0.70
2 = -0.20
Spectral density function for MA(1) Series
Spectral density functionAutoregressive Time series of order p, AR(p)
2211 tptpttt uxxxx
Let 1, 2, … p denote p + 1 numbers.
Let {ut|t T} denote a white noise time series with variance 2.
Let {xt|t T} denote a AR(p) time series with = 0.
Note: {ut|t T} is obtained from {xt|t T} by a linear filter.
or 2211 tptpttt uxxxx
Now 2
2
uf
x
p
s
sisxu fefAf
2
1
21
Hence
x
p
s
sis fe
2
1
2
12
or
2
1
2
12
or
p
s
sis
x
e
f
Example: p = 1
21
1
2
12
s
sis
x
e
f
2
1
2
12
ie
ii ee 11
2
112
ii ee
12
1
2
12
cos212 12
1
2
Example: p = 2
xf
2cos2cos1212 22122
21
2
Example : Sunspot Numbers (1770-1869) 1770 101 1795 21 1820 16 1845 40 1771 82 1796 16 1821 7 1846 64 1772 66 1797 6 1822 4 1847 98 1773 35 1798 4 1823 2 1848 124 1774 31 1799 7 1824 8 1849 96 1775 7 1800 14 1825 17 1850 66 1776 20 1801 34 1826 36 1851 64 1777 92 1802 45 1827 50 1852 54 1778 154 1803 43 1828 62 1853 39 1779 125 1804 48 1829 67 1854 21 1780 85 1805 42 1830 71 1855 7 1781 68 1806 28 1831 48 1856 4 1782 38 1807 10 1832 28 1857 23 1783 23 1808 8 1833 8 1858 55 1784 10 1809 2 1834 13 1859 94 1785 24 1810 0 1835 57 1860 96 1786 83 1811 1 1836 122 1861 77 1787 132 1812 5 1837 138 1862 59 1788 131 1813 12 1838 103 1863 44 1789 118 1814 14 1839 86 1864 47 1790 90 1815 35 1840 63 1865 30 1791 67 1816 46 1841 37 1866 16 1792 60 1817 41 1842 24 1867 7 1793 47 1818 30 1843 11 1868 37 1794 41 1819 24 1844 15 1869 74
189018601830180017700
100
200
Example B: Annual Sunspot Numbers (1790-1869)
Autocorrelation function and partial autocorrelation function
-1.0
0.0
1.0
rh
h
10 20 30 40
x t
-1.0
0.0
1.0
x t
kk
10 20 30 40k
Spectral density Estimate
0.50.40.30.20.10.00
2000
4000
6000
8000
Smoothed Spectral Estimator (Bandwidth = 0.11)
frequency
Period = 10 years
Assuming an AR(2) model
3.02.01.00.00
300
600
Spectral density of Sunspot data
f( )
Period = 8.733 years
A linear discrete time series
Moving Average time series of infinite order
332211 ttttt uuuux
Let 0 =1, 1, 2, … denote an infinite sequence of numbers.
Let {ut|t T} denote a white noise time series with variance 2.
– independent– mean 0, variance 2.
Let {xt|t T} be defined by the equation.
Then {xt|t T} is called a Linear discrete time series.Comment: A linear discrete time series is a Moving
average time series of infinite order
The AR(1) Time series
ttt uxx 11
Let {xt|t T} be defined by the equation.
2211 ttt uuu
Then tttt uuxx 1211
11122
1 1 ttt uux 111231
21 1 tttt uuux
22
1112
1133
1 1 tttt xuux
where
1
321 1
111
and
ii 1
An alternative approach using the back shift operator, B.
ttt uxx 11
tt uxBI 1
The equation:
can be written
Now 33221
11 11
BBBIBI
since
tt uxBI 1
The equation:
has the equivalent form:
IBBBIBI 332211 11
tt uBIBIx 11
11
3322
1 11 BBBI
tuBBBI 33221 11
32
1 11 I
33
22
11 11 tttt uuuu
The time series {xt |t T} can be written as a linear discrete time series
pp BBBIB 2
21 where
tt uxB
and
For the general AR(p) time series:
33
221
1 BBBIB
tt uBBx 11
[(B)]-1can be found by carrying out the multiplication
IBBBIB 33
221
can be written:
tt uxB
where
Thus the AR(p) time series:
221
1 BBIBB
tt uBBx
tuB
1
Hence tt uBx 1
2211 ttt uuu
This called the Random Shock form of the series
can be written:
tt uxB
where
Thus the AR(p) time series:
221
1 BBIBB
tt uBBx
tuB
1
Hence tt uBx 1
2211 ttt uuu
This called the Random Shock form of the series
An ARMA(p,q) time series {xt |t T} satisfies the equation:
pp BBBIB 2
21 where
tt uBxB
and
The Random Shock form of an ARMA(p,q) time series:
qq BBBIB 2
21
Again the time series {xt |t T} can be written as a linear discrete time series
namely
where
33
221
1 BBBIBBB
(B) =[(B)]-1[(B)] can be found by carrying out the multiplication
BBBBIB 33
221
tt uBBBx 11
tuB 11
Thus an ARMA(p,q) time series can be written:
2211 tttt uuux
where
33
221
1 BBBIBBB
p
211
1
and
The inverted form of a stationary time series
Autoregressive time series of infinite order
An ARMA(p,q) time series {xt |t T} satisfies the equation:
pp BBBIB 2
21 where
tt uBxB
and qq BBBIB 2
21
Suppose that
qq xxxx 2
21 1
exists. 1B
This will be true if the roots of the polynomial
all exceed 1 in absolute value.
The time series {xt |t T} in this case is called invertible.
Then
where
33
221
1 BBBIBBB
tt uBxBB 11
or tt uxB *
11 and 11*
B
Thus an ARMA(p,q) time series can be written:
tttt uxxx *
2211 where
33
221
1 BBBIBBB
q
21
*
11 and
This is called the inverted form of the time series.
This expresses the time series an autoregressive time series of infinite order.