Random Process Lecture 7. Wide Sense Stationary Random...
Transcript of Random Process Lecture 7. Wide Sense Stationary Random...
![Page 1: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/1.jpg)
Random ProcessLecture 7. Wide Sense Stationary Random
Processes
Husheng Li
Min Kao Department of Electrical Engineering and Computer ScienceUniversity of Tennessee, Knoxville
Spring, 2016
1/21
![Page 2: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/2.jpg)
Hilbert Space Generated byStationary Process
A random process Xt , t ∈ T , is called wide-sense stationary if thereexist a constant m and a function b(t) such that
EXt = m, E [XtXs] = b(t − s).
Let H be the subspace of L2(Ω,F ,P) consisting of functions which canbe represented as finite linear combinations of the form ξ =
∑s∈S csXs,
where S is an arbitrary finite subset of T .
Then we can define an inner product in H. The sum of an infinite series∑csXs is contained in H. Then, we say H is the Hilbert space
generated by the random process Xt .
We can define unitary operator (Uξ,Uη) = (ξ, η).
2/21
![Page 3: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/3.jpg)
Law of Large Numbers
Let Xn be a wide sense stationary process. Then, there exists η ∈ Hsuch that
limn→∞
Xk + ...+ Xk+n−1
n= η(∈ H).
von Neumann Ergodic Theorem: Let U be unitary, then we have
limn→∞
ξ + ...+ Un−1ξ
n= Pξ,
where P is an orthogonal projection onto the subspaceH0 = ψ : ψ ∈ H,Uψ = ψ.
3/21
![Page 4: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/4.jpg)
Bochner Theorem
Bochner Theorem: There is a 1-to-1 correspondence between the set ofcontinuous non-negative definite functions and the set of finite measureson the Borel σ-algebra of R. Namely if ρ is a finite measure then
f (x) =
∫R
ejλx dρ(λ),
is nonnegative definite. Conversely, each f can be represented in thisform.
4/21
![Page 5: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/5.jpg)
Spectrum Representation ofStationary RandomProcesses
Consider the autocorrelation function:
b(n1 − n2) = E[Xn1 Xn2
].
According to the Bochner’s Theorem, we have
b(n) =
∫[0,1)
e2πjλndρ(λ),
where ρ is the spectral measure of the process Xn.
5/21
![Page 6: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/6.jpg)
Spectrum Representation ofStationary RandomProcesses (Continued)
Theorem: Consider the space L2([0, 1),B([0, 1)), ρ) of square-integrablefunctions on [0, 1) (with respect to the measure ρ). There exists anisomorphism ψ : H → L2 if the spaces H and L2 such that
ψ(Uξ) = e2πjλψ(ξ).
Corollary: If ρ(0) = 0, then H0 = 0 and the time averages in the Lawof Large Numbers converge to zero.
6/21
![Page 7: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/7.jpg)
Orthogonal RandomMeasures
Consider a Borel subset ∆ ⊂ [0, 1). Define
Z (∆) = ψ−1(X∆).
A function Z with values in L2(Ω,F ,P) defined on a σ-algebra is calledan orthogonal random measure if it satisfies two conditions. If Z is givenby the above definition, it is called the random spectral measure of theprocess Xn.
Then, we can define an integral with respect to an orthogonal randommeasure.
7/21
![Page 8: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/8.jpg)
Linear Prediction
We consider stationary random processes with discrete time andassume that EXn = 0.
We define
Hk2k1
= closure
k2∑
n=k1
cnXn
.
Given m < k0, how to find the best approximation of Xk0 by using theelements in Hm
−∞.
8/21
![Page 9: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/9.jpg)
Linear Predictors
We defineh−m = inf ‖X0 −
∑n≤−m
cnXn‖.
A random process Xn is called linearly non-deterministic if h−1 > 0.
A random process Xn is called linear regular if P−m−∞X0 → 0 if m→∞.
Theorem: A process Xn is linearly regular if and only if it is linearlynon-deterministic and ρ is absolutely continuous with respect to theLebesgue measure.
9/21
![Page 10: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/10.jpg)
Kolmogorov-WienerTheorem
The spectrum measure ρ can be decomposed to ρ = ρ0 + ρ1.
Theorem: For any wide-sense stationary process with zero mean wehave
h−1 = exp
(12
∫[0,1)
log p0(λ)dλ
),
where the right hand side is set to be equal to zero if the integral in theexponent is equal to −∞.
10/21
![Page 11: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/11.jpg)
Linear Filtering
Kalman filtering is optimum not only for the Gaussian case, but alsooptimal among all linear estimators when the second order statistics ofthe model remain fixed.
We apply this to a general class of problems known asWiener-Kolmogorov filtering.
Consider two random sequences Xn and Yn. We want to estimateXn from Yn using linear estimation (Hb
a):
Xt =b∑
n=a
ht,nYn + ct .
11/21
![Page 12: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/12.jpg)
MMSE and OrthogonalPrinciple
Goal: to minimize the mean square error:
minXt∈Hb
a
E[(Xt − Xt )
2].
Orthogonal principle: if Xt solves the MMSE problem, then we have
E[(Xt − Xt )Z
]= 0, ∀Z ∈ Hb
a,
orE [Xt ] = E [Xt ], E
[(Xt − Xt )Yl
]= 0, ∀a ≤ l ≤ b.
12/21
![Page 13: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/13.jpg)
Illustration of OrthogonalPrinciple
13/21
![Page 14: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/14.jpg)
How to Determine theCoefficients?
We can apply the orthogonal principle to determine the coefficients inthe linear estimation.
Wiener-Hopf Equation:
Cov(Xt ,Yl ) =b∑
n=a
ht,nCov(Yn,Yl ),
or the matrix form:σX ,Y (t) = ΣY ht .
14/21
![Page 15: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/15.jpg)
Algorithms
Levinson Filtering: Levinson filtering is concerned with one-stepprediction of a random sequence whose second-order statistics arestationary in time:
CY (n, l) = CY (n − l , 0).
We want to estimate Yt from Yss<t , namely
Yt =t−1∑n=0
ht,nYn.
15/21
![Page 16: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/16.jpg)
Wiener-Kolmogorov Filtering
Noncausal W-K filtering: We estimate Xt using observations Ys from−∞ to∞:
Xt =∞∑
n=−∞
ht,nYn.
The Wiener-Hopf equation becomes
CXY (τ) =∞∑
a=−∞
haCY (τ − a).
Can you see the convolution structure in the equation?
16/21
![Page 17: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/17.jpg)
Frequency Domain
Carry out Fourier transformation for CXY , CY and ha and thus obtainφXY , φY and H.
The Wiener-Hopf equation becomes
φXY (w) = φY (w)H(w), −π ≤ w ≤ π.
Then, h(n) is obtained from the inverse transform of φXY (w)φY (w)
.
17/21
![Page 18: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/18.jpg)
Causal Wiener-KolmogorovFiltering
The main drawback of the noncausal Wiener-Kolmogorov filtering is thatit requires all future observations. To make it causal, we have
Xt =t∑
n=−∞
ht,nYn,
which is denoted by Ht−∞. We can reuse ht,n of the noncausal filter,
namely truncation.
According to the orthogonality principle, we need
E [(Xt − Xt )Z ] = 0,
for all Z ∈ Ht−∞. This is true when Yt is uncorrelated.
18/21
![Page 19: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/19.jpg)
Prewhittening
Now we need to whiten Y by using a causal linear operation:
Denote by ˆYt|t−1 the best prediction of Yt based on Y t−1−∞.
Evaluate σ2t = E
[(Yt − ˆYt|t−1)2
].
Define a new sequence
Zn =Yt − ˆYt|t−1
σn.
Zn is a uncorrelated sequence, which is used to estimate Xt .
19/21
![Page 20: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/20.jpg)
Spectral Factorization
Linear prediction: predict Yt+λ using Ynn≤t (λ > 0).
Spectral Factorization Theorem: Suppose that Yn satisfies
12π
∫ π
−πlogφY (w)dw > −∞.
Then φY can be written as φY (w) = φ+Y (w)φ−Y (w), which satisfy
12π
∫ π
−πφ+
Y ejnw (w)dw = 0, ∀n < 0,
and1
2π
∫ π
−πφ−Y ejnw (w)dw = 0,∀n > 0,
and|φ+
Y (w)| = |φ−Y (w)| = |φY (w)|.20/21
![Page 21: Random Process Lecture 7. Wide Sense Stationary Random ...web.eecs.utk.edu/~hli31/ECE504_2016_files/Lecture7.pdf · Hilbert Space Generated by Stationary Process A random process](https://reader036.fdocuments.net/reader036/viewer/2022081523/5fd9f8b095179004ed5a4d33/html5/thumbnails/21.jpg)
Causal Linear PredictionWe can use H(w) = 1
φ+Y (w)
as a causal linear filter to whiten Y : the
output W of the filter satisfies
φW (w) = 1, ∀ − π ≤ w ≤ π.
Wn is obtained from Yn. Meanwhile, Yn can also be obtainedfrom Wn:
Yt =t∑
n=−∞
ft−nWn,
where fn is obtained from the inverse Fourier transform of φ+Y .
The optimal prediction is then given by
Yt+λ =t∑
n=−∞
ft+λ−nWn.
Why?21/21