Nonlinear Filtering and Path Integral Method (Paper Review)
-
Upload
kohta-ishikawa -
Category
Technology
-
view
1.893 -
download
0
description
Transcript of Nonlinear Filtering and Path Integral Method (Paper Review)
Brief Paper Review
B.Balaji (2009)“Nonlinear filtering and quantum physics A Feynman path integral perspective”
Kohta IshikawaOct 30. 2011 第3回 確率の科学研究会
1
Outline• Continuous Filtering• DMZ equation (Solution of Continuous Filtering)• Yau equation (approximating DMZ eq)• Path Integrals and Stochastic Processes• Path Integral Representation of Continuous Filtering
• Continuous Filtering and Quantum Physics
2
Continuous Filtering• Discrete Filtering
• Continuous-Discrete Filtering
• Continuous-Continuous Filtering
System ModelObservation Model
dx(t) = f(x(t), t)dt+ e(x(t), t)dw(t) System Model(SDE)y(tk) = h(x(tk), tk) + wk
xk = f(xk�1, tk�1)xk�1 + e(xk�1, tk�1)
yk = h(xk, tk) + wk
Observation Model
dx(t) = f(x(t), t)dt+ e(x(t), t)dv(t)
dy(t) = h(x(t), t)dt+ dw(t)
System Model(SDE)
Observation Model(SDE)
3
• Optimal Estimation of Unnormalized Probability Density under Observation
Duncan-Mortensen-Zakai equation
unnormalized probability density
d�(t, x) = LY �(t, x)dt+mX
i=1
hi(x)�(t, x)dyi(t)
LY �(t, x) = �nX
i=1
@
@xi(fi(x)�(t, x)) +
1
2
nX
i=1
@
2�(t, x)
@x
2i
� 1
2
mX
i=1
h
2i (x)�(t, x)
�(0, x) = �0(x)
• Probability density is given as a conditional expectation under observation(4)
• Then, term is difficult to handledy(t)
4
Robust DMZ equation• Eliminate term dy(t)
u(t, x) = exp
�
mX
i=1
hi(x)yi(t)
!�(t, x)
@u(t, x)
@t
=1
2
nX
i=1
@
2u(t, x)
@x
2i
+nX
i=1
0
@�fi(x) +mX
j=1
yj(t)@hj(x)
@xi
1
A @u(t, x)
@xi
�✓ nX
i=1
@fi(x)
@xi+
1
2
mX
i=1
h
2i (x)�
1
2
mX
i=1
yi(t)�hi(x)
+mX
i=1
nX
j=1
yi(t)fj(x)@hi(x)
@xj� 1
2
mX
i,j=1
nX
k=1
yi(t)yj(t)@hi(x)
@xk
@hj(x)
@xk
◆u(t, x)
Then, DMZ equation translates to
u(0, x) = �0(x)
5
Yau equation• Approximated DMZ equation• Anyway, we can only handle observation term discretely :P
• Consider DMZ equation valid between each observation interval
• Replace y(t) as followsy(t) !
⇢y(⌧l)y(⌧l�1)
post-measurement formpre-measurement form
6
Yau equation• In the interval @ul(t, x)
@t
=1
2
nX
i=1
@
2ul(t, x)
@x
2i
+nX
i=1
0
@�fi(x) +mX
j=1
yj(⌧l)@hj(x)
@xi
1
A @ul(t, x)
@xi
�✓ nX
i=1
@fi(x)
@xi+
1
2
mX
i=1
h
2i (x)�
1
2
mX
i=1
yi(⌧l)�hi(x)
+mX
i=1
nX
j=1
yi(⌧l)fj(x)@hi(x)
@xj� 1
2
mX
i,j=1
nX
k=1
yi(⌧l)yj(⌧l)@hi(x)
@xk
@hj(x)
@xk
◆ul(t, x)
ul(⌧l, x) = ul�1(⌧l�1, x)
solution of previous step
⌧l�1 t ⌧l
• Observation term is constant in the equation• This approximates DMZ equation well
observation is available at times {⌧0, ⌧1, · · · }
7
Yau equation• Eliminate observation term from coefficientsul(t, x) = exp
mX
i=1
yi(⌧l)hi(x)
!ul(t, x)
@ul(t, x)
@t
=1
2
nX
i=1
@
2ul(t, x)
@x
2i
�nX
i=1
fi(x)@ul(t, x)
@xi
�
nX
i=1
@fi(x)
@xi+
1
2
mX
i=1
h
2i (x)
!ul(t, x)
ul(⌧l�1, x) = exp
mX
i=1
yi(⌧l)hi(x)
!ul�1(⌧l�1, x)
observation term onlycontributes to initial condition
• previous observation is also usableyi(⌧l�1)
it approximatesdirectly
�(t, x)
8
Yau algorithm1. Solve Yau eq along with initial condition at latest observation
2. Derive a probability density at next time step
3. Earn a observation at the time
4. Repeat from 1. with a time advancing
This is one of the few efficient algorithms for continuous nonlinear filtering
9
Path Integrals and Stochastic Processes• Stochastic Process, Path Measure and Fokker-Planck equation
Fokker-Planck equation Path Integral representation
Stochastic Process
path measure,functional differentiation
probability density of state
green’s function
10
Path Integral Representation of Continuous Filtering• Formal Solution directly wrote down from SDE
xi(t) = fi(x(t)) + vi(t)
yi(t) = hi(x(t)) + wi(t)
SDE for continuous filtering
Probability densityfunctional jacobian
functional measureP (t, x, y|t0, x0.y0) =
Z[d⇢(v(t))][d⇢(w(t))]
⇥ [dx(t)]�(x(t)� f(x(t))� v(t))
�x(t)�
n(x(t)� f(x(t))� v(t))
⇥ [dy(t)]�(y(t)� h(x(t))� w(t))
�y(t)�
m(y(t)� h(y(t))� w(t))
⇥ �
n(x(t)� x)|x(t0)=x0
�
m(y(t)� y)|y(t0) = y0
11
Path Integral Representation• Probability density with Gaussian noise measure
[d⇢(v(t))] = [Dv(t)] exp
� 1
2~v
nX
i=1
Z t
t0
vi(t)2dt
!
[d⇢(w(t))] = [Dw(t)] exp
� 1
2~w
mX
i=1
Z t
t0
vi(t)2dt
!
P (t, x, y|t0, x0.y0) =
Zy(t)=y
y(t0)=y0
Zx(t)=x
x(t0)=x0
[Dv(t)][Dw(t)][Dx(t)][Dy(t)]
⇥ exp
� 1
2~v
nX
i=1
Zt
t0
v
2i
(t)dt�nX
i=1
Zt
t0
@f
i
(x(t))
@x
i
dt� 1
2~w
mX
i=1
Zt
t0
w
2i
dt
!
⇥�
n
(x(t)� f(x(t))� v(t))�
m
(y(t)� h(y(t))� w(t))
=
Zy(t)=y
y(t0)=y0
Zx(t)=x
x(t0)=x0
[Dx(t)][Dy(t)] exp (�S)
S =1
2
Z t
t0
dt
"1
~v
nX
i=1
(xi(t)� fi(x(t)))2 +
nX
i=1
@fi(x(t))
@xi+
1
~w
mX
i=1
(yi � hi(x(t)))2
#
~v, ~wvariance of noise(assumed diagonal)
:
jacobian term
12
Path Integral Representation• Approximation with discrete measurements
measurement term in the action S
state independent(absorbed in the measure)
� 1
2~w
Z ti
ti�1
dt
mX
i=1
(yi � hi(x(t))) = � 1
2~w
Z ti
ti�1
dt
mX
i=1
⇥y
2i (t) + h
2i (x(t))� 2hi(x(t))yi(t)
⇤
measurementindependent
relevant term1
~w
Z ti
ti�1
dt
mX
j=1
hj(x(t))yj(t) ⇠⇢ 1
~w
Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj)� yj(tj�1)]
1~w
Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj�1)� yj(tj�2)]
13
Path Integral Representation• Approximation with discrete measurements
leads to approximated probability density
P (ti, xi, yi|ti�1, xi�1, yi�1) ⇠ ˜
P (ti, xi|ti�1, xi�1)
⇥⇢
exp
⇣1~w
Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj)� yj(tj�1)]
⌘
exp
⇣1~w
Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj�1)� yj(tj�2)]
⌘
˜
P (t
i
, x
i
|ti�1, xi�1) =
Zx(ti)=xi
x(ti�1)=xi�1
[Dx(t)] exp(�S(t
i�1, ti))
S(ti�1, ti) =1
2
Z ti
ti�1
dt
"1
~v
nX
i=1
(xi(t)� fi(x(t)))2 +
nX
i=1
@fi(x(t))
@xi+
1
~w
mX
i=1
h
2i (x(t))
#
satisfies Yau equation with δfunction initial condition (fundamental solution)P (t, x|ti�1, xi�1)
14
Continuous Filtering and Quantum Physics• Action term is interpreted as integration of Lagrangian
˜
P (t, x|t0, x0) =
Zx(t)=x
x(t0)=x0
[Dx(t)] exp
✓� 1
~v
S
◆
L = T � V
S =1
2
Zt
t0
dt
"nX
i=1
[x2i
(t) + f
2i
(x)� 2xi
(t)fi
(x(t))] + ~v
nX
i=1
@f
i
(x(t))
@x
i
+~v
~w
mX
i=1
h
2i
(x(t))
#
=1
2
Zt
t0
dt
"nX
i=1
[x2i
(t) + f
2i
(x)] + ~v
nX
i=1
@f
i
(x(t))
@x
i
+~v
~w
mX
i=1
h
2i
(x(t))
#
�nX
i=1
Zx(t)
x(t0)dx
i
(t)fi
(x(t))
⌘ 1
2
Zt
t0
dtL�nX
i=1
Zx(t)
x(t0)dx
i
(t)fi
(x(t))T =
1
2
Z t
t0
dt
nX
i=1
x
2i (t)
�V =1
2
Z t
t0
dt
"nX
i=1
f
2i (x) + ~v
@fi(x(t))
@xi
�+
~v~w
mX
i=1
h
2i (x(t))
#
15
General Advantages of Path Integral Representation• Straightforward to develop numerical algorithms• A lot of (quantum mechanical) approximation methods are applicable
• classical approximation (MAP estimation in Bayesian context)
• perturbation methods
16
Summary• Derive Yau equation for nonlinear continuous filtering
• Develop the Path Integral representation of the solution
17
Another Example of Path Integral Methods• Stochastic Optimal Control (3)• Hamilton-Jacobi-Bellman(HJB) equation
dx = (b(x, t) +Bu)dt+ dv
C(xint
, t
int
, u) =
⌧�(x(t
f
)) +
Ztf
tint
dt
✓1
2u(t)TRu(t) + V (x(t), t)
◆�
xint
J(x, t) = minu(t!tf )
C(x, t, u(t ! tf ))
Dynamics (with control)
Cost Function
Solution of the problem is optimal control function which minimizes the cost
end cost control potential
(Optimal control function is derived from minimized cost function)
18
Optimal Control• HJB equation and path integral representation
�@J
@t
= �1
2
@J
@x
�TBR
�1B
T @J
@x
+ V + b
T @J
@x
+1
2Tr
⌫
@
2J
@x
2
�
J(x, t) = �� log
Z
x(t)=x
[Dx(t)] exp
✓�S
�
◆
S = �(x(tf )) +
Z tf
td⌧
✓1
2(x(⌧)� b(x(⌧), ⌧))TR(x(⌧)� b(x(⌧), ⌧)) + V (x(⌧), ⌧)
◆
V (x(t), t) could be non-quadratic(difficult to solve with traditional methods)
Solution of the equation could be wrote as path integral under some conditions
19
References(1) B. Balaji, “Universal Nonlinear Filtering Using Feynman Path Integrals Ⅱ: The Continuous-Continuous Model with Additive Noise” PMC Physics A (2009) 3:2
(2) Shing-Tung Yau and Stephen S.-T. Yau, “Real Time Solution of Nonlinear Filtering Problem Without Memory Ⅰ” Mathematical Research Letters 7, 671-693 (2000)
(3) H. J. Kappen “Path Integrals and Symmetry Breaking for Optimal Control Theory” Journal of Statistical Mechanics (2005) P11011
(4) A. H. Jazwinski “Stochastic Processes and Filtering Theory”
20