MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... ·...
Transcript of MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... ·...
![Page 1: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/1.jpg)
MP for Forward-Backward DoublyStochastic Control Systems and
Applications
Liangquan Zhangjoint with Prof. Yufeng Shi
E-mail: Liangquan [email protected].
Laboratoire de MathematiquesUniversite de Bretagne Occidentale, France.
Stochastic Control Problems for FBSDEs and ApplicationsEssaouira, 16 December, 2010
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 2: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/2.jpg)
Main results in this talk
1 The doubly stochastic maximum principle in global form isobtained.
2 Optimal control problems of stochastic partial differentialequations.
3 Linear quadratic nonzero sum doubly stochastic differentialgames.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 3: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/3.jpg)
Previous Work
It is well known that optimal control problem is one of thecentral themes of control science. The necessary conditions ofoptimal problem were established for deterministic control systemby Pontryagin’s group in the 1950’s and 1960’s. Since then, a lotof work has been done on the forward stochastic system such asKushner , Bismut, Bensoussan, Haussmann and Peng etc.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 4: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/4.jpg)
Forward-Backward Fully Coupled Control Systems
Consider the following control systems,dxt = f (t, xt, yt, zt, vt) dt + σ (t, xt, yt, zt, vt) dBt,dyt = −g (t, xt, yt, zt, vt) dt + ztdBt,x0 = x, yT = ξ,
(1)
or dxt = b (t, xt, yt, zt, vt) dt + σ (t, xt, yt, zt) dBt,dyt = −f (t, xt, yt, zt, vt) dt + ztdBt,x0 = x, yT = h (xT ) .
(2)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 5: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/5.jpg)
Minimize the cost function
J(v(·))
= E[∫ T
0L (t, xt, yt, zt, vt) dt + Φ (xT ) + h (y0)
].
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 6: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/6.jpg)
References
1 S. Peng, Backward stochastic differential equations andapplication to optimal control. Appl. Math. Optim. 27(1993) 125-144.
2 Z. Wu, Maximum principle for optimal control problem offully coupled forward-backward stochastic systems. SystemsSci. Math. Sci. 11 (1998) 249-259.
3 J. Shi and Z. Wu, The maximum principle for fully coupledforward-backward stochastic control system. Acta AutomaticaSinica 32 (2006) 161-169.
4 S. Ji and X.Y. Zhou, A maximum principle for stochasticoptimal control with terminal state constraints, and itsapplications. Commun. Inf. Syst. 6 (2006) 321-338.
5 B. Øksendal, A. Sulem, Maximum principles for optimalcontrol of forward-backward stochastic differential equationswith jump. SIAM J. Control Optim. 48 (2009) 2945-2976.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 7: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/7.jpg)
Techniques
When the control domain is convex, we can apply convexperturbation corresponding to (1).
When the control domain is non-convex, we can apply spikevariations corresponding to (2).
However, when all the coefficients contain the controlvariables and the control domain is non-convex, the abovemethods fail. It is still an open problem.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 8: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/8.jpg)
Techniques
When the control domain is convex, we can apply convexperturbation corresponding to (1).
When the control domain is non-convex, we can apply spikevariations corresponding to (2).
However, when all the coefficients contain the controlvariables and the control domain is non-convex, the abovemethods fail. It is still an open problem.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 9: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/9.jpg)
Techniques
When the control domain is convex, we can apply convexperturbation corresponding to (1).
When the control domain is non-convex, we can apply spikevariations corresponding to (2).
However, when all the coefficients contain the controlvariables and the control domain is non-convex, the abovemethods fail. It is still an open problem.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 10: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/10.jpg)
A Kind of Stochastic Heat PDEs
In 1994, Pardoux and Peng introduced the following backwarddoubly stochastic differential equations (BDSDEs in short):
Yt = ξ +∫ T
tf(s, Ys, Zs)ds +
∫ T
tg(s, Ys, Zs)d
←−Bs −
∫ T
tZsd−→Ws,
which can be related to the following stochastic partial differentialequations (SPDEs in short)
u (t, x) = h (x) +∫ Tt [Lu (s, x) + f (s, x, u (s, x) , (∇uσ) (s, x))] ds
+∫ Tt g (s, x, u (s, x) , (∇uσ) (s, x)) d
←−Bs.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 11: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/11.jpg)
Motivation: Optimal Control for SPDEs
Consider the following SPDE control systems:
u (t, x) =
h (x)+∫ Tt [Lvu (s, x) + f (s, x, u (s, x) , (∇uσ) (s, x) , vs)] ds
+∫ Tt g (s, x, u (s, x) , (∇uσ) (s, x)) d
←−Bs, 0 ≤ t ≤ T,
where
Lvu =
Lu1...
Luk
,
with Lφ (x) = 12
∑di,j=1 (σσ∗)ij (x) ∂2φ(x)
∂xi∂xj+∑d
i=1 bi (x, v) ∂φ(x)∂xi
.
Minimize the following cost function
J(v(·))
= E[∫ T
0l (s, x, u (s, x) , (∇uσ) (s, x) , vs) ds + γ (u (0, x))
].
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 12: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/12.jpg)
Forward-Backward Doubly Stochastic Control Systems
In this paper, we assume the control domain is non-convex. Both yand Y are one-dimensional, and the control v is alsoone-dimensional.
dyt = f (t, yt, Yt, zt, Zt, vt) dt + g (t, yt, Yt, zt, Zt) d−→Wt − ztd
←−Bt,
dYt = −F (t, yt, Yt, zt, Zt, vt) dt−G (t, yt, Yt, zt, Zt) d←−Bt + Ztd
−→Wt,
y0 = x, YT = h (yT ) , t ∈ [0, T ] ,(3)
Minimize the following cost function
J(v(·)) .= E
[∫ T
0l (t, yt, Yt, zt, Zt, vt) dt + Φ (yT ) + γ (Y0)
]Remark The results in this paper can be extended tomultidimensional case.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 13: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/13.jpg)
Notations
Let Wt; 0 ≤ t ≤ T and Bt; 0 ≤ t ≤ T be two mutuallyindependent standard Brownian motions defined on (Ω,F , P ),with values respectively in Rd and in Rl. Let N denote the classof P -null elements of F .
For each t ∈ [0, T ], we define
Ft.= FW
t ∨ FBt,T
where FWt = N ∨ σ Wr −W0; 0 ≤ r ≤ t,
FBt,T = N ∨ σ Br −Bt; t ≤ r ≤ T
Let M2 (0, T ;Rn) denote the space of all (classes of dP ⊗ dta.e. equal) Rn-valued Ft-progressively measurable stochastic
processes vt; t ∈ [0, T ] which satisfy E∫ T0 |vt|2dt <∞.
Let L2 (Ω,FT , P ;R) denote the space of all FT -measurableone-valued random variable ξ satisfying E |ξ|2 <∞.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 14: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/14.jpg)
For a given u ∈M2(0, T ;Rd
)and v ∈M2
(0, T ;Rl
), one can
define the (standard) forward Ito’s integral∫ ·0 usd
−→Ws and the
backward Ito’s integral∫ T· vsd
←−Bs.
Definition
A stochastic process X = Xt; t ≥ 0 is called Ft-progressivelymeasurable, if for any t ≥ 0, X on Ω× [0, t] is measurable with
respect to(FW
t × B ([0, t]))∨(FB
t,T × B ([t, T ])).
Denote
ζ =
yYzZ
, A (t, ζ) =
−Ff−Gg
(t, ζ) .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 15: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/15.jpg)
Uniqueness and Existence of solutions for FBDSDEs
Assumptions(H1)For each ζ ∈ R1+1+l+d, A (·, ζ) is an Ft-measurable
process defined on [0, T ] with A (·, 0) ∈M2(0, T ;R1+1+l+d
).
(H2)A (t, ζ) and h (y) satisfy Lipschitz conditions: thereexists a constant k > 0, such that ∣∣A (t, ζ)−A
(t, ζ)∣∣ ≤ k
∣∣ζ − ζ∣∣ , ∀ζ, ζ ∈ R1+1+l+d, ∀t ∈ [0, T ] ,
|h (y)− h (y)| ≤ k |y − y| , ∀y, y ∈ R.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 16: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/16.jpg)
(H3)
⟨A (t, ζ)−A
(t, ζ), ζ − ζ
⟩≤ −µ
∣∣ζ − ζ∣∣2 ,
∀ζ = (y, Y, z, Z) , ζ =(y, Y , z, Z
)∈ R×R×Rl×Rd, ∀t ∈ [0, T ] .
〈h (y)− h (y) , y − y〉 ≥ 0, ∀y, y ∈ R,or
(H3’)
⟨A (t, ζ)−A
(t, ζ), ζ − ζ
⟩≥ µ
∣∣ζ − ζ∣∣2 ,
∀ζ = (y, Y, z, Z) , ζ =(y, Y , z, Z
)∈ R×R×Rl×Rd, ∀t ∈ [0, T ] .
〈h (y)− h (y) , y − y〉 ≤ 0, ∀y, y ∈ R,(H4) F, f, G, g, h, l, Φ, γ are continuously differentiable withrespect to (y, Y, z, Z) , y and Y . They and all their derivatives arebounded by a constant C.
Proposition
For any given admissible control v(·), we assume (H1), (H2) and(H3) (or (H1), (H2) and (H3)’) hold. Then FBDSDE (3) has aunique solution (yt, Yt, zt, Zt) ∈M2
(0, T ;R1+1+l+d
)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 17: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/17.jpg)
Variational Equations and Variational Inequalities
We first introduce the following spike variational control:
uεt =
v, τ ≤ t ≤ τ + ε,ut, otherwise,
and variational equations:
dy1t =
[fyy
1t + fY Y 1
t + fzz1t + fZY 1
t + f (uεt )− f (ut)
]dt
+[gyy
1t + gY Y 1
t + gzz1t + gZZ1
t
]d−→Wt − z1
t d←−Bt,
y10 = 0,
dY 1t = −
[Fyy
1t + FY Y 1
t + Fzz1t + FZZ1
t + F (uεt )− F (ut)
]dt
−[Gyy
1t + GY Y 1
t + Gzz1t + GZZ1
t
]d←−Bt + Z1
t d−→Wt,
Y 1T = hy (yT ) y1
T .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 18: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/18.jpg)
Estimates
Lemma
We assume (H1)-(H4) hold. Then we have
E∫ T
0
∣∣y1t
∣∣2 dt ≤ Cε,
E∫ T
0
∣∣Y 1t
∣∣2 dt ≤ Cε,
E∫ T
0
∣∣z1t
∣∣2 dt ≤ Cε,
E∫ T
0
∣∣Z1t
∣∣2 dt ≤ Cε.
However, the order of the estimate for(y1
t , Y1t , z1
t , Z1t
)is too
low to get the variational inequalities. We need to give some moreelaborate estimates.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 19: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/19.jpg)
Estimates
Lemma
Assuming (H1)-(H4) hold, then we have
sup0≤t≤T
(E∣∣y1
t
∣∣2) ≤ Cε,
sup0≤t≤T
(E∣∣Y 1
t
∣∣2) ≤ Cε.
Lemma
Assuming (H1)-(H4) hold, then we have
E
(sup
0≤t≤T
∣∣y1t
∣∣2) ≤ Cε,
E
(sup
0≤t≤T
∣∣Y 1t
∣∣2) ≤ Cε,
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 20: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/20.jpg)
Estimates
Next, we will give some elaborate estimates for(y1
t , Y1t , z1
t , Z1t
)by
virtue of the techniques of FBDSDEs.
Lemma
Assuming (H1)-(H4) hold, then we have
E∫ T
0
∣∣y1t
∣∣2 dt ≤ Cε32 ,
E∫ T
0
∣∣Y 1t
∣∣2 dt ≤ Cε32 ,
E∫ T
0
∣∣z1t
∣∣2 dt ≤ Cε32 ,
E∫ T
0
∣∣Z1t
∣∣2 dt ≤ Cε32 .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 21: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/21.jpg)
High Order Estimates
Lemma
Assuming (H1)-(H4) hold, then we have
E∫ T
0
∣∣yεt − yt − y1
t
∣∣2 dt ≤ Cε32 ,
E∫ T
0
∣∣Y εt − Yt − Y 1
t
∣∣2 dt ≤ Cε32 ,
E∫ T
0
∣∣zεt − zt − z1
t
∣∣2 dt ≤ Cε32 ,
E∫ T
0
∣∣Zεt − Zt − Z1
t
∣∣2 dt ≤ Cε32 ,
sup0≤t≤T
[E∣∣yε
t − yt − y1t
∣∣2] ≤ Cε32 ,
sup0≤t≤T
[E∣∣Y ε
t − Yt − Y 1t
∣∣2] ≤ Cε32 .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 22: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/22.jpg)
Variational Inequality
Lemma
Under the assumptions (H1)-(H4), it holds thatE∫ T0
[lyy
1t + lY Y 1
t + lzz1t + lZZ1
t + l (uεt )− l (ut)
]dt
+E[Φy (yT ) y1
T
]+ E
[γY (Y0) Y 1
0
]≥ o (ε) .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 23: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/23.jpg)
Adjoint Equations
dpt = (FY pt − fY qt + GY kt − gY ht − lY )dt
+(FZpt − fZqt + GZkt − gZht − lZ)d−→Wt − ktd
←−Bt,
dqt = (Fypt − fyqt + Gykt − gyht − ly)dt
+(Fzpt − fzqt + Gzkt − gzht − lz)d←−Bt + htd
−→Wt,
p0 = −γY (Y0) , qT = −hy (yT ) PT + Φy (yT ) , 0 ≤ t ≤ T,(4)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 24: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/24.jpg)
Hamilton Function
H (t, y, Y, z, Z, v, p, q, k, h) .= 〈q, f (t, y, Y, z, Z, v)〉− 〈p, F (t, y, Y, z, Z, v)〉− 〈k, G (t, y, Y, z, Z)〉+ 〈h, g (t, y, Y, z, Z)〉+l (t, y, Y, z, Z, v) .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 25: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/25.jpg)
Adjoint Equations
dpt = −HY dt−HZdWt − ktd
←−Bt,
dqt = −Hydt−Hzd←−Bt + htdWt,
p0 = −γY (Y0) ,qT = −hy (yT ) PT + Φy (yT ) , 0 ≤ t ≤ T.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 26: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/26.jpg)
The Maximum Principle in Global Form
Theorem
Suppose (H1)-(H4) hold. Let(y(·), Y(·), z(·), Z(·), u(·)
)be an
optimal control and its corresponding trajectory of (3),(p(·), q(·), k(·), h(·)
)be the corresponding solution of (4). Then the
maximum principle holds, that is
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht)≥ H (t, yt, Yt, zt, Zt, ut, pt, qt, kt, ht) ,
∀v ∈ U , a.e, a.s..
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 27: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/27.jpg)
Example
Example
Let the control domain be U = [−1, 1] . Consider the followinglinear forward-backward doubly stochastic control system which isa simple case of (2). We assume that l = d = 1.
dyt = (zt − Zt + vt) d−→Wt − ztd
←−Bt,
dYt = − (zt + Zt + vt) d←−Bt + Ztd
−→Wt,
y0 = 0, YT = 0, t ∈ [0, T ] ,
where T > 0 is a given constant and the cost function is
J(v(·))
=12E∫ T
0
(y2
t + Y 2t + z2
t + Z2t + v2
t
)dt +
12Ey2
T +12EY 2
0 .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 28: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/28.jpg)
Example
Optimal control is u(·) ≡ 0;
The Hamilton function is
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht) =12v2.
For any v ∈ U , we always have
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht)≥ H (t, yt, Yt, zt, Zt, ut, pt, qt, kt, ht) = 0, a.e, a.s..
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 29: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/29.jpg)
Example
Optimal control is u(·) ≡ 0;
The Hamilton function is
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht) =12v2.
For any v ∈ U , we always have
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht)≥ H (t, yt, Yt, zt, Zt, ut, pt, qt, kt, ht) = 0, a.e, a.s..
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 30: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/30.jpg)
Example
Optimal control is u(·) ≡ 0;
The Hamilton function is
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht) =12v2.
For any v ∈ U , we always have
H (t, yt, Yt, zt, Zt, v, pt, qt, kt, ht)≥ H (t, yt, Yt, zt, Zt, ut, pt, qt, kt, ht) = 0, a.e, a.s..
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 31: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/31.jpg)
Application to Optimal Control of SPDEs
Consider the following quasilinear SPDEs with control variable:u (t, x) = h (x) +
∫ Tt [Lu (s, x) + f (s, x, u (s, x) , (∇uσ) (s, x) , vs)] ds
+∫ Tt g (s, x, u (s, x) , (∇uσ) (s, x)) d
←−Bs, 0 ≤ t ≤ T,
(5)where u : [0, T ]×R→ R and ∇u (s, x) denote the first orderderivative of u (s, x) with respect to x, and
Lu =
Lu1...
Luk
,
with Lφ (x) = 12
∑di,j=1 (σσ∗)ij (x) ∂2φ(x)
∂xi∂xj+∑d
i=1 bi (x, v) ∂φ(x)∂xi
.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 32: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/32.jpg)
Application to Optimal Control of SPDEs
We give the following assumptions for sake of completeness(A1)
b ∈ C3l,b (R×R;R) , σ ∈ C3
l,b (R;R) , h ∈ C3p (R;R) ,
f (t, ·, ·, ·, v) ∈ C3l,b (R×R×R;R) , f (·, x, y, z, v) ∈M2 (0, T ;R) ,
g (t, ·, ·, ·) ∈ C3l,b (R×R×R;R) , g (·, x, y, z) ∈M2 (0, T ;R)
∀t ∈ [0, T ] , x ∈ R, y ∈ R, z ∈ R, v ∈ R.
(A2)There exist some constant c > 0 and 0 < α < 1 such that for
all (t, x, yi, zi, v) ∈ [0, T ]×R×R×R×R, (i = 1, 2),|f (t, x, y1, z1, v)− f (t, x, y2, z2, v)|2 ≤ c
(|y1 − y2|2 + |z1 − z2|2
),
|g (t, x, y1, z1)− g (t, x, y2, z2)|2 ≤ c |y1 − y2|2 + α |z1 − z2|2 .
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 33: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/33.jpg)
Application to Optimal Control of SPDEs
Find v∗(·) ∈ Uad, such that
J(v∗(·)
).= inf
v(·)∈Uad
J(v(·)),
where J(v(·))
is its cost function as follows:
J(v(·))
= E[∫ T
0l (s, x, u (s, x) , (∇uσ) (s, x) , vs) ds + γ (u (0, x))
].
(6)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 34: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/34.jpg)
Application to Optimal Control of SPDEs
Consider the following FBDSDEs control systemsXt,x
s = x +∫ st b(Xt,x
r , vr
)dr +
∫ st σ(Xt,x
r
)d−→Wr,
Y t,xs = h
(Xt,x
T
)+∫ Ts f
(r, Xt,x
r , Y t,xr , Zt,x
r , vr
)dr
+∫ Ts g
(r, Xt,x
r , Y t,xr , Zt,x
r
)d←−Br
−∫ Ts Zt,x
r d−→Wr, 0 ≤ t ≤ s ≤ T,
(7)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 35: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/35.jpg)
Application to Optimal Control of SPDEs
Adjoint equation dpt = (fY pt + gY kt − lY ) dt + (fZpt − gZkt − lZ) dWt − ktd←−Bt,
dqt = (fXpt − bXqt + gXkt − σXht − lX) dt + htdWt,
p0 = −γY (Y0) , qT = −hX (XT ) pT , 0 ≤ t ≤ T.(8)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 36: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/36.jpg)
Application to Optimal Control of SPDEs
Find an optimal control v∗(·) ∈ Uad, such that
J(v∗(·)
).= inf
v(·)∈Uad
J(v(·)),
where J(v(·))
is the cost function same as (6):
J(v(·))
= E[∫ T
0l (s,Xs, Ys, Zs, vs) ds + γ (Y0)
].
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 37: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/37.jpg)
Pardoux, Peng 1994
Proposition
For any given admissible control v(·), we assume (A1) and (A2)hold. Then (7) has a unique solution(Xt,x
(·) , Y t,x(·) , Zt,x
(·)
)∈M2 (0, T ;R×R×R).
Proposition
For any given admissible control v(·), we assume (A1) and (A2)hold. Let u (t, x) ; 0 ≤ t ≤ T, x ∈ R be a random field such thatu (t, x) is FB
t,T -measurable for each (t, x) , u ∈ C0,2 ([0, T ]×R;R)a.s., and u satisfies SPDE (5). Then u (t, x) = Y t,x
t .
Proposition
For any given admissible control v(·), we assume (A1) and (A2)
hold. Then
u (t, x) = Y t,xt ; 0 ≤ t ≤ T, x ∈ R
is a unique
classical solution of SPDE (5).
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 38: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/38.jpg)
Theorem
Suppose (A1)-(A2) hold. Let(X(·), Y(·), Z(·), u(·)
)be an optimal
control and its corresponding trajectory of (7),(p(·), q(·), k(·), h(·)
)be the solution of (8). Then the maximum principle holds, that is,for t ∈ [0, T ], ∀v ∈ U ,
H (t, Xt, Yt, Zt, v, pt, qt, kt, ht)≥ H (t, Xt, Yt, Zt, v
∗t , pt, qt, kt, ht) , a.e., a.s..
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 39: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/39.jpg)
Main result
Theorem
Suppose u (t, x) is the optimal solution of SPDE (5) correspondingto the optimal control v∗(·) of (5). Then we have, for any v ∈ Uand t ∈ [0, T ] , x ∈ R,
H (t, x, u (t, x) , (∇uσ) (t, x) , v, pt, qt, kt, ht)≥ H (t, x, u (t, x) , (∇uσ) (t, x) , v∗t , pt, qt, kt, ht) , a.e., a.s.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 40: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/40.jpg)
Application to Nonzero Sum Doubly Stochastic DifferentialGames
We consider the linear quadratic non-zero sum doubly stochasticdifferential games problem as following. Now the control system is
dxvt =
[Axv
t + B1v1t + B2
t v2t + Ckv
t + αt
]dt
+ [Dxvt + Ekv
t + βt] dWt − kvt d←−Bt,
xv0 = a, t ∈ [0, T ] ,
(10)
where A, C, D and E are n× n bounded matrices, further, Esatisfies 0 < |E| < 1, v1
t and v2t , t ∈ [0, T ] , are two admissible
control processes, that is Ft-progressively measurable squareintegrable processes taking values in Rk. B1 and B2 are n× kbounded matrices. αt and βt are two adapted square-integrableprocesses.
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 41: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/41.jpg)
Application to Nonzero Sum Doubly Stochastic DifferentialGames
We denote byJ1 (v (·))= 1
2E[∫ T
0
(⟨R1xv
t , xvt
⟩+⟨N1v1
t , v1t
⟩+⟨P 1kv
t , kvt
⟩)dt +
⟨Q1xv
T , xvT
⟩],
J2 (v (·))= 1
2E[∫ T
0
(⟨R2xv
t , xvt
⟩+⟨N2v2
t , v2t
⟩+⟨P 2kv
t , kvt
⟩)dt +
⟨Q2xv
T , xvT
⟩].
(1)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 42: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/42.jpg)
Application to Nonzero Sum Doubly Stochastic DifferentialGames
We denote v (·) =(v1 (·) , v2 (·)
). and here, Qi, Ri, and P i
(i = 1, 2), are n× n nonnegative symmetric bounded matrices, N1
and N2 are k × k positive symmetric bounded matrices andinverses
(N1)−1
,(N2)−1
are also bounded. The problem is tofind the feedback controls
(u1 (·) , u2 (·)
)which is called Nash
equilibrium point for the game, such thatJ1(u1 (·) , u2 (·)
)≤ J1
(v1 (·) , u2 (·)
), ∀v1 (·) ∈ Rk;
J2(u1 (·) , u2 (·)
)≤ J2
(u1 (·) , v2 (·)
), ∀v2 (·) ∈ Rk.
(11)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 43: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/43.jpg)
Application to Nonzero Sum Doubly Stochastic DifferentialGames
Note that the actions of the two players are described by a classicalBDSDE in which we indicates that the players should make somestrategy to overcome the disturbed information. In order tointroduce the main result, we need the followingassumptions(i = 1, 2):
Bi(N i)−1 (
Bi)T
AT = AT Bi(N i)−1 (
Bi)T
Bi(N i)−1 (
Bi)T
CT = CT Bi(N i)−1 (
Bi)T
Bi(N i)−1 (
Bi)T
DT = DT Bi(N i)−1 (
Bi)T
Bi(N i)−1 (
Bi)T
ET = ET Bi(N i)−1 (
Bi)T
Bi(N i)−1 (
Bi)T
P 1 = P 1Bi(N i)−1 (
Bi)T
Bi(N i)−1 (
Bi)T
P 2 = P 2Bi(N i)−1 (
Bi)T
(12)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 44: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/44.jpg)
Theorem
The pair of functionsu1
t = −(N1)−1 (
B1)T
y1t ,
u2t = −
(N1)−1 (
B1)T
y2t , t ∈ [0, T ] ,
is one Nash equilibrium point for the above game problem, where(xt, y
1t , y
2t , kt, h
1t , h
2t
)is the solution of the following FBDSDEs:
dxt =
[Axt −B1
(N1)−1 (
B1)T
y1t −B2
(N2)−1 (
B2)T
y2t
+Ckt + αt
]dt
[Dxt + Ekt + βt] dWt − ktd←−Bt,
dy1t = −
[Ay1
t + DT h1t + R1xt
]dt−
(CT y1
t + ET h1t + P 1kt
)d←−Bt
+h1t dWt,
dy2t = −
[Ay2
t + DT h2t + R2xt
]dt−
(CT y2
t + ET h2t + P 2kt
)d←−Bt
+h2t dWt,
x0 = a, y1T = Q1xT , y2
T = Q2xT .(2)
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications
![Page 45: MP for Forward-Backward Doubly Stochastic Control Systems and Applications talks workshop... · 2012-01-09 · Main results in this talk 1 The doubly stochastic maximum principle](https://reader033.fdocuments.net/reader033/viewer/2022060416/5f13af76d1869a19e8340afe/html5/thumbnails/45.jpg)
Thanks for your attention!
Merci!
Liangquan Zhang MP for FBD Stochastic Control Systems and Applications