No-gap second-order optimality conditions for …hermant/HermantCFG07.pdfCf Kawasaki-P´ales-Zeidan...

Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion

No-gap second-order optimality conditions foroptimal control problems with state constraints

Application to the shooting algorithm

Audrey Hermant

CMAP Ecole Polytechnique and INRIA Futurs, France

13th Czech-French-German Conference on OptimizationHeidelberg, September 19, 2007

Joined work with J. Frederic Bonnans

CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 1/25


Outline

Introduction

Definitions

Regularity

Second-order analysis

Shooting algorithm

Remarks & conclusion



Optimal control problem

(P) min(u,y)

∫ T

0`(u(t), y(t))dt + φ(y(T ))

s.t. y(t) = f (u(t), y(t)) a.e. [0,T ], y(0) = y0 (1)

gi (y(t)) ≤ 0 on [0,T ], i = 1, . . . , r

ci (u(t), y(t)) ≤ 0 a.e. [0,T ], i = r + 1, . . . , r + s.

I Control u ∈ U := L∞(0,T ; Rm),state y ∈ Y := W 1,∞(0,T ; Rn).

I Assumption (A0) Data `, φ, f , g , c of class C∞, f Lipschitzcontinuous, gi (y0) < 0, ∀ i = 1, . . . , r .



Problem 1: second-order optimality conditions

I Second-order sufficient conditions (SSC) are useful to: showlocal optimality of solutions, stability/sensitivity analysis,convergence of algorithms.

I SSC as weak as possible if as close as possible to thesecond-order necessary condition (’no-gap’).

I No-gap conditions known for control constraints, mixedcontrol-state constraints (Osmolovskii).

Pure state constraints: is there a gap?Cf Kawasaki-Pales-Zeidan (necessary cond., additional term),Malanowski-Maurer (SSC).



Problem 2: Well-posedness of the shooting algorithm

I Useful algorithm to obtain solutions with a high precision andlow complexity.

I Principle: reduce the problem to a multi-points boundaryvalue problem and solve the finite-dimensional shootingequation using a Newton method.

I Theoretical difficulties due to pure state constraints:reformulation of the optimality conditions, the algorithm takesinto account only a part of the optimality conditions.

Is this algorithm well-posed? (Jacobian of the shooting mappinginvertible).



Structure of the contact set

I Contact set : I (gi (y)) := {t ∈ [0,T ] : gi (y(t)) = 0}.I Junction points of gi : Ti = ∂I (gi (y)).

g(y(t))

t

boundary arc [τ ien, τ

iex ] touch point {τ i

to}

τ ien : entry point, τ i

ex : exit point.

I Similar definitions for mixed control-state constraints.



Order of a state constraint

Definition

The order of the state constraint gi , denoted by qi , is the smallestnumber of derivation of t → gi (y(t)), when y satisfies y = f (u, y),to have an explicit dependence in u.

I More precisely, the time derivatives of gi satisfy:

g(j)i (u, y) = g

(j−1)i ,y (y)f (u, y) = g

(j)i (y), j = 1, . . . , qi − 1

g(qi )i (u, y) = g

(qi−1)i ,y (y)f (u, y), g

(qi )i ,u 6≡ 0.

I For mixed control-state constraints, we set

qi := 0, g(qi )i (u, y) := ci (u, y), i = r + 1, . . . , r + s.



Linear Independence Condition

Set Iε(t) := {i : gi (y(t)) ≥ −ε, ci (u(t), y(t)) ≥ −ε} and

Γ(t) :=(∇ug

(qi )i (u(t), y(t))

)i∈Iε(t)

.

(LIC ) ∃ γ, ε > 0, γ|ξ| ≤ |Γ(t)ξ|, ∀ξ ∈ R|Iε(t)|, ∀t ∈ [0,T ] .

Proposition (Normal form)

Assume that (LIC) holds and u is continuous. Then there exists alocal change of variables z = Φ(y), v = Ψ(u, y) such that, in thenew coordinates, the dynamics and the constraints write locally{

z(qi )i = vi , i = 1, . . . , r

˙z = f (v , z)

zi ≤ 0, i = 1, . . . , r ,vi ≤ 0, i = r + 1, . . . , r + s.



First-order optimality condition

Hamiltonian H : Rm × Rn × Rn∗ × Rs∗ → R,

H(u, y , p, λ) := `(u, y) + pf (u, y) + λc(u, y).

Definition

(u, y) stationary point of (P), if exist p ∈ BV ([0,T ]; Rn∗),dη ∈M([0,T ]; Rr∗) and λ ∈ L∞(0,T ; Rs∗) satisfying (1),

−dp = Hy (u, y , p, λ)dt + dηgy (y), p(T ) = φy (y(T ))

0 = Hu(u(t), y(t), p(t), λ(t)) a.a. t ∈ [0,T ]

0 ≥ g(y(t)), dη ≥ 0,∫ T0 dη(t)g(y(t)) = 0,

0 ≥ c(u(t), y(t)), λ ≥ 0,∫ T0 λ(t)c(u(t), y(t))dt = 0.

(u, y) local solution + (LIC) => (u, y) stationary point .



Assumptions

(A1) The control u is continuous on [0,T ] and the strenghtenedLegendre-Clebsch condition holds: ∃ α > 0,

Huu(u(t), y(t), p(t), λ(t))(v , v) ≥ α|v |2, ∀ v ∈ Rm, ∀ t ∈ [0,T ].

(A2) The Linear Independance Condition (LIC) holds.

(A3) The set of junction times Ti is finite ∀i = 1, . . . , r + s(⇒ finitely many boundary arcs and touch points).

(A4) The junction times of state constraints do not coincide, i.e.i 6= j ⇒ Ti ∩ Tj = ∅, and gi (y(T )) < 0, ∀ i .



Remark: Sufficient condition for the continuity of u

Proposition

Assume that

I The Hamiltonian is uniformly strongly convex w.r.t. u and themixed control-state constraints are convex w.r.t. u, i.e.

∃ α > 0, Huu(u, y(t), p(t), λ)(v , v) ≥ α|v |2,

∀ u, v ∈ Rm, ∀ λ ∈ Rs∗+ , ∀ t ∈ [0,T ].

I The linear independence condition holds for mixedcontrol-state constraints.

Then u is continuous on [0,T ].



Regularity and junction conditions

Proposition

Let (u, y) be a stationary point of (P) satisfying (A1)-(A4). Then

I Outside the set of junction times, u, y , p, η, λ are C∞.

I The multipliers λi , ηi associated with constraints of orderqi = 0, 1 are continuous on [0,T ].

I Let τ ∈ Ti . If qi ≥ 3:

• the time derivatives of u are continuous until order qi − 2.

• If qi is odd, and τ is an entry/exit point, the timederivatives of u are continuous until order qi − 1.

Ref : Jacobson et al. (71) in the scalar case m = r = 1.Maurer (79) for the first item in the case when m = r .



Definitions

Definition

A touch point of a state constraint τ ∈ Ti is essential, if[ηi (τ)] > 0. The set of essential touch points is denoted by T ess

i .

Remark : qi = 1 ⇒ no essential touch points (T essi = ∅).

Definition

A touch point τ of a state constraint of order qi ≥ 2 is reducible, if

g(2)i (u(τ), y(τ)) < 0.

We denote by T redi a finite set (possibly empty) of essential and

reducible touch points.



Assumptions

(A5) • For all entry/exit times τ ∈ Ti of state constraints:if qi is odd (resp. even), the derivative of order 2qi (resp.2qi − 1) of t → gi (y(t)) is discontinuous at τ .

• All essential touch points τ ∈ T essi (qi ≥ 2) are reducible,

i.e.g

(2)i (u(τ), y(τ)) < 0.

(A6) Strict complementarity on boundary arcs for state constraints:

dηi

dt> 0 a.e. on int I (gi (y)).



Abstract formulation

I For u ∈ U , let yu ∈ Y be the unique solution of the stateequation (1) yu = f (u, yu), yu(0) = y0, and let

J(u) :=∫ T0 `(u, yu)dt + φ(yu(T )), G (u) := g(yu),

G(u) := c(u, yu), K := C ([0,T ]; Rr−), K := L∞(0,T ; Rs

−).Then (P) writes

(P) minu∈U

J(u), G (u) ∈ K , G(u) ∈ K.

I Lagrangian

L(u; η, λ) := J(u) + 〈η, G (u)〉+ 〈λ,G(u)〉.

I Given v ∈ V := L2(0,T ; Rm), denote by zv the solution of

zv = fy (u, yu)zv + fu(u, yu)v a.e. [0,T ], zv (0) = 0.



Quadratic form and critical cone

I Set Q(v) := D2uuL(u; η, λ)(v , v) given by

Q(v) =

∫ T

0D2

(u,y)(u,y)H(u, y , p, λ)((v , zv ), (v , zv ))dt

+

∫ T

0gyy (y)(zv , zv )dη + φyy (y(T ))(zv (T ), zv (T )).

I Critical cone C (u): set of v ∈ V satisfying

gi ,y (y)zv = 0 on supp(dηi ),

gi ,y (y)zv ≤ 0 on I (gi (y)) \ supp(dηi ),

ci ,y (u, y)zv + ci ,u(u, y)v = 0 a.e. on supp(λi ),

ci ,y (u, y)zv + ci ,u(u, y)v ≤ 0 a.e. on I (ci (u, y)) \ supp(λi ).



Second-order necessary optimality condition

Theorem

Let (u, y) be a local solution of (P) satisfying (A1)-(A6). Then

Q(v)−r∑

i=1

∑τ∈T ess

i

[ηi (τ)](g

(1)i ,y (y(τ))zv (τ))2

g(2)i (u(τ), y(τ))

≥ 0, ∀v ∈ C (u).

Idea of proof

I Computation of the curvature term by Kawasaki 88, 90 forstate constraints.

I No contribution of mixed constraints (polyhedricity).

I Using the normal form, we combine both arguments.



Second-order sufficient optimality condition

Theorem

Let (u, y) be a stationary point satisfying (A1). If

Q(v)−r∑

i=1

∑τ∈T red

i

[ηi (τ)](g

(1)i ,y (y(τ))zv (τ))2

g(2)i (u(τ), y(τ))

> 0, ∀v ∈ C (u) \ {0}

then (u, y) is a local solution of (P) satisfying the quadraticgrowth condition: ∃ β, r > 0 such that:

J(u) ≥ J(u) + β ‖u − u‖22 ∀ u ∈ U : G (u) ∈ K , ‖u − u‖∞ < r .

Idea of proof: Use of a reduction approach (cf semi-infiniteprogramming).



Characterization of quadratic growth

Corollary

Let (u, y) be a stationary point satisfying (A1)-(A6). Then

Q(v)−r∑

i=1

∑τ∈T ess

i

[ηi (τ)](g

(1)i ,y (y(τ))zv (τ))2

g(2)i (u(τ), y(τ))

> 0, ∀v ∈ C (u) \ {0}

iff (u, y) is a local solution of (P) satisfying the quadratic growthcondition.



The shooting algorithm (unconstrained case)

I By (A1), Hu(u(t), y(t), p(t)) = 0 iff u(t) = Υ(y(t), p(t)).I The first-order optimality condition writes (two-points

boundary value problem):

y = f (Υ(y , p), y), y(0) = y0

−p = Hy (Υ(y , p), y , p), p(T ) = φy (y(T )).

I Shooting algorithm: Find a zero of the shooting mappingp0 → p(T )− φy (y(T )) with

y = f (Υ(y , p), y), y(0) = y0

−p = Hy (Υ(y , p), y , p), p(0) = p0.

I Constrainted case: when the structure of the trajectory isknown, introduce junction times as unknown of the shootingmapping, as well as jump parameters of the costate for stateconstraints (Bryson et al. 63).



Assumptions

(A7) • Mixed constraints have no touch points.

• For all mixed constraints, ddt ci (u, y) is discontinuous at

entry and exit points.

(A8) Strict complementarity:

supp(dηi ) = I (gi (y)), supp(λi ) = I (ci (u, y)).



Well-posedness of the shooting algorithm

Theorem

Let (u, y) be a local solution of (P) satisfying (A1)-(A8). Thenthe shooting algorithm is well-posed in the neighborhood of (u, y)(invertible Jacobian of the shooting mapping), iff:(i) State constraints of order qi ≥ 3 have no boundary arc;(ii) The no-gap sufficient second-order condition

Q(v)−r∑

i=1

∑τ∈T ess

i

[ηi (τ)](g

(1)i ,y (y(τ))zv (τ))2

g(2)i (u(τ), y(τ))

> 0, ∀v ∈ C (u) \ {0}

holds, i.e. (u, y) satisfies the quadratic growth condition.



Verification of sufficient second-order optimality condition

Open problem!Using Riccati equations (cf Maurer), assuming (A1), we can checkthe stronger condition below:

Q(v)−r∑

i=1

∑τ∈T red

i

[ηi (τ)](g

(1)i ,y (y(τ))zv (τ))2

g(2)i (u(τ), y(τ))

> 0, ∀v ∈ C (u) \ {0}

where C (u) ⊃ C (u) is the set of v ∈ V satisfying

ci ,y (u, y)zv + ci ,u(u, y)v = 0 a.e. on boundary arcs,

g(qi )i ,y (u, y)zv + g

(qi )i ,u (u, y)v = 0 a.e. on boundary arcs.



Constraints on the final state

Extension possible to finitely many equality and inequalityconstraints on the final state

Ψeq(y(T )) = 0, Ψin(y(T )) ≤ 0 (2)

if we assume in addition a strong controllability condition:for κ = 2,∞, for all ϕ ∈

∏i W

qi ,κ(0,T )× Lκ(0,T ; Rs) and allµ ∈ R|Ψac |, there exists v ∈ Lκ(0,T ; Rm) such that

gi ,y (y)zv = ϕi on a neighborhood of I (gi (y))

ci ,y (u, y)zv + ci ,u(u, y)v = ϕi on a neighborhood of I (ci (u, y))

DΨac(y(T ))zv (T ) = µ,

with Ψac the equality and active inequality components of (2).



Conclusion & Outlook

I We obtain no-gap second-order optimality conditions for purestate constraints of arbitrary orders and mixed control-stateconstraints, and a characterization of the well-posedness ofthe shooting algorithm.

I Outlook: Numerical applications of the shooting algorithm,using homotopy/continuation methods to automaticallydetect the structure of the trajectory and initialize some of theshooting parameters.

Reference of this talk: J.F. Bonnans, A.H., Second-order analysis for

optimal control problems with pure and mixed state constraints, INRIA

Research Report 6199 (2007), submitted.

http://hal.inria.fr/inria-00148946


No-gap second-order optimality conditions for …hermant/HermantCFG07.pdfCf Kawasaki-P´ales-Zeidan...

Documents

Transcript of No-gap second-order optimality conditions for …hermant/HermantCFG07.pdfCf Kawasaki-P´ales-Zeidan...