Computation of Value Functions in

Nonlinear Differential Games

N. Botkin, K.-H. Hoffmann, N.Mayer, V.Turova

Technische Universität München, Department of Mathematics Chair of Mathematical Modelling

25th TC7 International Conference on System Modeling and Applied Optimization September 12 – 16, 2011, Berlin, Germany

• Formulation of the problem

• Diverse objective functionals

• Viscosity solutions of H.-J. equations

• Numerical procedure, time step operator

• Convergence result

• Examples and application

• Algorithm efficiency

Outline

1/20

Differential game. Statement of the problem

2/20

Conflict-control system

Payoff functional

Payoff function

Differential game. Statement of the problem 3/20

Objectives of the players

1st player (control ) minimizes the payoff functional

2nd player (control ) maximizes the payoff functional

Feedback strategies

Bundles of step-by-step solutions

N.N.Krasovskii, A.I.Subbotin, Game-Theoretical Control Problems, New York: Springer, 1988. A.I.Subbotin, A.G.Chentsov, Optimization of Guaranteed Result in Control Problems, Moscow: Nauka, 1981.

Step-by-step solutions

Existence of value function 4/20

is Lipschitzian in

uniformly continuous and bounded on

is bounded and Lipschitzian in

satisfies the saddle point condition

Value function

Standard assumptions

is bounded and Lipschitzian in

Hamilton-Jacobi-Bellman-Isaacs-equation 5/20

with terminal condition

Value function is a unique viscosity solution of

the H-J-B-I equation

Hamiltonian

Our intention: to solve (1)-(2) numerically

(1)

(2)

Other functionals

6/20

(a)

(b)

(c)

Meaning:

(a) result at time

(b) result by time

(c) result by time subject to the state constraint

Explanation to the functional (c):

7/20

Consider the following target and state constraint sets:

such that

1. which means that

2. which means that

for some

for all

,

.

Viscosity solutions 8a/20

(i) for any ; ,

(ii) for any point

and function s.t. attains a local minimum at

the following inequality holds

(iii) for any point

and function s.t. attains a local maximum at

the following inequality holds

A Lipschitz function is the value function of (1)+(a) if and only if

Crandall, Lions, Subbotin, Taras‘ev Numerics: Souganidis, Ushakov

Viscosity solutions 8b/20

(i) for any ; ,

(ii) for any point s.t.

and function s.t. attains a local minimum at

the following inequality holds

(iii) for any point

and function s.t. attains a local maximum at

the following inequality holds

A Lipschitz function is the value function of (1)+(a) if and only if

Crandall, Lions, Subbotin, Taras‘ev Numerics: Souganidis, Ushakov

Numerics: Bardi, Botkin, Falcone, Mitchell, Ushakov,…

and

Viscosity solutions 8c/20

(i) for any ; and

(ii) for any point s.t.

and function s.t. attains a local minimum at

the following inequality holds

(iii) for any point s.t.

and function s.t. attains a local maximum at

the following inequality holds

A Lipschitz function is the value function of (1)+(a) if and only if

Crandall, Lions, Subbotin, Taras‘ev Numerics: Souganidis, Ushakov

Numerics: Botkin, Ushakov, Bardi, Falcone, Mitchell

Numerics: B.-H.-M.-T. (accepted for publication in „Analysis“)

Numerical schemes 9/20

Discretization:

Upwind operator

where

are the RHS of the control system

are the right and left divided differences

with spatial steps

O.A.Malafeyev, M.S.Troeva, A weak solution of Hamilton-Jacobi equation for a differential two-person zero-sum game, in: Proceedings of 8th Int. Symp. on Differential Games and Applications, Maastricht, 1998.

Numerical schemes 10/20

• Cost functional (a): fixed terminal time

• Cost functional (b): nonfixed terminal time

• Cost functional (c): nonfixed terminal time and state constraint

Convergence 11/20

Theorem

Let be a bound of the right-hand side of the control system.

If , then the grid functions obtained by the procedures

(a), (b), and (c) converge pointwise to the value functions of problem (1)

with the corresponding cost functionals as , ,

and the convergence rate is .

Proof 1. Monotonicity of the operator :

2. Generator property of the operator :

for any , and fixed .

12/20 Example 1: two dimensions

target set

State constraint

target set

without state constraint with state constraint

(pursuer)

(evader)

Level sets:

Acoustic version of the homicidal chauffeur game

13/20

Example 2: three dimensions

Game of two cars with state constraints

P

E

y

x

Vp

VE

(x,y)

pursuer

evader

Target:

A.W.Merz, The game of two identical cars, JOTA 9(5),1972

,

State

constraint

without state

constraint

Example 3: four dimensions

Isotropic rockets

Target:

target

14/20

R.Isaacs, Differential Games, John Wiley, New York, 1965.

Grid:

Time step:

15/20

Optimal freezing of living cells

Application

,

.

is the “chamber” temperature, the cooling rate, errors in data

are the internal energies of the extra- and intracellular spaces ,

interpreted as disturbances:

are the dependencies of the temperatures on the internal energies,

estimates the difference of

the ice content in the intra-

and extracellular regions

Functions are recovered from experiments.

Feedback control

Current time , current position

are the right-hand sides of the controlled system computed at

Extremal aiming

and

16/20

17/20

Value function

Ice fractions inside and

outside the cell

Simulation results (freezing)

Optimal control

18/20

Cell thawing with minimization of osmotic inflow

Functional

concentration of physiologic salt solution

some limiting concentration

State constraints: ° ° °

65 % less water inflow with optimal control!

expresses amount of water inflow into the cell

- salt amount in the cell

- initial cell volume

- water volume in the cell

- salt concentration in the cell

Algorithm efficiency 19/20

Linux SMP computer with 8xQuad-Core AMD Opteron processors, shared 64 GB memory.

Time step 0.001, number of steps 5100.

2D

3D

20/20

References

P.E.Souganidis, Approximation schemes for viscosity solutions of Hamilton-Jacobi equations, Journal of Differential Equations 59, pp. 1-43, 1985.

P.E.Souganidis, Max-min representations and product formulars for the viscosity solutions of Hamilton-Jacobi equations with applications to differential games, Nonlinear Analysis, Theory, Methods and Applications 9, pp. 217-257, 1985.

E.Cristiani, M.Falcone, Fully-discrete schemes for value function of pursuit-evasion games with state constraints, in: P. Bernhard, V. Gaitsgory, O.Pourtalier (eds.), Advances in Dynamic Games and Their Applications, Annals of the Int. Society of Dynamic Games X, Birkhäuser, Boston, pp. 177-206, 2009.

S.V.Grigor´eva, V.Yu.Pakhotinskikh, A.A.Uspenskii, V.N.Ushakov, Construction of solutions in certain differential games with phase constraint, Mat. Sbornik 196(4), pp. 51-78, 2005.

M.Bardi, S.Koike, P.Soravia, Pursuit-evasion games with state constraints: dynamic programming and discrete-time approximations, Discrete and Continuous Dynamical Systems – Series A, 2(6), pp. 361-380, 2000.

P.Cardaliaguet, M.Quincampoix, P.Saint-Pierre, Pursuit differential games with state Constraints, SIAM J. on Control and Optimization, 39, pp. 1615-1632, 2001.

Acknowledgement

The work is supported by the German Research Society (DFG),

Project SPP 1253