Applications of Game Theory - Lake Como School · Di⁄erential games are o⁄springs of game...

Applications of Game TheoryIntroduction to Differential Games

Georges ZaccourChair in Game Theory & Management

GERAD, HEC Montréal

September 5, 2016

G. Zaccour (Chair in Game Theory & Mgt) Differential Games September 5, 2016 1 / 52

Introduction

Differential games are offsprings of game theory and optimal control.

Initiated by R. Isaacs at the Rand Corporation in the late 1950s andearly 1960s.

Initial focal points: military applications and zero-sum games.

Now, applications are found in many areas, e.g., in managementscience (operations management, marketing, finance), economics(industrial organization, macro, resource, environmental economics,etc.), biology, ecology, military, etc.

Textbooks: Basar and Olsder (1982, 1995), Petrosjan (1993),Dockner et al. (2000), Jørgensen and Zaccour (2004), Engwerda(2005), Yeung and Petrosjan (2005), Haurie, Krawczyk and Zaccour(2012).


Elements of a differential game

A deterministic differential game (DG) played on a time interval [t 0,T ]involves the following elements:

A set of players M = {1, . . . ,m} ;For each player j ∈ M, a vector of controls uj (t) ∈ Uj ⊆ Rmj , whereUj is the set of admissible control values for Player j ;

A vector of state variables x(t) ∈ X ⊆ Rn, where X is the set ofadmissible states. The evolution of the state variables is governed bya system of differential equations, called the state equations:

x(t) =dxdt(t) = f (x(t),u(t), t) , x(t0) = x0, (1)

where u(t) , (u1(t), . . . ,um(t));


Elements of a differential game (cont’d)

A payoff for Player j , j ∈ M,

Jj ,∫ T

t0gj (x(t),u(t), t) dt + Sj (x(T )) (2)

where function gj is Player j’s instantaneous payoff and function Sj ishis terminal payoff;

An information structure, i.e., information available to Player j whenhe selects uj (t) at t;A strategy set Γj , where a strategy γj ∈ Γj is a decision rule thatdefines the control uj (t) ∈ Uj as a function of the informationavailable at time t.



Assumption: All feasible state trajectories remain in the interior of the setof admissible states X .

Assumption: Functions f and g are continuously differentiable in x,u andt. The Sj functions are continuously differentiable in x.

Control set: uj (t) ∈ Uj , with Uj set of admissible controls (or controlset).

Control set could be:

Time-invariant and independent of the state;Depend on the position of the game (t, x(t)), i.e.,uj (t) ∈ Uj (t, x(t))).Depend also on controls of other players (coupled constraints).



Information structure:

Open loop: players base their decision only on time and an initialcondition;

Feedback or Markovian: players use the position of the game (t, x(t)) asinformation basis;

Non-Markovian: players use history when choosing their strategies.



Strategies:

Open-loop strategy: selects the control action according to a decision ruleµj , which is a function of the initial state x

0:uj (t) = µj (x

0, t).

As x0 is fixed, no need to distinguish between uj (t) and µj (x0, t). Player

commits to a fixed time path for his control.

Markovian strategy: selects the control action according to a feedback ruleuj (t) = σj (t, x(t)) . Player j’s reaction to any position ofthe system is predetermined.

The decision rule σj can be, e.g., linear or quadratic function of xwith coeffi cients depending on t.

It also can be a nonsmooth function of x and t (e.g., bang-bangcontrols). Complicated problem....



State equations (system dynamics, evolution equations or equations ofmotion):

x(t) =dxdt(t) = f (x(t),u(t), t) , x(t0) = x0,

State vector’s rate of change depends on t, x(t) and u(t).OL strategies are piecewise continuous in time. A unique trajectorywill be generated from x0.For feedback strategies, we make the following simplifyingassumption:

Assumption For every admissible strategy vector σ = (σj : j ∈ M), theDE x(t) admit a unique solution, i.e., a unique statetrajectory, which is an absolutely continuous function of t.

Assumption met when: (i) f (x(t),u(t)) is continuous in t for each x anduj , j ∈ M; (ii) f (x(t),u(t), t) is uniformly Lipschitz in x , u1, . . . ,um ; and(iii) σj (t, x) is continuous in t for each x and uniformly Lipschitz in x.



Time horizon:

T can be finite or infinite;

T can be prespecified or endogenous (as in, e.g., pursuit-evasiongames and patent-race games).


Nash Equilibrium: The definition

Normal form representation: Set of players’admissible strategies;payoffs expressed as functions of strategies rather than actions.

Assume that Player j , j ∈ M, maximizes a stream of discounted gains,that is,

Jj ,∫ T

t0e−ρj tgj (x(t),u(t), t) dt + e−ρjT Sj (x(T )), (3)

where ρj is the discount rate satisfying ρj ≥ 0.


Nash Equilibrium: The definition (cont’d)

Open-loop Nash equilibrium.The payoff functions with the state equations and initial data (t0, x0)define the normal form of an OL differential game:

u(·) = (u1(·), . . . ,uj (·), . . . ,um(·)) 7→ Jj (t0, x0;u(·)), j ∈ M. (4)

DefinitionThe control m-tuple u∗(·) = (u∗1(·), . . . ,u∗m(·)) is an open-loop Nashequilibrium (OLNE) at (t0, x0) if the following holds:

Jj (t0, x0;u∗(·)) ≥ Jj (t0, x0; [uj (·),u∗−j (·)]), ∀uj (·), j ∈ M,

where uj (·) is any admissible control of Player j and [uj (·),u∗−j (·)] is them-vector of controls obtained by replacing the j-th component in u∗(·) byuj (·).



Open-loop Nash equilibrium.Player j solves the optimal-control problem

maxuj (·)

{∫ T

t0e−ρj tgj

(x(t), [uj (t),u∗−j (t)], t

)dt + e−ρjT Sj (x(T ))

},

subject to the state equations

x(t) =dxdt(t) = f

(x(t), [uj (t),u∗−j (t)], t

), x(t0) = x0. (5)



Markovian (feedback)-Nash equilibrium.Players use feedback strategies σ(t, x) = (σj (t, x) : j ∈ M). The normalform of the game, at (t0, x0) is defined by

Jj (t0, x0; σ) =∫ T

t0e−ρj tgj (σ(t, x), t) dt + e−ρjT Sj (x(T )),

x(t) =dxdt(t) = f (x(t), σ(t, x(t)), t) , x(t0) = x0.

Define the (m− 1)−vector

σ−j (t, x(t)) , (σ1(t, x(t)), . . . , σj−1(t, x(t)), σj+1(t, x(t)), . . . , σm(t, x(t))) .



Markovian (feedback)-Nash equilibrium.

DefinitionThe feedback m-tuple σ∗(·) = (σ∗1(·), . . . , σ∗m(·)) is a feedback orMarkovian-Nash equilibrium (MNE) on [0,T ]× X if for each (t0, x0) in[0,T ]× X , the following holds:

Jj (t0, x0; σ∗(·)) ≥ Jj (t0, x0; [σj (·),σ∗−j (·)]; ), ∀σj (·), j ∈ M,

where σj (·) is any admissible feedback law for Player j and [σj (·),σ∗−j (·)]is the m-vector of controls obtained by replacing the j-th component inσ∗(·) by σj (·).



Markovian (feedback)-Nash equilibrium.In other words, u∗j (t) ≡ σ∗j (t, x

∗(t)), where x∗(·) is generated by σ∗ from(t0, x0), solves the optimal-control problem

maxuj (·)

{∫ T

t0e−ρj tgj

(x(t),

[uj (t),σ∗−j (t, x(t))

], t)dt

+e−ρjT Sj (x(T )),}

s.t. x(t) = f (x(t),[uj (t),σ∗−j (t, x(t))

], t), x(t0) = x0.


Nash Equilibrium: Identification of a MNE with maximumprinciple

Theorem

Assume functions Sj are continuously differentiable and concave. Suppose thatan m-tuple σ∗ = (σ∗1, . . . , σ∗m) of feedback law functions σj : X × [t0,T ] 7→Rmj , j ∈ {1, . . . ,m}, is such that

(i) σ∗ (t, x) is continuously differentiable in x almost everywhere, andpiecewise continuous in t;

(ii) σ∗ (t, x) generates at (t0, x0) a unique trajectoryx∗(·) : [t0,T ]→ X , solution of

x(t) = f (x(t), σ∗(t, x), t), x(t0) = x0,

which is absolutely continuous and remains in the interior of X ;


Theorem

(iii) there exist m costate vector functions λj (·) = [t0,T ]→ IRn ,which are absolutely continuous and such that, for allj ∈ {1, . . . ,m}, if we define the Hamiltonians

Hj (x∗ (t) ,λj (t) , [uj ,u−j ], t) = gj(x∗(t), [uj ,u−j ], t

)+λ′j (t)f (x

∗(t), [uj ,u−j ], t),

and the maximized Hamiltonians

H∗j (x∗(t),λj (t), t) = max

uj∈Uj

{gj(x∗(t)), [uj ,σ∗−j (t, x

∗(t))], t)

+ λ′j (t)f (x∗(t), [uj , σ∗−j (t, x

∗(t))], t)},

then the maximum is reached at σ∗j (t, x∗(t)), i.e.,

H∗j (x∗(t),λj (t), t) = gj (x∗(t)), σ∗(t, x(t)), t) (6)

+λ′j (t)f (x∗(t), σ∗(t, x∗(t)), t);


Theorem

(iv) that the functions x 7→ H∗j (x ,λj (t), t) where H∗j is defined as

above but at position (t, x), are continuously differentiable andconcave for all t ∈ [t0,T ] and j ∈ M.

(v) that the following adjoint differential equations are satisfied foralmost all t ∈ [t0,T ] :

λj (t) = ρjλj (t)−∂H∗j (x

∗(t),λj (t), t)

∂x′, (7)

and the so-called transversality conditions

λj (T ) =∂Sj

∂x′(T )(x(T ),T ) , (8)

are also satisfied.

Then (σ∗1, . . . , σ∗m) is an MNE at (t0, x0).


Nash Equilibrium: Identification of an MNE with maximumprinciple

Main diffi culty:solving the adjoint equations. They depend on theunknown strategies σ∗−j (t, x(t))) of the rival players:

λj (t) = ρjλj (t)−∂H∗j (x

∗(t),λj (t), t)

∂x′

= ρjλj (t)−∂

∂x′Hj (x∗(t),λj (t),

[u∗j (t),σ

∗−j (t, x

∗(t))], t)

− ∂

∂u−jHj (x∗(t),λj (t),

[u∗j (t),σ

∗−j (t, x

∗(t))], t)

∂

∂x′σ∗−j (t, x

∗(t)).

Complication: term involving the partial derivatives of the unknownstrategies of the −j players, i.e., ∂

∂x′σ∗−j (t, x

∗(t)).


Nash Equilibrium: Identification of an OLNE withmaximum principle

OL strategy is a (special) feedback law that does not vary with x.Previous Theorem is applicable.Adjoint equations are tractable because ∂σ−j (t ,x(t)))

∂x′ vanish. We have

λj (t) = ρjλj (t)−∂

∂x′gj (x(t),u∗(t), t)− λ′j (t)

∂

∂x′f (x(t),u∗(t), t) ,

λj (T ) =∂Sj

∂x′(T )(x(T )) .

Adjoint equations with state equation

x(t) = f (x(t),u∗(t), t), x(t0) = x0,

define a two-point boundary-value problem (TPBVP), which consists of 2nODE with n initial and n terminal conditions.


Nash Equilibrium: Identification of an MNE with HJB

TheoremSuppose that an m-tuple σ∗ = (σ∗1, . . . , σ∗m) of feedback laws is given, such that

(i) for any admissible initial point (t0, x0) there is a unique,absolutely continuous solution t ∈ [t0,T ] 7→ x∗(t) ∈ X ⊂ IRnof the differential equation

_x∗(t) = f (x∗(t), σ∗1(t, x∗(t)), . . . , σ∗m (t, x

∗(t)) , t), x∗(t0) = x0;

(ii) there exist continuously differentiable functionalsV ∗j : [t0,T ]× X → IR, such that the following HJB equationsare satisfied for all (t, x) ∈ [t0,T ]× X

ρjV∗j (t, x)−

∂V ∗j (t, x)∂t

= maxuj∈Uj

{gj(x, [uj ,σ∗−j (t, x)], t

)+

∂V ∗j (t, x)∂x

f (x,[uj ,σ∗−j (t, x)

], t)

}(9)

= gj (x, σ∗(t, x), t) +∂V ∗j (t, x)

∂xf (x, σ∗(t, x), t); (10)



Theorem

(iii) the boundary conditions

V ∗j (T , x) = Sj (x) (11)

are satisfied for all x ∈ X and j ∈ M.Then, σ∗j (t, x), is a maximizer of the right-hand side of theHamilton-Jacobi-Bellman equation for Player j and the m−tupleσ∗ = (σ∗1, . . . , σ∗m) is an MNE at every initial point (t0, x0) ∈ [t0,T ]× X .



V1(t, x(t)), . . . ,Vm(t, x(t)), are the players’equilibrium valuefunctions

Main diffi culty: solving the HJB equations. These are PDE (andpossibly highly nonlinear) involving the functions Vj (t, x(t)), j ∈ M.To obtain a solution for Vj , we need to know the strategies of allother players “−j”and these strategies are usually not available.When the state space is one-dimensional, HJB becomes an ODE(easier to solve than a PDE);

In infinite-horizon differential games with stationary strategies, theHJB equations are algebraic equations.


Nash Equilibrium: Remarks on the two theorems

RemarkBoth theorems provide suffi cient conditions, that is, given that theassumptions of a theorem can be verified, we have identified an MNE.

RemarkUnder the additional regularity condition that the value functions Vj (t, x)are twice differentiable in x, it can be established that costate variablesλj (t) are equal to the partial derivatives of the value function along theequilibrium trajectory, i.e.,

∂Vj (t, x)∂x′

|x=x∗(t) = λj (t) .


Nash Equilibrium: Infinite-horizon case

Issue: integral payoff may not converge for all feasible control andtrajectory paths (u(·), x(·)).

Jj ,∫ ∞

t0gj (x(t),u(t), t) dt, (12)

Several alternative objective functionals:Discounted payoff

Jj ,∫ ∞

0e−ρj tgj (x(t),u(t)) dt, (13)

Limit of average payoff, when the reward does not explicitly depend on t

Jj , limT→∞

inf1T

∫ T

0gj (x(t),u(t)) dt. (14)

Focus on autonomous problems, i.e., f (system’s dynamics) and gj(instantaneous utility or payoff) do not depend explicitly on time, andconfine our interest to stationary feedback strategies: uj (t) = σj (x(t)).


Nash Equilibrium: Infinite-horizon case

PropositionWhen Player j , j ∈ M, optimizes the discounted payoff functional in (13),then the (suffi cient) transversality conditions in (11) and (8) become,respectively,

limT→∞

e−ρj tVj (t, x(t)) = 0, (15)

limT→∞

e−ρj tλj (t) = 0. (16)


An infinite horizon example

Let the differential game data be given by

Jj ,∫ ∞

0e−ρt

(uj (t)

(κ − 1

2uj (t)

)− 12

ϕx2 (t))dt

s.t.

x(t) = u1 (t) + u2 (t)− αx(t), x(0) = x0,

where ϕ and κ are positive parameters and 0 < α < 1.


An infinite horizon example: OLNE

Hamiltonian of Player j :

Hj (x ,λj , u1, u2) = uj

(κ − 1

2uj

)− 12

ϕx2 + λj (u1 + u2 − αx), j = 1, 2,

Necessary and suffi cient (maximized Hamiltonian is concave in x)equilibrium conditions:

∂Hj∂uj

= 0 ⇔ uj = κ + λj ,

λj = ρλj −∂Hj∂x

= (ρ+ α)λj + ϕx , limt→∞

e−ρtλj (t) = 0,

x = u1 + u2 − αx , x(0) = x0.


An infinite horizon example: OLNE

Consider

∂Hj∂uj

= 0 ⇔ uj = κ + λj ,

λj = ρλj −∂Hj∂x

= (ρ+ α)λj + ϕx , limt→∞

e−ρtλj (t) = 0,

Clearly λ1 (t) = λ2 (t) , ∀t ∈ [0,∞), and therefore,u1 (t) = u2 (t) , ∀t ∈ [0,∞). Replacing the expression

u1 = u2 = κ + λ, (17)

in x and λ, we get a two-equation differential system:(xλ

)=

(−α 2ϕ ρ+ α

)(xλ

)+

(2γ0

).


An infinite horizon example: OLNE (cont’d)

The solution is given:

x(t) = (x0 − xss )eω1t + xss ,

λ(t) = −(x0 − xss )2ϕ

2α+ ρ+√(2α+ ρ)2 + 8ϕ

eω1t + λss ,

where the steady state is given by

(xss ,λss ) =(

2κ(α+ ρ)

α2 + αρ+ 2ϕ,− 2κϕ

α2 + αρ+ 2ϕ

),

and ω1 is the negative eigenvalue of the matrix associated with thedifferential-equations system:

ω1 =12(ρ−

√(2α+ ρ)2 + 8ϕ).

It suffi ces to replace λ(t) by its value in (17) to obtain the equilibriumstrategies.


An infinite horizon example: MNE


Jj ,∫ ∞

0e−ρt

(uj (t)

(κ − 1

2uj (t)

)− 12

ϕx2 (t))dt

s.t.

x(t) = u1 (t) + u2 (t)− αx(t), x(0) = x0,

The HJB equation for Player j is

ρVj (x) = maxuj≥0

[uj

(κ − 1

2uj

)− 12

ϕx2 + V ′j (x) (u1 + u2 − αx)]. (18)

Differentiating the RHS and equating to zero yields

uj (x) = κ + V ′j (x). (19)


An infinite horizon example: MNE (cont’d)

Game is symmetric. We focus on symmetric equilibrium strategies. Gameis linear-quadratic, same quadratic value for both players

Vj (x) =A2x2 + Bx + C , i = 1, 2.

We haveuj (x) = uj (x) = κ + V ′j (x) = κ + Ax + B.

Recall in the open-loop case, we had

uj = κ + λj .

Replacing for uj (x) = κ + Ax + B, into the RHS of HJB, we get

ρ

(A2x2 + Bx + C

)=12

(3A2 − 2Aα− ϕ

)x2+

(3AB − Bα+ 2Aκ)x +12

(3B2 + 4Bκ + κ2

).


An infinite horizon example: MNE (cont’d)

Solving, we get

A =ρ+ 2α±

√(ρ+ 2α)2 + 12ϕ

6,

B =−2Aκ

3A− (ρ+ α), C =

κ2 + 4Bκ + 3B2

2ρ.

Selection of the negative root

A =ρ+ 2α−

√(ρ+ 2α)2 + 12ϕ

6,

guarantees the global stability of the state trajectory given by

x(t) =[x0 +

2(κ + B)2A− α

]e(2A−α)t − 2(κ + B)

2A− α.

Indeed, the state dynamics of the game have a globally asymptoticallystable steady state if 2A− α < 0.



We could have used the max principle to determine a feedback-Nashequilibrium. Hamiltonian of Player j :

Hj (x ,λj , u1, u2) = uj

(κ − 1

2uj

)− 12

ϕx2 + λj (u1 + u2 − αx), j = 1, 2.

Assuming an interior solution, the necessary equilibrium conditions

∂Hj∂uj

= 0 ⇔ uj = κ + λj ,

x = u1 + u2 − αx , x(0) = x0,

are as before. However, the costate equations now read (see Eq. (7))

λj = ρλj −∂Hj∂x− ∂Hj

∂ui

∂u∗i∂x

= (ρ+ α)λj + ϕx − λj∂u∗i∂x,

with the transversality conditions

limt→∞

e−ρtλj (t) = 0, j = 1, 2.



The term ∂Hj∂ui

∂u∗i∂x , which does not appear in the open-loop case, shows the

interaction between the control of the other player and the state variable.In general, we do not know the shape u∗i (x) and hence the value of the

derivative ∂u∗i∂x . Suppose, as we did in HJB characterization, that Player i’s

strategy is linear in the state and hence its derivative is a constant Zi .Invoking symmetry, we assume that Z1 = Z2 = Z . Therefore, the costateequations become

λj = ρλj −∂Hj∂x− ∂Hj

∂ui

∂u∗i∂x

= (ρ+ α− Z )λj + ϕx , j = 1, 2.

Solving the above differential equations gives λj as function of x , say

k (x). Substituting in the maximizing condition ∂Hj∂uj= 0, yields

uj = κ + k (x) ,

which is the desired markovian equilibrium strategy.G. Zaccour (Chair in Game Theory & Mgt) Differential Games September 5, 2016 35 / 52

A finite horizon example: OLNE


Jj ,∫ T

t0

(κ

2u2j (t)−

ϕ

2x2 (t)

)dt, j ∈ M, κ, ϕ > 0

x(t) = u1 (t) + u2 (t)− αx(t), x(0) = x0 > 0, 0 < α < 1.

Hamiltonian of Player j = 1, 2:

Hj (x ,λ1, u1, u2) =κ

2u2j −

ϕ

2x2 + λj (u1 + u2 − αx) ,

Maximizing the Hamiltonians with respect to u1 and u2, respectively, yields

uj = −λjκ, j = 1, 2. (20)

The costate equations and their transversality conditions are

λj = ϕx + αλj , λj (T ) = 0, j = 1, 2.



The above differential equations

λj = ϕx + αλj , λj (T ) = 0, j = 1, 2.

are independent of the control variables, and clearlyλ1 (t) = λ2 (t) ≡ λ (t) , ∀t ∈ [0,T ] . Therefore, we need to solve only oneadjoint equation:

λ = ϕx + αλ, λ(T ) = 0.

Therefore, u1 (t) = u2 (t) ≡ u (t) , ∀t ∈ [0,T ]. Substituting theequilibrium strategies u into the state equation yields

x = −2λ

κ− αx , x(0) = x0.



u∗j are independent of x , and the Hamiltonians are concave in x . Then themaximized Hamiltonians are concave in x and, by previousTheorem, strategies u∗j (t) provide an OLNE.Two-point boundary-value problem (TPBVP). In matrix form, this systemreads (

xλ

)=

(−α −2/κϕ α

)(xλ

),

with x(0) = x0 and λ(T ) = 0.



This problem can be solved explicitly by standard methods of lineardifferential equations. It is easy to verify (for instance, using any of theavailable software) that the solution is given by

x (t) = x0(α+ Z ) e(T−t)Z − (α− Z )(α+ Z ) eTZ − (α− Z ) e−tZ ,

λ (t) =x0 (α+ Z ) (α− Z )

(e(t−T )Z − 1

)β [(α+ Z ) eTZ − (α− Z ) e−TZ ] ,

where Z =√

α2 − βϕ. It suffi ces to substitute for λj (t) in (20) to obtainthe equilibrium strategies uj (t).


A finite horizon example: MNE

We use Theorem (based on HJB) to identify an equilibrium withnondegenerate Markovian strategies σ1(t, x(t)) and σ2(t, x(t)). Introducevalue functions Vj (t, x(t)) and the HJB equations:

−∂Vj (t, x)∂t

= maxuj

{κ

2u2j −

ϕ

2x2 +

∂Vj (t, x)∂x

(u1 + u2 (t)− αx)},

Vj (T , x (T )) = 0.

Maximization of the RHS yields

σj (t, x) = −1κ

∂Vj (t, x)∂x

, j = 1, 2. (21)

Notice that costates λj (t) of the open-loop case have now been replaced

by the derivatives ∂Vj (t ,x )∂x . This was expected since both quantities

represent a shadow price of the stock x .



Inserting the equilibrium strategies σj (t, x) into the RHS of the HJBequations yields two partial differential equations for the value functionsVj (t, x), j = 1, 2 :

−∂Vj (t, x)∂t

=12κ

(∂Vj (t, x)

∂x

)2− ϕ

2x2 +

∂Vj (t, x)∂x

(−1κ

∂V1(t, x))∂x

− 1κ

∂V2(t, x)∂x

− αx),

Vj (T , x(T )) = 0.

As the game is symmetric, we focus on symmetric strategiesσ1(t, x) = σ2(t, x) = σ(t, x) and symmetric value functions. The aboveequations now read

−∂V (t, x)∂t

=

{− 34κ

(∂V (t, x)

∂x

)2− ϕ

2x2 − αx

∂V (t, x)∂x

},

V (x ,T ) = 0.



The differential game at hand is homogenous linear-quadratic, and it canbe shown that the quadratic value function

V (t, x) =12vx2,

solves the HJB equations, provided that the time function v satisfies thefollowing Riccati differential equation:

v(t) =32κv2(t) + ϕ+ 2αv(t), v(T ) = 0.

It suffi ces to substitute the solution of the above differential equation in(21) to get the strategy of Player j , j = 1, 2. As expected, in afinite-horizon setting, the decision rule changes over time via thetime-dependency of coeffi cient v .


Time consistency and subgame perfectness

Let (t i , xi ) ∈ [0,T ]× X be any initial position.We call subgame Φ(t i , xi ), the differential game defined on the timeinterval [t i ,T ] by the following data:

For s ∈ [t i ,T ], the system dynamics are described by

x(s) = f (x(s),u1(s), . . . ,um(s), s), x(t i ) = xi ;

The payoff functional of Player j is given by∫ T

t ie−ρj (s−t)gj (x(s),u1(s), . . . ,um(s), s) ds + e−ρj (T−t)Sj (x(T )).

The original game can be considered as the subgame Φ(0, x0).



By construction, an MNE, or feedback-Nash equilibrium σ∗ is anequilibrium for all subgames Φ(t i , xi ) at all (t i , xi ) ∈ [0,T ]× X . Wesay that an MNE is a subgame perfect equilibrium.

Consider an OLNE for the game Φ(0, x0), and the equilibriumtrajectory x∗(·) that is generated by these equilibrium open-loopstrategies.

Then for every position (t, x∗(t)) along the equilibrium trajectory, therestriction of the equilibrium open-loop controls on the time interval[t,T ] is still an OLNE for the game Φ(t, x∗(t)). We say that anOLNE is a time-consistent equilibrium.



Remark1 Every subgame-perfect equilibrium is time consistent;2 An MNE is subgame perfect and hence also time consistent;3 An OLNE is time consistent but, in general, not subgame perfect.

Time consistency, subgame perfectness and credibility

time consistency can be seen as a minimal requirement and subgameperfectness a maximal one.

Even if the players found themselves, for whatever reason, at position(t, x), which is off the equilibrium-state trajectory, subgameperfectness provides the (strong) insurance that the equilibrium of thegame Φ(t, x) is still the one announced at initial position (0, x0) forgame Φ(0, x0).


Differential games with special structures

Tractable differential games: linear-quadratic, linear-state and exponentialdifferential games.Let the payoff of Player j ∈ M, be given by

Jj ,∫ T

t0gj (x(t), u(t), t) dt + Sj (x(T )),

and the state dynamics by

x(t) =dxdt(t) = f (x(t), u(t), t) , x(t0) = x0,

with u(t) , (u1(t), . . . , um(t)).Linear-quadratic differential games (LQDG).

1 gj (·) and Sj (·) are quadratic in the state and control variables;2 f (·) is linear in the state and control variables.



For LQDG, it holds that Player j’s MNE equilibrium strategy is alinear feedback law, i.e.,

uj (t) = σj (t, x(t))) = aj (t)x(t) + bj (t),

and the value function is quadratic in x ,

Vj (t, x(t)) =12Aj (t)x2 (t) + Bj (t)x(t) + Cj (t),

Coeffi cients aj (t), bj (t),Aj (t),Bj (t) and Cj (t) depending on time.

The solution of HJB equations involves a set of coupled Riccatidifferential equations.



If the LQDG is homogenous, then Player j’s strategy is reduced to

σj (t, x(t))) = aj (t)x(t)

and the value function to

Vj (t, x(t)) =12Aj (t)x2 (t) .

If the game is autonomous and played over an infinite horizon, thenthe coeffi cients

aj (t), bj (t),Aj (t),Bj (t) and Cj (t)

are constant.



Jj ,∫ T

t0gj (x(t), u(t), t) dt + Sj (x(T )),

x(t) =dxdt(t) = f (x(t), u(t), t) , x(t0) = x0,

Linear-state differential games1 gj (·) and Sj (·) are linear in the state variable;2 f (·) is linear in the state variable.

It holds that Player j’s MNE equilibrium strategy is constant

uj (t) = cst, and Vj (t, x(t)) = Dj (t)x(t) + Ej (t),

where Dj (t) and Ej (t) are time functions that can be determined.If T = ∞ game is autonomous, then Dj (t) = Dj and Ej (t) = Ej .Important: because the feedback strategies are degenerate, open-loopand feedback-Nash equilibria coincide.



Examples of recent contributions with tractable DGEngwerda (2005),

General Martín-Herrán and Zaccour (2005),Rincón-Zapatero and Martín-Herrán (2003),Jørgensen, Martín-Herrán and Zaccour (2003, 2005),Martín-Herrán and Rincón-Zapatero (2002)

Macroeconomics Di Bartolomeo et al. (2006), Luckraz (2006),Engwerda et al. (2002), Kemp et al. (2001)

Environmental Breton, Zaccour, Zahaf (2006b), Cabo et Martín-Herrán (2006),Economics Martín-Herrán, et al. (2006), Cabo et al. (forthcoming),

Fredj, Martín-Herrán and Zaccour (2004, 2006),Fernández (2002), Rubio and Casino (2002),List and Mason (2001)Cellini and Lambertini (2005a), Rubio (2006),



Capital accumulation Calzolari and Lambertini (2006),Cellini and Lambertini (2006), Figuieres (2002)

Advertising Bass et al. (2005a,b), Prasad and Sethi (2004),Fruchter (2001)

Shelf-space allocation Martin-Herran and Taboubi (2005a,b),Martín-Herrán et al. (2005)

Marketing channels Jørgensen, Taboubi, Zaccour (2006, 2003),Rubel and Zaccour (2006),Jørgensen, Sigue, Zaccour (2001a,b)Jørgensen and Zaccour (2003)

Resource economics Long and Sorger (2006), Sorger (2005),Martín-Herrán and Rincón-Zapatero (2005),Rubio and Casino (2003, 2001),Rubio and Escriche (2001)



Industrial Organisation Jun and Vives (2004), Laussel et al. (2004),Benchekroun (2003), Miravete (2003),Thepot (2002)

Pricing Fruchter and Tapiero (2005),Cellini and Lambertini (2006),Dockner and Fruchter (2004)Kim and El Ouardighi (2006),

R&D Cellini and Lambertini (2005b),Breton, Turki, Zaccour (2004a,b)


Applications of Game Theory - Lake Como School · Di⁄erential games are o⁄springs of game...

Documents

Transcript of Applications of Game Theory - Lake Como School · Di⁄erential games are o⁄springs of game...