Mean field games on networks

Fabio Camilli ("Sapienza" Università di Roma)

joint work with Y.Achdou (Paris 7) and C.Marchi (Padova)

Fabio Camilli (“Sapienza" Univ. di Roma) Mean field games on networks Bayreuth, settembre 2013 1 / 28

The Mean Field Games (MFG) model was proposed by Lasry-Lions,and independently by Huang-Malhamé-Caines, in 2006.

Distinctive features of the model:

The MFG theory is a model to describe interactions among a verylarge number of agents. Aim of MFG theory is to show how torelate individual actions to mass behavior, typically in thesituations where there is a collective behavior (fashion trends,Mexican wave, financial crisis, crowd dynamics,...).The MFG model shares some analogy with Statistical Mechanics,where an external field (usually a statistics of some given physicalquantity) influences the behavior of the particles. But in MFGtheory the agent is not a black-box, since it can decide a strategybased on a set of preferences.The single agent by itself cannot influence the collective behavior,it can only optimize its own strategy given the environmentalsituation. The mean field is given by the collective behavior of thepopulation.

The mathematical model: from microscopic to macroscopic

The MFG model is built in two steps:1st step: Description of a differential game for a finite number ofagents.2nd step: Limit for the number of the agents to∞ to obtain themacroscopic model describing the collective behavior of the populationwhich is described by a density function.

Main points in the theoretical approach:

• Rigorous justification of the limit procedure and deduction of thelimit system of PDEs;• Study of the limit system: several issues concerning existence,

uniqueness of the solution, asymptotic behavior, etc.• Construction of numerical schemes to compute the solution

The mathematical model: from microscopic to macroscopic

The MFG model is built in two steps:1st step: Description of a differential game for a finite number ofagents.2nd step: Limit for the number of the agents to∞ to obtain themacroscopic model describing the collective behavior of the populationwhich is described by a density function.

Main points in the theoretical approach:

• Rigorous justification of the limit procedure and deduction of thelimit system of PDEs;• Study of the limit system: several issues concerning existence,

uniqueness of the solution, asymptotic behavior, etc.• Construction of numerical schemes to compute the solution

Basic References for MFG theory:Lasry, J.-M., Lions, P.-L. Mean field games. Jpn. J. Math. 2(2007), no. 1, 229-260.Huang, M.; Malhamé, R.; Caines, P. Large population stochasticdynamic games: closed-loop McKean-Vlasov systems and theNash certainty equivalence principle. Commun. Inf. Syst. 6(2006), no. 3, 221-251Lions online course at College de Francewww.college-de-france.fr

P.Cardaliaguet, Notes on Mean Field Games (from P.-L. Lions’lectures at College de France),www.ceremade.dauphine.fr/ cardalia/

N-person games with ergodic cost

Consider N players whose dynamics is

dX it = −αi

tdt + σidW it , X i

0 = x i ∈ Rd , i = 1, . . . ,N

where the disturbs W it are independent brownian motions(very

important!). The i-player choose the control αit to optimize the

long-time average cost control

J i(X , α1, ·, αN) := lim infT→∞1TEx{∫ T

0[Li(αi

t ) + F i(X 1t , ·,X N

t )]dt}

where X = (X 1, . . . ,X N). Hence in this model the player follows adynamic involving some controlling action, but its decision criteriondepends also on the strategy of the other players. What is the optimalstrategy for the player?

Nash equilibria

A group of players is said in Nash equilibrium if each player is makingthe best decision that it can, taking into account the best decisions ofthe others.In mathematical terms, the control α = (αi)i=1,...,N is a Nashequilibrium for the state X = (x1, . . . , xN) if for any i ∈ {1, . . . ,N} andfor any control αi

J i(X , α1, . . . , αi−1, αi , αi+1, . . . , αN) ≤ J i(X , α1, . . . , αi−1, αi , αi+1, . . . , αN)

(note that each player is assumed to know the equilibrium strategies ofthe other players).

Nash equilibria

A group of players is said in Nash equilibrium if each player is makingthe best decision that it can, taking into account the best decisions ofthe others.In mathematical terms, the control α = (αi)i=1,...,N is a Nashequilibrium for the state X = (x1, . . . , xN) if for any i ∈ {1, . . . ,N} andfor any control αi

J i(X , α1, . . . , αi−1, αi , αi+1, . . . , αN) ≤ J i(X , α1, . . . , αi−1, αi , αi+1, . . . , αN)

(note that each player is assumed to know the equilibrium strategies ofthe other players).

Nash equilibria can be characterized via a system of partial differentialequations (Bensoussan-Frehse, J.Reine Angew.Math, 1984)

There exists λ1, . . . , λN ∈ R, v1, . . . , vN ∈ C2(Rd ),m1, . . . ,mN ∈W 1,p(Rd ) (1 ≤ p <∞) s.t.−ν i∆v i + H i(Dv i) + λi =

∫T d−1 F i(x1, . . . , x , . . . , xN)

∏j 6=i mj(x j)dx j

ν i∆mi + div(∂H i

∂p (Dv i)mi)

= 0∫T d v i(x)dx = 0,

∫T d mi(x)dx = 1, mi ≥ 0

Moreover α = (αi)i=1,...,N with αi(X ) = ∂H i

∂p (Dv i(X )) defines a Nashequilibrium for all X ∈ T n and

λi = J i(X , α1, ·, αN) = lim infT→∞ 1T Ex

{ ∫ T0 [Li(αi

t ) + F i(X 1t , ·,X N

t )]dt}

Remark: mi is the asymptotic distribution of the state of the agent i , ifit behaves optimally

Nash equilibria can be characterized via a system of partial differentialequations (Bensoussan-Frehse, J.Reine Angew.Math, 1984)

There exists λ1, . . . , λN ∈ R, v1, . . . , vN ∈ C2(Rd ),m1, . . . ,mN ∈W 1,p(Rd ) (1 ≤ p <∞) s.t.−ν i∆v i + H i(Dv i) + λi =

∫T d−1 F i(x1, . . . , x , . . . , xN)

∏j 6=i mj(x j)dx j

ν i∆mi + div(∂H i

∂p (Dv i)mi)

= 0∫T d v i(x)dx = 0,

∫T d mi(x)dx = 1, mi ≥ 0

Moreover α = (αi)i=1,...,N with αi(X ) = ∂H i

∂p (Dv i(X )) defines a Nashequilibrium for all X ∈ T n and

λi = J i(X , α1, ·, αN) = lim infT→∞ 1T Ex

{ ∫ T0 [Li(αi

t ) + F i(X 1t , ·,X N

t )]dt}

Remark: mi is the asymptotic distribution of the state of the agent i , ifit behaves optimally

If the players are indistinguishable (σi ≡ σ and F i ≡ F ) and couplingterm F depends only on the position of the other players(F (X ) ≡ F

( 1N−1

∑j 6=i δxj

), sending the number of the players to +∞,

one obtains the stationary MFG system−ν∆v + H(Dv) + λ = V [m](x)

ν∆m + div(∂H∂p (Dv)m

)= 0∫

v(x)dx = 0,∫

m(x)dx = 1, m ≥ 0

The first equation is an Hamilton-Jacobi-Bellman equationdescribing the expected value v and the corresponding optimalcontrol for an agent in the state x .The second equation is a Fokker-Planck equation describing thedensity of the players m at x .The coupling is via V [m] in the HJB equation and via ∂H

∂p (Dv) inthe FP equation.

Several phenomena in physics, engineering, chemistry and biologycan be described by some physical quantities defined on networks.

(a) traffic (b) internet (c) blood

Our aim is to consider MFG systems on a network Γ. Inside the edgesthe equations are defined in standard sense (pointwise, weak,viscosity, etc.).Which transition conditions have to be imposed at the vertices of thenetwork in order to have a global well posed problem?

Several phenomena in physics, engineering, chemistry and biologycan be described by some physical quantities defined on networks.

(d) traffic (e) internet (f) blood

Our aim is to consider MFG systems on a network Γ. Inside the edgesthe equations are defined in standard sense (pointwise, weak,viscosity, etc.).Which transition conditions have to be imposed at the vertices of thenetwork in order to have a global well posed problem?

Networks

A network is a connected set Γ consisting of vertices V := {vi}i∈I andedges E := {ej}j∈J connecting the vertices. We assume that thenetwork is embedded in the Euclidian space so that any two edges canonly have intersection at a vertex.

(g) An example of networkFabio Camilli (“Sapienza" Univ. di Roma) Mean field games on networks Bayreuth, settembre 2013 10 / 28

Some Notations

Inci := {j ∈ J : ej incident to vi} is the set of edges incident thevertex vi .A vertex vi is a boundary vertex if it has only one incident edge.We denote by ∂Γ = {vi , i ∈ IB} the set of boundary vertices.A vertex vi is a transition vertex if it has more than one incidentedge. We denote by ΓT = {vi , i ∈ IT} the set of transition verticesThe graph is not oriented, but the parametrization of the arcs ejinduces an orientation on the edges expressed expressed by thesigned incidence matrix A = {aij}i∈I,j∈J

aij :=

1 if vi ∈ ej and πj(0) = vi ,

−1 if vi ∈ ej and πj(lj) = vi ,

0 otherwise.

where πj : [0, lj ]→ R is the parametrization of edge ej .

Some Notations

aij :=

0 otherwise.

Some Notations

aij :=

0 otherwise.

Some functional spaces

u ∈ Lp(Γ), p ≥ 1 if uj ∈ Lp(0, lj) for each j ∈ J. We set

‖u‖Lp :=

∑j∈J

‖u‖Lp(ej )

‖u‖L∞ := supj∈J‖uj‖L∞(ej ).

u ∈W k ,p(Γ), for k ∈ N, k ≥ 1 and p ≥ 1 if u ∈ C0(Γ) anduj ∈W k ,p(0, lj) for each j ∈ J. We set

‖u‖W k,p :=

‖∂ luj‖pLp

u ∈ Ck (Γ), k ∈ N, if u ∈ C0(Γ) and uj ∈ Ck ([0, lj ]) for each j ∈ J.The space Ck (Γ) is a Banach space with the norm

‖u‖Ck = maxβ≤k‖∂βu‖L∞

Elliptic PDEs on networks: the role of transitionconditions

Maximum Principle

Assume that the function w ∈ C2(Γ) satisfies

∂2w(x) ≥ 0, x ∈ Γ,S(w) :=

∑j∈Inci

aij∂jw(vi) ≥ 0, i ∈ IT

with {aij} is the incidence matrix. Then w cannot attain a maximum inthe interior of Γ, i.e. in Γ \ ∂Γ.

Remark:By ∂2w ≥ 0 for x ∈ Γ, we intend ∂2

j w(x) ≥ 0 for x ∈ ej , ∀j ∈ J.

Proof.: Assume first ∂2w > 0 and S(w) > 0 and there existsx0 ∈ Γ \ ∂Γ such that w attains a maximum at x0.

If x0 ∈ ej for some j ∈ J, then it follows that ∂jw(x0) = 0 and∂2

j w(x0) ≤ 0, a contradiction to ∂2w > 0.

If x0 = vi for some i ∈ IT , then aij∂jw(vi) ≤ 0 for all j ∈ Inci , henceSw(vi) ≤ 0, a contradiction to S(w) > 0.

For the general case, it can be proved that there exists ϕ ∈ C2(Γ) suchthat ∂2ϕ(x) > 0 for x ∈ ej , j ∈ J and Sϕ(vi) > 0 for i ∈ IT . Considerwδ = w + δϕ, δ > 0, then by the previous argument wδ cannot attain amaximum in Γ \ ∂Γ. The statement is obtained sending δ → 0.

It is clear that some sort of transition condition at the internalvertices is necessary for the validity of the maximum principle.The simplest form of a linear transition condition is the Kirchhoffcondition ∑

j∈Inci

aij∂jw(vi) = 0

from electrical circuits theory (the sum of the incoming currents isnull).More general transition conditions are of the form∑

j∈Inci

aijβij∂jw(vi) + γiw(vi) = 0 i ∈ IT

with βij > 0 for i ∈ I, j ∈ J, and also nonlocal conditions(depending on the value of w at the other vertices).

It is clear that some sort of transition condition at the internalvertices is necessary for the validity of the maximum principle.The simplest form of a linear transition condition is the Kirchhoffcondition ∑

j∈Inci

aij∂jw(vi) = 0

from electrical circuits theory (the sum of the incoming currents isnull).More general transition conditions are of the form∑

j∈Inci

aijβij∂jw(vi) + γiw(vi) = 0 i ∈ IT

with βij > 0 for i ∈ I, j ∈ J, and also nonlocal conditions(depending on the value of w at the other vertices).

A formal derivation of the MFG system

We formally deduce the FP equation and the corresponding transitionand boundary conditions using the fact that the FP equation is theadjoint of the linearized HJB equation.Consider the HJB equation

−ν∂2j u + H(x , ∂u) + λ = V [m] x ∈ ej , j ∈ J∑

j∈Inciaijνj∂ju(vi) = 0 i ∈ IT

∂ju(vi) = 0 i ∈ IB

and the linearized equation−ν∂2w + ∂pH(x , ∂u)∂w = 0 x ∈ ej , j ∈ J∑

j∈Inciaijνj∂jw(vi) = 0 i ∈ IT

∂jw(vi) = 0 i ∈ IB

Writing the weak formulation for a test function m, we get

0 =∑j∈J

(− νj∂

2w + ∂pHj(x , ∂u)∂w)m dx

=∑j∈J

(νj∂jw∂jm − ∂(m(x)∂pHj(x , ∂u))w

−∑i∈IT

∑j∈Inci

aij(νjmj(vi)∂jw(vi)−mj(vi)∂pHj(vi , ∂u)w(vi)

)−∑i∈IB

∑j∈Inci

)=∑j∈J

(− νj∂

2j m(x)− ∂(m(x)∂pHj(x , ∂u))

−∑i∈IT

∑j∈Inci

mj(vi) aijνj∂jw(vi)− aij(νj∂jm(vi) + ∂pH(vi , ∂u)mj(vi)

)w(vi)

−∑i∈IB

∑j∈Inci

aij(νj∂jm(vi) + mj(vi)∂pHj(x , ∂u)w(vi)

)Fabio Camilli (“Sapienza" Univ. di Roma) Mean field games on networks Bayreuth, settembre 2013 17 / 28

Writing the weak formulation for a test function m, we get

0 =∑j∈J

(− νj∂

2w + ∂pHj(x , ∂u)∂w)m dx

=∑j∈J

(νj∂jw∂jm − ∂(m(x)∂pHj(x , ∂u))w

−∑i∈IT

∑j∈Inci

)−∑i∈IB

∑j∈Inci

)=∑j∈J

(− νj∂

2j m(x)− ∂(m(x)∂pHj(x , ∂u))

−∑i∈IT

∑j∈Inci

mj(vi) aijνj∂jw(vi)− aij(νj∂jm(vi) + ∂pH(vi , ∂u)mj(vi)

)w(vi)

−∑i∈IB

∑j∈Inci

aij(νj∂jm(vi) + mj(vi)∂pHj(x , ∂u)w(vi)

)Fabio Camilli (“Sapienza" Univ. di Roma) Mean field games on networks Bayreuth, settembre 2013 17 / 28

By the integral terms we obtain the FP equation

ν∂2m + ∂(m ∂pH(x , ∂u)) = 0

Recalling the Kirchhoff condition satisfied by w , the first of three termscomputed at the transition nodes vanishes since by the continuity of m

mj(vi) = mk (vi), j , k ∈ Inci , i ∈ IT

The vanishing of the other two terms, i.e.∑j∈Inci

(νj∂jm(vi) + ∂pH(vi , ∂u)mj(vi)

)= 0 i ∈ IT

gives the transition condition at the vertices i ∈ IT , while the termcomputed at the boundary nodes gives the boundary condition

νj∂jm(vi) + m∂pHj(vi , ∂u) = 0 i ∈ IB, j ∈ Inci

Summarizing the MFG system on the network is

−ν∂2u + H(x , ∂u) + λ = V [m](x) x ∈ Γ

ν∂2m + div (∂pH(x , ∂u)m) = 0 x ∈ Γ∑j∈Inci

aij∂ju(vi) = 0 i ∈ IT∑j∈Inci

aij(νj∂jm(vi) + ∂pH(vi , ∂u)mj(vi)

)= 0 i ∈ IT

∂ju(vi) = 0, i ∈ IBνj∂jm(vi) + aijm∂pHj(vi , ∂u) = 0 i ∈ IB∫

Γ v(x)dx = 0,∫

Γ m(x)dx = 1, m ≥ 0

Remark:The condition∑

j∈Inci

aij(νj∂jm(vi) + ∂pH(vi , ∂u)mj(vi)) = 0, i ∈ IT

gives the conservation of the total flux of the density m at thetransition vertices. A similar condition is also considered in trafficflow problems on networks (Coclite-Garavello ’10).

Existence and uniqueness for theHamilton-Jacobi-Bellman equation

Consider the Hamilton-Jacobi-Bellman equation−ν∂2u + H(x , ∂u) + λu = f (x), x ∈ Γ∑j∈Inci

aijνj∂ju(vi) = 0 i ∈ IT

∂u(x) = 0 i ∈ IB

where• An Hamiltonian H : Γ× R→ R is a family {H j}j∈J withH j : [0, lj ]× R→ R continuous.• The viscosity coefficient ν and the discount factor λ may depend onthe arc ej , i.e. ν = νj and λ = λj for j ∈ J.

Remark: The Hamiltonian can be discontinuous across a vertex vi ,i ∈ IT .

TheoremAssume

H j ∈ C2([0, lj ]× R), H j(x , ·) convex in p for x ∈ [0, lj ] andH j(x ,p)| ≤ C(1 + |p|2) for x ∈ [0, lj ];ν = {νj}j∈J , λ = {λj}j∈J with νj , λj > 0 ∀j ∈ J;f ∈ C0,α(Γ)

Then there exists a unique smooth solution to (HJB). Moreoveru ∈ C2,α(Γ) for α ∈ (0,1) and

‖u‖C2,α ≤ C(‖f‖C0,α)

The control theoretic interpretation of the Kirchhoff condition

By the Dynamic Programming Principle the solution u of the (HJB)equation is the value function of an optimal control problem for acontrolled dynamics defined on Γ which on each edge ej satisfies thestochastic differential equation

dXs = αs ds +√

2ν dWs,

where αs = ∂pH(Xs, ∂u(Xs)) is the optimal control.The condition Kirchhoff condition implies that the process almostsurely spends zero time at each transition vertex vi ,(Freidlin-Wentzell, Ann.Probab. ’93)the term νj/(

∑j∈Inci

νj) is the probability that the process Xs entersin the edge ej when it is at the vertex vi .

An existence result for Fokker-Planck equation

ν∂2m + ∂(b(x) m) = 0 x ∈ Γ∑j∈Inci

aij [νj∂jm(vi) + b(x)mj(vi)] = 0 i ∈ IT

νj∂jm(vi) + b(x)mj(vi) = 0 vi ∈ IB, j ∈ Inci∫Γ m(x)dx = 1, m ≥ 0

where b = (bj)j∈J with bj : [0, lj ]→ R, bj ∈ C1([0, lj ]) for all j ∈ J.

PropositionThen there exists a unique smooth solution m to the FP equation.Moreover

‖m‖H1 ≤ C(‖b‖L∞).

An existence result for MFG system

TheoremAssume

H j ∈ C2([0, lj ]× R), H j(x , ·) convex in p for x ∈ [0, lj ] andH j(x ,p)| ≤ C(1 + |p|2) for x ∈ [0, lj ];ν = {νj}j∈J , λ = {λj}j∈J with νj , λj > 0 ∀j ∈ J;V [m(t , ·)](x) = F (m(t , x)) where F ∈ C1(R+)

Then there exists a classical solution (u,m) to the MFG system

−ν∂2u + H(x , ∂u) + λu = V [m](x) x ∈ Γν∂2m + div (∂pH(x , ∂u)m) = 0 x ∈ Γ∑

j∈Inciaij∂ju(vi) = 0 i ∈ IT∑

j∈Inciaij(νj∂jm(vi) + ∂pH(vi , ∂u)mj(vi)

)= 0 i ∈ IT

∂ju(vi) = 0, νj∂jm(vi) + aijm∂pHj(vi , ∂u) = 0 i ∈ IB∫Γ m(x)dx = 1, m ≥ 0

Proof:Set K = {µ ∈ C0,α(Γ) :

∫Γ µdx = 1} and define T : K → K according

to the schemeµ→ u → m

as follows.Given µ ∈ K, solve (HJB) with f (x) = V [µ](x) for the unknownu = uµ which is uniquely defined .Given u = uµ by the previous step solve (FP) withb(x) = ∂pH(x , ∂u) for the unknown m which is uniquely defined.

Set m = T (µ). By the estimate

‖u‖C2,α ≤ C(‖V [m]‖C0,α)

‖m‖H1 ≤ C‖∂pH(x , ∂u)‖L∞

uniformly in µ. It follows that the map T is continuous with compactimage. Hence by Schauder’s fixed point Theorem there exists a fixedpoint of T , i.e. a solution of the (MFG) system.

‖u‖C2,α ≤ C(‖V [m]‖C0,α)

‖m‖H1 ≤ C‖∂pH(x , ∂u)‖L∞

‖u‖C2,α ≤ C(‖V [m]‖C0,α)

‖m‖H1 ≤ C‖∂pH(x , ∂u)‖L∞

‖u‖C2,α ≤ C(‖V [m]‖C0,α)

‖m‖H1 ≤ C‖∂pH(x , ∂u)‖L∞

Comments and Remarks:The previous results for MFG system on network are very preliminary.There are several problems to study

Interpretation of the problem in terms of Nash equilibriaExistence result for the evolutive MFG systemThe planning problem on networks (an evolutive MFG system withinitial and terminal conditions for m)1st MFG systemsMore general transitions conditionsApproximation schemes for MFG systems on networks

Thank You!

Mean field games on networks -...

Documents

Transcript of Mean field games on networks -...