Deviations and Fluctuations for Mean-Field Games

Kavita Ramanan,Brown University

AMS Short CourseJMM, Denver, Colorado

January 14, 2020

Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 1 / 45

Outline

I. Interacting Diffusions: Fluctuations

II. From Interacting Diffusions to MFG: Fluctuations

III. Large Deviations for Interacting Diffusions and MFG

IV. Open Problems

I. Interacting Diffusions:Fluctuations

Interacting Diffusions and the McKean-Vlasov limit

Consider n diffusions interacting through their empirical measure:

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

where {X n,i0 = ξi }i∈N are iid with common law µ0 and (W i)n

i=1 are iidd-dimensional Brownian motions.

Under suitable regularity conditions, X n,i ⇒ X and µnt → µt , where X is

a non-linear Markov (or McKean-Vlasov) process:

dXt = b(Xt , µt)dt + dBt , µt = Law(Xt),

with X0 ∼ µ0 independent of B, a d-dimensional Brownian motion.Alternatively, µ solves the (nonlinear) Fokker-Planck equation

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

with X0 ∼ µ0 independent of B, a d-dimensional Brownian motion.

Alternatively, µ solves the (nonlinear) Fokker-Planck equation

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

with X0 ∼ µ0 independent of B, a d-dimensional Brownian motion.Alternatively, µ solves the (nonlinear) Fokker-Planck equation

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Rate of Convergence to the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)

1 How can one measure distance between µnt and µt , µn and µ?

2 Some notation:Define Cd = C([0,T ] : Rd ), equipped with the uniform topologyGiven a separable Banach space (E , ‖ · ‖), let P(E) denote thespace of Borel probability measures on ENote that µn

t , µt ∈ P(R) and µn, µ ∈ P(Cd ),Given p ∈ [1,∞), let Pp(E) = {ν ∈ P :

∫E ‖x‖

pν(dx) <∞} equippedwith the p-Wasserstein metric:

Wp,E(ν, ν′) = inf

(∫E×E

||x − y ||pπ(dx ,dy))1/p

where the infimum is over all probability measures π on E × E withfirst and second marginals µ and ν respectively.

dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)

∫E ‖x‖

(∫E×E

||x − y ||pπ(dx ,dy))1/p

dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)

2 Some notation:Define Cd = C([0,T ] : Rd ), equipped with the uniform topology

Given a separable Banach space (E , ‖ · ‖), let P(E) denote thespace of Borel probability measures on ENote that µn

∫E ‖x‖

(∫E×E

||x − y ||pπ(dx ,dy))1/p

dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)

t , µt ∈ P(R) and µn, µ ∈ P(Cd ),

Given p ∈ [1,∞), let Pp(E) = {ν ∈ P :∫

E ‖x‖pν(dx) <∞} equipped

with the p-Wasserstein metric:

(∫E×E

||x − y ||pπ(dx ,dy))1/p

dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)

∫E ‖x‖

(∫E×E

||x − y ||pπ(dx ,dy))1/p

Fluctuations around the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

andddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Recall µ = (µt)t≥0 is the McKean-Vlasov limit.We are interested in the limit of the signed measures capturingfluctuations:

Snt :=

√n(µn

t − µt), t ≥ 0.

Do you expect this sequence to converge to another signed measure?

A Simple ExampleHow bad can the weak limit of a signed measure be?

Let ν ∈ P(R) be a probability measure with a cdf that is notdifferentiable at any point:

F (x) := ν((−∞, x ]), x ∈ R.

For n ∈ N, define νn to be a measure with cdfFn(x) := F (x + n−1/2):

νn((−∞, x ]) = ν((−∞, x + n−1/2]) = F (x + n−1/2),

and define the signed measure

νn :=√

n[νn − ν].

Then, since F is non-differentiable,

limn→∞ νn((−∞, x ]) = lim

n→∞√

x +1√n

)− F (x)

]doesn’t exist!

A Simple Example (contd.)For any measure ν and integrable function ϕ, define

〈ϕ,ν〉 =∫ϕ(x)ν(dx)

Recall that νn((−∞, x ]) = ν((−∞, x + n−1/2]), and so

〈ϕ,νn〉 =∫ϕ(x)νn(dx) =

(x −

)ν(dx) =

(·− 1

⟩Recall that νn :=

√n[νn − ν].

Thus, for any infinitely differentiable function ϕ : R 7→ R withcompact support,

So limn→∞〈νn, ϕ〉 = lim

n→∞√

n[〈νn, ϕ〉− 〈ν,ϕ〉]

= limn→∞

√n⟨ν,

(·− 1√

)−ϕ(·)

]⟩= −〈ν,ϕ′〉.

(x −

)ν(dx) =

(·− 1

Recall that νn :=√

n[νn − ν].

n→∞√

n[〈νn, ϕ〉− 〈ν,ϕ〉]

= limn→∞

√n⟨ν,

(·− 1√

)−ϕ(·)

]⟩= −〈ν,ϕ′〉.

(x −

)ν(dx) =

(·− 1

⟩Recall that νn :=

√n[νn − ν].

n→∞√

n[〈νn, ϕ〉− 〈ν,ϕ〉]

= limn→∞

√n⟨ν,

(·− 1√

)−ϕ(·)

]⟩= −〈ν,ϕ′〉.

A Primer on the Theory of Distributions

ν ↔ ϕ 7→ 〈ν,ϕ〉Let D be a space of test functions, e.g.,

D = {f ∈ C∞ : supp f is compact }

equipped with a suitable topology: ϕn → ϕ in D if there exists acompact set K such that each ϕn and ϕ have support in K and∂αϕn →∂αϕ uniformly on K for all α.

Definition. A mapping ν : D 7→ R is said to be a linear functional if

ν(α1ϕ1 + α2ϕ2) = α1ν(ϕ1) + α2ν(ϕ2), ∀αi ∈ R, ϕi ∈ D, i = 1,2.

Let D′ denote the corresponding space of distributions, defined tobe a linear functional ν : D 7→ R, that also satisfies the followingcontinuity property:

ϕn → ϕ in D ⇒ ν(ϕn) → ν(ϕ) in R, ∀ϕ ∈ D.

ϕn → ϕ in D ⇒ ν(ϕn) → ν(ϕ) in R, ∀ϕ ∈ D.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 9 / 45

A Primer on the Theory of Distributions (contd.)

Examples of Distributions

1 The space of distributions is clearly a vector space on R

2 Given a σ-finite measure ν on R, the linear functionalϕ 7→ 〈ν,ϕ〉 = ∫

ϕ(x)ν(dx) defines a distributionExercise: Prove this by verifying the continuity propertyNote: For general ν ∈ D′, ν(ϕ) is often written as 〈ν,ϕ〉.

3 Given a distribution ν ∈ D′, and a C∞ function ψ, ψν denotes thedistribution

(ψν)(ϕ) = 〈ν,ψϕ〉, ∀ϕ ∈ D

This is clearly a linear functional.Exercise: Verify the continuity property to show ψν is adistribution.

1 The space of distributions is clearly a vector space on R2 Given a σ-finite measure ν on R, the linear functionalϕ 7→ 〈ν,ϕ〉 = ∫

(ψν)(ϕ) = 〈ν,ψϕ〉, ∀ϕ ∈ D

1 The space of distributions is clearly a vector space on R2 Given a σ-finite measure ν on R, the linear functionalϕ 7→ 〈ν,ϕ〉 = ∫

(ψν)(ϕ) = 〈ν,ψϕ〉, ∀ϕ ∈ D

Exercise: Which of the following maps w : D(R) 7→ R are distributions:here, ϕ(k) is the k th derivative of ϕ

1 〈w , ϕ〉 :=∫R f (x)ϕ(x)dx for a locally integrable function f .

2 〈w , ϕ〉 :=∑∞

k=0ϕ(k).3 〈w , ϕ〉 :=

∑∞k=0ϕ

(k)(√

2),4 〈w , ϕ〉 :=

∫Rϕ

2(x)dx

5 〈w , ϕ〉 :=∫∞

0ϕ(x)−ϕ(−x)

Exercise: Show that for any k ∈ N, 〈w , ϕ〉 := (−1)k 〈w , ϕ(k)〉 is adistribution.

Convergence of Distributions

Let (ν`) be a sequence in D′ and ν ∈ D′.1 Then (ν`) is said to converge to ν in D′, denoted ν` → ν if

lim`→∞〈ν`, ϕ〉 = 〈ν,ϕ〉.

2 Moreover, (ν`) is Cauchy in D′ if ∀ϕ ∈ D, (〈ν`, ϕ〉) is Cauchy in R.

Returning to the Example:

νn((−∞, x ]) = ν((−∞, x + n−1/2]), νn :=√

n[νn − ν].

We showed that for all ϕ ∈ D,

limn→∞〈νn, ϕ〉 := −〈ν,ϕ′〉,

The linear functional ϕ 7→ −〈ν,ϕ′〉 lies in D′ (by the last exercise).It is in fact denoted by ∂ν and is said to be the derivative of ν.Thus, we showed that

νn → ∂ν in D′.

Convergence of Distributions

Let (ν`) be a sequence in D′ and ν ∈ D′.1 Then (ν`) is said to converge to ν in D′, denoted ν` → ν if

lim`→∞〈ν`, ϕ〉 = 〈ν,ϕ〉.

2 Moreover, (ν`) is Cauchy in D′ if ∀ϕ ∈ D, (〈ν`, ϕ〉) is Cauchy in R.

Returning to the Example:

νn((−∞, x ]) = ν((−∞, x + n−1/2]), νn :=√

n[νn − ν].

We showed that for all ϕ ∈ D,

limn→∞〈νn, ϕ〉 := −〈ν,ϕ′〉,

The linear functional ϕ 7→ −〈ν,ϕ′〉 lies in D′ (by the last exercise).It is in fact denoted by ∂ν and is said to be the derivative of ν.Thus, we showed that

νn → ∂ν in D′.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 12 / 45

A Stochastic Example

Let (Bk )k∈N be a sequence of independent 1-dimensional BMswith initial distribution π0.For t > 0, A ∈ B(R), define

Nnt (A) :=

n∑k=1

I{Bkt ∈A}.

Exercise: Calculate limn→∞ Nnt (A).

Soln. Since (I{Bkt ∈A})k∈N are iid Bernoulli random variables, the

SLLN tells us that Nnt (A) → γt(A) almost surely, where γt is a

centered Gaussian distribution with variance t , because

γt(A) = E[I{B1

t ∈A}

]= P(B1

t ∈ A) = P(Bkt ∈ A).

Note: In fact, one can prove convergence in P(Rd): a.s.,

limn→∞ Nn

t (·) → γt

Nnt (A) :=

n∑k=1

I{Bkt ∈A}.

γt(A) = E[I{B1

t ∈A}

]= P(B1

t ∈ A) = P(Bkt ∈ A).

limn→∞ Nn

t (·) → γt

Nnt (A) :=

n∑k=1

I{Bkt ∈A}.

γt(A) = E[I{B1

t ∈A}

]= P(B1

t ∈ A) = P(Bkt ∈ A).

limn→∞ Nn

t (·) → γt

Stochastic Example (contd.)

Nnt (A) =

n∑k=1

I{Bkt ∈A}, E[Nn

t (·)] = γt , A ∈ B(R), t ≥ 0.

Exercise*: Find the limit of the (random) signed measure-valued proc.:

Snt (A) :=

√n (Nn

t (A) − γt(A)) , t ≥ 0,A ∈ B(R).

Soln. View Sn as a distribution-valued process: for t > 0, ϕ ∈ D,

Snt (ϕ) :=

∫Rϕ(x)Sn

t (dx) =√

Rϕ(x)Nn

t (dx) −∫

Rϕ(x)γt(dx)

is a random variable, in fact it is equal to

Snt (ϕ) = n−1/2

n∑k=1

(ϕ(Bk

t ) − 〈ϕ,γt〉)= n−1/2

n∑k=1

(ϕ(Bk

t ) − E[ϕ(Bkt )])

• Can show ∀ϕ ∈ D, t 7→ Snt (ϕ) is a.s. continuous and that, t 7→ Sn

t (·)is a continuous D′-valued process.

Nnt (A) =

n∑k=1

I{Bkt ∈A}, E[Nn

t (·)] = γt , A ∈ B(R), t ≥ 0.

Exercise*: Find the limit of the (random) signed measure-valued proc.:

Snt (A) :=

√n (Nn

t (A) − γt(A)) , t ≥ 0,A ∈ B(R).

Soln. View Sn as a distribution-valued process: for t > 0, ϕ ∈ D,

Snt (ϕ) :=

∫Rϕ(x)Sn

t (dx) =√

Rϕ(x)Nn

t (dx) −∫

Rϕ(x)γt(dx)

is a random variable, in fact it is equal to

Snt (ϕ) = n−1/2

n∑k=1

(ϕ(Bk

t ) − 〈ϕ,γt〉)= n−1/2

n∑k=1

(ϕ(Bk

t ) − E[ϕ(Bkt )])

• Can show ∀ϕ ∈ D, t 7→ Snt (ϕ) is a.s. continuous and that, t 7→ Sn

t (·)is a continuous D′-valued process.

Snt (A) :=

√n (Nn

t (·) − γt(·)) .

Use the multidimensional CLT to show there exists a centeredGaussian D′-valued process (St(ϕ))t ,ϕ such that every finite-dim.distribution of Sn

t = (Snt (ϕ))t ,ϕ converges to the corresponding

finite-dim. distribution of the solution S = (St(ϕ))t ,ϕ to the S(P)DE

dSt = (∂ ◦√πt)dbt +

12∂2Stdt ,

dSt(ϕ) = (∂ ◦√πt)dbt(ϕ) +

12∂2St(ϕ)dt , ϕ ∈ D,

where{bt } = {bt(ϕ), ϕ ∈ H2} is a standard Wiener H′2-valued process.πt = π0 ∗ γt√πt ∈ C∞(R) is viewed as a multiplication operator in D′

∂ is differentiation in D′

∂ ◦ √πt denotes a composition of these operators.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 15 / 45

Recap so far

We started with interacting diffusions:

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

and recalled the McKean-Vlasov limit µ that satisfies:

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

To capture rate of convergence, we wanted to understand the limit of

Snt :=

√n(µn

t − µt), t ≥ 0.

Recap so far (contd.)

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Snt :=

√n(µn

t − µt), t ≥ 0.

To understand the form of potential limits of processes such as Snt , we

1. considered sequences of scaled centered deterministic signedmeasures of a similar form, and showed that their limits are oftendistributions, not (signed) measures;

2. provided a brief introduction to the theory of distributions;

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Snt :=

√n(µn

t − µt), t ≥ 0.

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Snt :=

√n(µn

t − µt), t ≥ 0.

Back to: Fluctuations around the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Snt :=

√n(µn

t − µt), t ≥ 0.

3. considered the simplest stochastic case, namely to study the limitof fluctuations (or CLT – central limit theorems) for non-interactingdiffusions, that is, where b ≡ 0, and characterized the limit as asolution to distribution-valued process, governed by an “SPDE”.

4. We now consider the interacting case, b 6= 0 and, in analogy withthe non-interacting case, will view Sn as a suitabledistribution-valued process.

Back to: Fluctuations around the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

Snt :=

√n(µn

t − µt), t ≥ 0.

3. considered the simplest stochastic case, namely to study the limitof fluctuations (or CLT – central limit theorems) for non-interactingdiffusions, that is, where b ≡ 0, and characterized the limit as asolution to distribution-valued process, governed by an “SPDE”.

4. We now consider the interacting case, b 6= 0 and, in analogy withthe non-interacting case, will view Sn as a suitabledistribution-valued process.

Some CLT Results for Interacting Particle Systems

References assuming affine dependence of drift on the empiricalmeasure

1 H. Tanaka and M. Hitsuda. Central limit theorem for a simplediffusion model of interacting particles. Hiroshima MathematicalJournal 11 (1981), no. 2, 415–423.

2 A.S. Sznitman. A fluctuation result for nonlinear diffusions.Infinite-dimensional analysis and Stochastic Processes (1985),145–160.

3 S. Méléard. Asymptotic behaviour of some interacting particlesystems: McKean-Vlasov and Boltzman models, Probabilisticmodels for nonlinear partial differential equations, Lecture Notesin Math, vol. 1627, Springer, 1996, pp. 42–95.

Some CLT Results for Interacting Particle Systems

References that consider more general dependence on the empiricalmeasure

1 T.G. Kurtz and J. Xiong. A stochastic evolution equation arisingfrom fluctuations of a class of interacting particle systems,Communications in Mathematical Sciences 2 (2004) no. 3,325–358.Comment: Only in the case where each particle takes values in R– one-dimensional case

2 F. Delarue, D. Lacker and K.R., “From the master equation tomean field game limit theory: a central limit theorem”, Electron. J.Probab., Volume 24 (2019), paper no. 51, 54 pp.Comment 1: This covers more general dependence and particlestaking values in Rd for general d ∈ N.Comment 2: The precise space in which the limit process liesends up depending on the dimension d .

General CLT for Interacting Particle Systems

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

and, with µ the McKean-Vlasov limit,

Snt :=

√n(µn

t − µt), t ≥ 0.

Theorem: Under suitable regularity assumptions on b, Sn convergesweakly to S = (St(ϕ))t ,ϕ in H ′d , where H ′d is a suitable distributionspace with test function space Hd , where S solves the SPDE:

d〈St , ϕ〉 = 〈St ,At ,µtϕ〉dt + dWt(ϕ), ϕ ∈ Hd

where W is a centered H′d -valued continuous centered Gaussianprocess with covariance functional

E[Wt(ϕ1)Ws(ϕ2)] =

∫ s∧t

0〈µr ,Dxϕ1 · Dxϕ2〉dr , ϕ1, ϕ2 ∈ Hd ,

and where At ,µt is some suitable (nonlocal) operator.

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

Snt :=

√n(µn

t − µt), t ≥ 0.

E[Wt(ϕ1)Ws(ϕ2)] =

∫ s∧t

0〈µr ,Dxϕ1 · Dxϕ2〉dr , ϕ1, ϕ2 ∈ Hd ,

and where At ,µt is some suitable (nonlocal) operator.

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt

Snt :=

√n(µn

t − µt), t ≥ 0.

E[Wt(ϕ1)Ws(ϕ2)] =

∫ s∧t

0〈µr ,Dxϕ1 · Dxϕ2〉dr , ϕ1, ϕ2 ∈ Hd ,

and where At ,µt is some suitable (nonlocal) operator.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 21 / 45

Interacting Particle Systems

2. From Interacting Diffusionsto MFG:

Fluctuations

Motivation and Context

Multi-agent or Many-player Systems

Symmetric n-player Differential Games

W 1, . . . ,W n ind. d-dim BMs

Polish action space

drift functional b : Rd × Pp(Rd)× 7→ Rd

State Dynamics

dX n,it = b(X n,i

t , µnt , α

i(t ,X t))dt + dW it , µn

n∑k=1

δX n,kt

where αi : [0,T ]× (Rd)n → is a Markovian control that is chosen tominimize the i th objective function

Jni (α

1, . . . , αn) = E

0f (X i

t , µnt , α

i(t ,X t))dt + g(X iT ,m

for suitable cost functionals f and g.

Nash Equilibria

Definition A (closed-loop) Nash equilibrium is defined in the usual wayas a vector of feedback functions or controls (α1, . . . , αn), whereαi : [0,T ]× (Rd)n → are such that the SDE

dX n,it = b(X n,i

t , µnt , α

i(t ,X t))dt + dW it , µn

n∑k=1

δX n,kt

is unique in law, and

Jn,i(α1, . . . , αn) ≤ Jn,i(α1, . . . , αi−1, α, αi+1, . . . , αn),

for any alternative choice of feedback control α.

Characterization of n-player Nash equilibriaMain Point (Cardialaguet et al ’15)

A verification theorem tells us that if we have a unique solution{vn,i }i=1,...,n to a coupled system of n PDEs called the Nash systemsuch that vn,i lies in C1,2 for each i = 1, . . . ,n, then the controls

(0,T ]× (Rd)n 3 (t ,x) 7→ αn,i(

x ,mnX ,Dxi v

n,i(t ,x))

form a closed-loop Nash equilibrium, where mnx = 1

i=1 δxi

The corresponding Nash equilibrium dynamics, given by

dX n,it = b(X n,i

t , µnt ,Dxi v

n,i(t ,X nt ))dt + dW i

t , µnt =

n∑k=1

δX n,kt

defines a collection of interacting diffusions, with

b(x ,m, y) = b(x ,m, α(x ,m, y)),

being the Nash equilibrium drift, where α takes the explicit form:

α(x ,m, y) ∈ arg mina∈A

[b(x ,m,a) · y + f (x ,m,a)] .

(0,T ]× (Rd)n 3 (t ,x) 7→ αn,i(

x ,mnX ,Dxi v

n,i(t ,x))

i=1 δxi

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

defines a collection of interacting diffusions,

b(x ,m, y) = b(x ,m, α(x ,m, y)),

[b(x ,m,a) · y + f (x ,m,a)] .

(0,T ]× (Rd)n 3 (t ,x) 7→ αn,i(

x ,mnX ,Dxi v

n,i(t ,x))

i=1 δxi

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

defines a collection of interacting diffusions, with

b(x ,m, y) = b(x ,m, α(x ,m, y)),

[b(x ,m,a) · y + f (x ,m,a)] .

Nash Equilibrium n-player dynamics

Recall that the corresponding Nash equilibrium dynamics has the form

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

In other words, it is a system of weakly interacting diffusions:

dX it = bn(t ,X i

t , µnt )dt + σdBi

t , i = 1, . . . ,n,

wherebn(t , x ,m) = b(x ,m,Dxi v

n,i(t , x))

But the drift is n-dependent, so this is not of the form we consideredearlier. Instead, replace the n-dependent control vn,i by a quantitycoming from the master equation.

Nash Equilibrium n-player dynamics

Recall that the corresponding Nash equilibrium dynamics has the form

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

In other words, it is a system of weakly interacting diffusions:

dX it = bn(t ,X i

t , µnt )dt + σdBi

t , i = 1, . . . ,n,

wherebn(t , x ,m) = b(x ,m,Dxi v

n,i(t , x))

But the drift is n-dependent, so this is not of the form we consideredearlier. Instead, replace the n-dependent control vn,i by a quantitycoming from the master equation.

An Approximating System

Recall: Interacting diffusions describing Nash equilibrium dynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

Instead: Consider the modified system coming from the limit system:“Replace” vn,i by un,i , where

un,i(t , x1, . . . , xn) = U(t , xi ,mnx), mn

n∑k=1

δxi .

the dependence of un,i on n is only through the empiricalmeasureThat is, consider the sequence of IPS:

dX n,it = b(t , X n,i

t ,mnX t) + dW i

t , i = 1, . . . ,n,

whereb(t , x ,m) = b(x ,m,DxU(t , x ,m))

Overall Philosophy:Transferring LLN/CLT Results

Interacting diffusions describing Nash equilibrium dynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

Approximating diffusions in the form of an IPS

t , µnt ) + dW i

t , µnt =

n∑k=1

δX n,kt

b(t , x ,m) = b(x ,m,DxU(t , x ,m))

1 Analyze master equation + Nash PDE to proveE[W2,Cd (µn, µn)] = O(n−2)

2 Invoke IPS results to deduce LLN/CLT for {µn}.3 Use estimate in 1. to deduce LLN/CLT for {µn}.

Overall Philosophy:Transferring LLN/CLT Results

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

Approximating diffusions in the form of an IPS

t , µnt ) + dW i

t , µnt =

n∑k=1

δX n,kt

b(t , x ,m) = b(x ,m,DxU(t , x ,m))

1 Analyze master equation + Nash PDE to proveE[W2,Cd (µn, µn)] = O(n−2)

2 Invoke IPS results to deduce LLN/CLT for {µn}.3 Use estimate in 1. to deduce LLN/CLT for {µn}.

Interacting Particle Systems

3. Large Deviations forInteracting Particle Systems

and MFG

Large Deviations Theory

Large deviations (LD) is an asymptotic theory that characterizesthe asymptotic probability of rare events

When a sequence of probabilities of events decay to 0, largedeviations characterizes the asymptotic exponential decay rate:suppose νn, n ∈ N take values in (some topological space) E , andsatisfies for all “nice” A ⊂ S,

P(νn ∈ A) ∼ e−snI(A),

where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) IThus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.

Large deviations (LD) is an asymptotic theory that characterizesthe asymptotic probability of rare eventsWhen a sequence of probabilities of events decay to 0, largedeviations characterizes the asymptotic exponential decay rate:suppose νn, n ∈ N take values in (some topological space) E , andsatisfies for all “nice” A ⊂ S,

where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).

One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) IThus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.

where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) I

Thus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.

where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) IThus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.

A Rigorous Statement of the Large Deviation Principle

• E - topological space• {νn} - sequence of E-valued random elements

Definition (Large Deviation Principle (LDP))

{νn} is said to satisfy a large deviations principle (LDP) with speed {sn}

and rate function I : R 7→ [0,∞) if for all measurable A,

− infw∈A◦

I(w)≤ lim infn→∞ 1

s(n)logP(νn ∈ A)

≤ lim supn→∞

logP(νn ∈ A) ≤ − infw∈A

where I is lower semicontinuous and has compact level sets.

In short, the LDP says that for all “nice” sets A ⊂ E ,

P(νn ∈ A) ≈ e−sn infw∈A I(w)

The Contraction Principle

Theorem. Let Y and Y ′ be topological spaces and let F : Y 7→ Y ′ bea continuous mapping. Suppose a sequence {Yn} of Y-valued randomvariables satisfies an LDP with rate function I : Y 7→ [0,∞]. Then thesequence {Y ′n := F (Yn)} satisfies an LDP with rate functionJ : Y ′ 7→ [0,∞], given by

J(y ′) = inf{I(y) : F (y) = y ′ for some y ∈ Y}.

Exercise 3: Prove the contraction principle.

Theory of Large Deviations (Sanov’s Theorem)Suppose Yi , i = 1, . . . , are iid on some Polish space Y with commondistribution θ ∈ P(Y), and define

νn :=1n

n∑i=1

Also, define relative entropy: given ν, µ ∈ P(Y),

H(ν|θ) :=

dνdθ

(x))ν(dx).

if ν� θ and H(ν|θ) = ∞ otherwise.

Sanov’s Theorem

Then {νn} satisfies an LDP in P1(Y) with good rate function H(·|θ).

Exercise 4: Prove Sanov’s theorem when Yi take values in a finitestate space.

Theory of Large Deviations (Sanov’s Theorem)Suppose Yi , i = 1, . . . , are iid on some Polish space Y with commondistribution θ ∈ P(Y), and define

νn :=1n

n∑i=1

Also, define relative entropy: given ν, µ ∈ P(Y),

H(ν|θ) :=

dνdθ

(x))ν(dx).

if ν� θ and H(ν|θ) = ∞ otherwise.

Sanov’s Theorem

Then {νn} satisfies an LDP in P1(Y) with good rate function H(·|θ).

Exercise 4: Prove Sanov’s theorem when Yi take values in a finitestate space.

Large Deviations in the non-interacting Case (b = 0)

X n,it = X n,i

0 + W it , Qn

n∑k=1

δX n,k0 ,W i ,

{X i0}i∈N iid with common distribution µ0; {W i }i∈N iid Brownian motions.

Theorem: As an immediate consequence of Sanov’s theorem wehave {Qn} satisfies an LDP on P1(Rd × Cd

0) with rate function

R(ν|µ0 ×W),

where recall µ0 is the initial distribution of X i0 and W is d-dimensional

Wiener measure on Cd0

References (LDPs for Interacting Particle Systems)

dX n,it = b(t ,X n,i

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

{X i0}i∈N iid with common distribution µ0; {W i }i∈N iid Brownian motions

D. Dawson and J. Gärtner, “Large deviations from theMcKean-Vlasov limit for weakly interacting diffusions”,Stochastics: An International Journal of Probability and StochasticProcesses 20 (1987), 247-308.

A. Budhiraja, P. Dupuis, and M. Fischer, “Large deviationproperties of weakly interacting processes via weak convergencemethods”, Annals of Probability (2012), 74-102.

M.Fischer, “On the form of the large deviation rate function for theempirical measures of weakly interacting systems”, Bernoulli 20(2014), 1765-1801.

References (LDPs for Interacting Particle Systems)

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

{X i0}i∈N iid with common distribution µ0; {W i }i∈N iid Brownian motions

For the ultimate application to MFG, need to consider an extensionbeyond those references that includes

random initial conditions,

time-dependent drift

and a weaker continuity condition on the drift b, namely,continuous as a map from [0,T ]× Rd × P1(Rd) to Rd ,in particular, b need not be continuous in the third variable withrespect to the weak topology

And also allows for common noise, which we do not include here.

Large Deviations in the Interacting Case

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

(X n,i0 )i∈N iid with common distribution µ0.

Idea: To use the contraction principleNeed to express the law of X n as a continuous functional of the law{Qn} of the non-interacting particle system and a Brownian motion

Key Result: Girsanov’s TheoremRelates the law µ of the solution X to the SDE

dXt = dWt

with the law of µb of the solution X b to the SDE with an adapted(suitably regular) drift (rt)t≥0

dX bt = rtdt + Wt

t , µnt )dt + dW i

t , µnt =

n∑k=1

δX n,kt,

(X n,i0 )i∈N iid with common distribution µ0.

Idea: To use the contraction principleNeed to express the law of X n as a continuous functional of the law{Qn} of the non-interacting particle system and a Brownian motion

Key Result: Girsanov’s TheoremRelates the law µ of the solution X to the SDE

dXt = dWt

with the law of µb of the solution X b to the SDE with an adapted(suitably regular) drift (rt)t≥0

dX bt = rtdt + Wt

Large Deviations in the Interacting CaseExercise 5. Prove the LDP for {µn

t } via the following steps:

1. Canonical setup: Define the mappings

e : (y × f ) ∈ Rd × Cd0 7→ y

wt : (y × f ) ∈ Rd × Cd0 7→ ft , t ≥ 0.

2. For Q ∈ P1(Rd × Cd0), define the McKean-Vlasov equation map:

xt = e +

0b(s, xs,Q ◦ x−1

s )ds + wt , (1)

where e and wt denote the canonical maps on Rd × Cd0 , as

defined above. Here,• Q represents the joint law of the initial condition e and drivingnoise w• Q ◦ x−1

s ∈ P1(Rd) represents the marginal law at time s of thesolution x to equation (1), under Q.

e : (y × f ) ∈ Rd × Cd0 7→ y

wt : (y × f ) ∈ Rd × Cd0 7→ ft , t ≥ 0.

xt = e +

0b(s, xs,Q ◦ x−1

s )ds + wt , (1)

defined above.

Here,• Q represents the joint law of the initial condition e and drivingnoise w• Q ◦ x−1

e : (y × f ) ∈ Rd × Cd0 7→ y

wt : (y × f ) ∈ Rd × Cd0 7→ ft , t ≥ 0.

xt = e +

0b(s, xs,Q ◦ x−1

s )ds + wt , (1)

defined above. Here,• Q represents the joint law of the initial condition e and drivingnoise w• Q ◦ x−1

Recall: X n,i0 iid, and

t , µnt )dt+dW i

t , µnt =

n∑k=1

δX n,kt, Qn =

n∑k=1

δ(X n,k0 ,W k )

and for Q ∈ P1(Rd × Cd0), the McKean-Vlasov equation map is:

xt = e +

0b(s, xs,Q ◦ x−1

s )ds + wt ,

where e and wt denote canonical variables on Rd × Cd0 .

3. Let Φ : P1(Rd × Cd0) 7→ C([0,T ] : P1(Rd)) be the mapping that

takes Q to the flow of marginal measures (Q ◦ x−1t )t≥0, and

observeµn = Φ(Qn).

4. Prove Φ : P1(Rd × Cd0 ) 7→ C([0,T ] : P1(Rd)) is uniformly

continuous.

t , µnt )dt+dW i

t , µnt =

n∑k=1

δX n,kt, Qn =

n∑k=1

δ(X n,k0 ,W k )

xt = e +

0b(s, xs,Q ◦ x−1

s )ds + wt ,

continuous.

t , µnt )dt+dW i

t , µnt =

n∑k=1

δX n,kt, Qn =

n∑k=1

δ(X n,k0 ,W k )

xt = e +

0b(s, xs,Q ◦ x−1

s )ds + wt ,

continuous.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 40 / 45

Large Deviations in the Interacting Case: Summary

Given the IPS

t , µnt )dt+dW i

t , µnt =

n∑k=1

δX n,kt, Qn =

n∑k=1

δ(X n,k0 ,W k )

We have shown• {Qn} satisfies an LDP with rate function R(Q|µ0 ×W)

• µn = Φ(Qn)

• Φ : P1(Rd × Cd0 ) 7→ C([0,T ] : P1(Rd)) is uniformly continuous.

5. Apply the contraction principle to conclude that if b is boundedand continuous and Lipschitz continuous in the second and thirdarguments (uniformly in time), then {µn} satisfies an LDP with ratefunction

J(ν) = inf{R(Q|µ0 ×W) : Φ(Q) = ν}.

This concludes Exercise 5 and the proof of the LDP for IPS.

Given the IPS

t , µnt )dt+dW i

t , µnt =

n∑k=1

δX n,kt, Qn =

n∑k=1

δ(X n,k0 ,W k )

• µn = Φ(Qn)

J(ν) = inf{R(Q|µ0 ×W) : Φ(Q) = ν}.

Given the IPS

t , µnt )dt+dW i

t , µnt =

n∑k=1

δX n,kt, Qn =

n∑k=1

δ(X n,k0 ,W k )

• µn = Φ(Qn)

J(ν) = inf{R(Q|µ0 ×W) : Φ(Q) = ν}.

Large Deviations for Mean-Field Games

Recall the form of interacting diffusions describing Nash equilibriumdynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt,

Aim:To prove an LDP for the sequence (µn)n∈N of empirical measures of

the sequence of Nash equilibria state processes

Principle: Transfer LDP results from IPS to MFG

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

In view of the previous results on LDPs for IPS, recall theapproximating diffusions we considered earlier that were in the form ofan IPS:

t , µnt ) + dW i

t , µnt =

n∑k=1

δX n,kt

b(t , x ,m) = b(x ,m,DxU(t , x ,m))

1 Originally, had only E[W2,Cd (µn, µn)] = O(n−2)

2 Invoke IPS results to get LDP for {µn}.3 Use a sharper exponential estimate in 1. to deduce LDP for {µn}.

Principle: Transfer LDP results from IPS to MFG

dX n,it = b(X n,i

t , µnt ,Dxi v

t , µnt =

n∑k=1

δX n,kt

In view of the previous results on LDPs for IPS, recall theapproximating diffusions we considered earlier that were in the form ofan IPS:

t , µnt ) + dW i

t , µnt =

n∑k=1

δX n,kt

b(t , x ,m) = b(x ,m,DxU(t , x ,m))

1 Originally, had only E[W2,Cd (µn, µn)] = O(n−2)

2 Invoke IPS results to get LDP for {µn}.3 Use a sharper exponential estimate in 1. to deduce LDP for {µn}.

Additional References on LDP

Lacker and Ramanan, “Rare Nash equilibria and the price ofanarchy in large static games,” (2019) Mathematics of OperationsResearch 44 (2019) no. 2, 400-422.

Cardaliaguet, Delarue, Lasry and Lions, “The master equationand the convergence problem in mean-field games” (2019)

F. Delarue, D. Lacker and K. Ramanan, “From the master equationto mean field game limit theory: Large deviations andconcentration of measure," (2018) to appear in Annals ofProbability

IV. Open Problems

Study refined convergence theorems for open-loop Nashequilibria.

Investigate if there are cases when the MFG LDP exists, butdiffers from the interacting paricle system LDP obtained from themaster equation.

Establish large deviation principles for stochastic differentialgames in the presence of non-uniqueness (as has been done inthe static case)

Use LDPs to find interesting conditional limit laws in both the staticand stochastic differential settings.

Deviations and Fluctuations for Mean-Field Games · Deviations and Fluctuations for Mean-Field...

Transcript of Deviations and Fluctuations for Mean-Field Games · Deviations and Fluctuations for Mean-Field...

Deviations and Fluctuations for Mean-Field Games · Deviations and Fluctuations for Mean-Field...

Documents

Transcript of Deviations and Fluctuations for Mean-Field Games · Deviations and Fluctuations for Mean-Field...

Expansions for statistics involving the mean absolute ... · mean and the sample median. Edgeworth expansions for some scale invariant statistics involving the mean absolute deviations

Annual Mean Air Temperature and Precipitation …...Table 3: R2, standard deviations from protocols of regression & quadratic mean (RMS) The fit of the MT model is excellent (R 2 between

Event-by-Event Fluctuations of the Mean Transverse Momentum at SPS Energy

Major Points Formal Tests of Mean Differences Review of Concepts: Means, Standard Deviations, Standard Errors, Type I errors New Concepts: One and Two.

Employability among Statistics Graduates: Graduates ...downloads.hindawi.com/journals/edri/2019/7285491.pdfe mean age of graduates was 26.19 years with 1.85 standard deviations. 73.4%

Altitude Deviations

Converting to a Standard Normal Distribution Think of me as the measure of the distance from the mean, measured in standard deviations.

Scanned by CamScanner · 2019-12-11 · mean and 2 standard deviations above the mean. .9/5 ) c. Determine the percent of newboms that weioh between I standard deviation above the

npcil.etenders.in · Dimensional deviations may be according to "mean DIN 7168". ... Proposed chemical composition of fine grain silver, Vickers hardness (H VI) for

Minimizing Noise and Bias in 3D DIC - Correlated Solutions · Overview of Noise and Bias Noise: random, zero-mean deviations from the correct result Bias: systematic deviations from

)JOEBXJ1VCMJTIJOH$PSQPSBUJPO …downloads.hindawi.com/journals/jfp/2016/6041671.pdf · scale. e mean diameters of beads were calculated and presented with standard deviations (Table

Classical latent variable models for medical research · Classical latent variable models for medical research ... deviations of subjects’ true scores from the population mean ...

ocular Deviations

Tables-04222005... · STANDARD NORMAL DISTRIBUTION Area or probability Entries in the table give the area under the curve between the mean and z stan- dard deviations above the mean.

Pilot Deviations

Goals Mean p T fluctuations Net-charge fluctuations Conclusions

International Standards Supporting Nanotechnology … 2578-07 (2012) Standard Practice for Calculation of Mean Diameters and Standard Deviations of Particle Size Distributions ASTM

Masters Thesis (v6)€¦ · List of Tables 1. Description of stimulus videos. 2. Mean scores and standard deviations of negative emotions. 3. Mean scores, standard deviations, t-scores

Statistical properties of nuclei: beyond the mean field Yoram Alhassid (Yale University) Introduction Beyond the mean field: correlations via fluctuations.

Abstract - Missouri State University€¦ · Web viewMissouri State University Baccalaureate Graduate Mean Scores and Standard Deviations and Comparative Means and Standard Deviations