Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP...
Transcript of Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP...
![Page 1: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/1.jpg)
Characterization andComputation of Correlated
Equilibria in Polynomial Games
Noah Stein
Joint work with Profs. Pablo Parrilo and
Asuman Ozdaglar
![Page 2: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/2.jpg)
Outline
• Introduction
• Characterizing correlated equilibria in finite games
• Characterizing correlated equilibria in continuousgames [new]
• An overview of SDP / SOS methods
• Computing correlated equilibria in polynomialgames with SDP relaxations [new]
MIT Laboratory for Information and Decision Systems 1
![Page 3: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/3.jpg)
Game theoretic setting
• Standard strategic (normal) form game
• Players (rational agents) numbered i = 1, . . . , n
• Each has a set Ci of strategies si
• Players choose their strategies simultaneously
• Rationality: Each player seeks to maximize his ownutility function ui : C → R, which represents all hispreferences over outcomes
MIT Laboratory for Information and Decision Systems 2
![Page 4: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/4.jpg)
Goals
• Much is known theoretically about correlatedequilibria: existence (under some topologicalassumptions), relation to Nash equilibria, etc.
• Efficient algorithms are known for computing thesewhen the strategy sets are finite
• Much less on games with infinite strategy sets
• Our goal is to find classes of infinite games forwhich correlated equilibria can be computed, as wellas associated algorithms
MIT Laboratory for Information and Decision Systems 3
![Page 5: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/5.jpg)
Previous work
• Definition of correlated equilibrium [Aumann 1974]
• Rationality argument for playing correlatedequilibria [Aumann 1987]
• Elementary existence proof [Hart & Schmeidler1989]
• Efficient algorithms for computing correlatedequilibria of finite games [Papadimitriou 2005]
• Sum of squares techniques [Parrilo 2000, . . .]
• Algorithm for computing minimax strategies ofpolynomial games [Parrilo 2006]
MIT Laboratory for Information and Decision Systems 4
![Page 6: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/6.jpg)
Chicken and correlated equilibria(u1, u2) Wimpy Macho
Wimpy (4, 4) (1, 5)
Macho (5, 1) (0, 0)
• Nash equilibria (self-enforcing independent distrib.)
– (M,W ) yields utilities (5, 1); (W,M) yields (1, 5)
–(
12W + 1
2M, 12W + 1
2M)
yields expected utility(2 1
2 , 212 )
• Correlated equilibria (self-enforcing joint distrib.)
– e.g. 12 (W,M) + 1
2 (M,W ) yields (312 , 3
12 )
– 13 (W,W ) + 1
3 (W,M) + 13 (M,W ) yields (3 2
3 , 323 )
MIT Laboratory for Information and Decision Systems 5
![Page 7: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/7.jpg)
Correlated equilibria in gameswith finite strategy sets
• ui(ti, s−i)− ui(s) is change in player i’s utility whenstrategy ti replaces si in s = (s1, . . . , sn)
• A probability distribution π is a correlatedequilibrium if∑
s∈{ri}×C−i
[ui(ti, s−i)− ui(s)]π(s|si = ri) ≤ 0
for all players i and all strategies ri, ti ∈ Ci
• No player has an incentive to deviate from hisrecommended strategy ri
MIT Laboratory for Information and Decision Systems 6
![Page 8: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/8.jpg)
LP characterization
• A probability distribution π is a correlatedequilibrium if and only if∑
{s−i∈C−i}
[ui(ti, s−i)− ui(s)]π(s) ≤ 0
for all players i and all strategies si, ti ∈ Ci
• Proof: Use definition of conditional probability, pulldenominator out of sum, and cancel it (reversible).
• Set of correlated equilibria of a finite game is apolytope
MIT Laboratory for Information and Decision Systems 7
![Page 9: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/9.jpg)
Departure functioncharacterization
• A probability distribution π is a correlatedequilibrium if and only if∑
s∈C[ui(ζi(si), s−i)− ui(s)]π(s) ≤ 0
for all players i and all departure functionsζi : Ci → Ci
• Proof: Let ti = ζi(si) on previous slide and sumover si ∈ Ci. Conversely, define ζi(ri) = ri for allri 6= si and ζi(si) = ti, then cancel terms.
• Interpretation as Nash equilibria of extended game
MIT Laboratory for Information and Decision Systems 8
![Page 10: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/10.jpg)
Continuous games
• Finitely many players (still)
• Strategy spaces Ci are compact metric spaces
• Utility functions ui : C → R are continuous
• Examples:
– Finite games: |Ci| <∞, ui arbitrary
– Polynomial games: Ci = [−1, 1], ui polynomial
– . . .
• Main property of continuous games: Correlated(and Nash) equilibria always exist
MIT Laboratory for Information and Decision Systems 9
![Page 11: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/11.jpg)
Defining correlated equilibria incontinuous games
• Definition in literature: The probability measure πis a correlated equilibrium if∫
[ui(ζi(si), s−i)− ui(s)]dπ ≤ 0
for all i and all (measurable) departure functions ζi
• Equivalent to above def. if strategy sets are finite
• Quantifier ranging over large set of functions
• Unknown function ζi inside ui
• Is there a characterization without these problems?
MIT Laboratory for Information and Decision Systems 10
![Page 12: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/12.jpg)
An instructive failed attempt
• The following “characterization” fails:∫{si}×C−i
[ui(ti, s−i)− ui(s)]π(s) ≤ 0
for all players i and all strategies si, ti ∈ Si
• Holds for any continuous probability distribution π
• This condition is much weaker than correlatedequilibrium
MIT Laboratory for Information and Decision Systems 11
![Page 13: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/13.jpg)
Simple departure functions
• A probability measure π is a correlated equilibriumif and only if∫
[ui(ζi(si), s−i)− ui(s)]dπ ≤ 0
for all i and all simple (having finite range)measurable departure functions ζi
• Proof: Approximate any ζi as a pointwise limit ofsimple functions ξki (possible because Ci is compactmetric). Then ui(ξki (si), s−i)→ ui(ζi(si), s−i)pointwise by continuity of ui and the result followsby Lebesgue’s dominated convergence theorem.
MIT Laboratory for Information and Decision Systems 12
![Page 14: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/14.jpg)
No departure functions
• A probability measure π is a correlated equilibriumif and only if
µi,ti(Bi) :=∫Bi×C−i
[ui(ti, s−i)− ui(s)]dπ ≤ 0
for all i, ti ∈ Ci, and measurable subsets Bi ⊆ Ci• Equivalently, −µi,ti is a positive measure for all i, ti
• Proof: The above integral is the corresponds to thedeparture function given by ζi(ri) = ri for ri 6∈ Biand ζi(ri) = ti for ri ∈ Bi. Conversely, the integralfor any simple departure function is a finite sum ofterms of this type for some values of Bi and ti.
MIT Laboratory for Information and Decision Systems 13
![Page 15: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/15.jpg)
Integration against test functions
• A probability measure π is a correlated equilibriumif and only if∫
fi(si)dµi,ti =∫fi(si)[ui(ti, s−i)− ui(s)]dπ ≤ 0
for all i, ti ∈ Ci, and fi : Ci → [0,∞) in somesufficiently rich class of test functions, e.g.
– Measurable characteristic functions
– Measurable functions
– Continuous functions
– Squares of polynomials (if Ci ⊂ Rk)
MIT Laboratory for Information and Decision Systems 14
![Page 16: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/16.jpg)
Semidefinite programming
• A semidefinite program (SDP) is anoptimization problem of the form
min L(S) ← L is a given linear functional
s.t. T (S) = v ← T is a given linear transformation,
v is a given vector
S � 0 ← S is a symmetric matrix
of decision variables
• SDPs generalize linear programs and can be solvedefficiently using interior point methods
MIT Laboratory for Information and Decision Systems 15
![Page 17: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/17.jpg)
Sums of squares + SDP
• A polynomial p(x) is ≥ 0 for all x ∈ R iff it is a sumof squares of polynomials qk (SOS)
p(x) =∑
q2k(x) for all x ∈ R
• A polynomial p(x) is ≥ 0 for all x ∈ [−1, 1] iff thereare SOS polynomials s(x), t(x) such that
p(x) = s(x) + (1− x2)t(x)
• Coefficients of SOS polynomials can be described inan SDP
MIT Laboratory for Information and Decision Systems 16
![Page 18: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/18.jpg)
Multivariate SOS + SDP
• A polynomial p(x1, . . . , xk) is ≥ 0 for all x ∈ Rk iffthere is an integer r(p) ≥ 0 such that(
x21 + . . .+ x2
k
)rp(x1, . . . , xk)
is a sum of squares of polynomials ql(x1, . . . , xk)
• Sequence of sufficient SDP conditions characterizingmultivariate nonnegative polynomials exactly “inthe limit”
• Similar conditions for nonnegativity on [−1, 1]k, . . .
• Such conditions are generally not exact for a fixed r
independent of p, with some important exceptions
MIT Laboratory for Information and Decision Systems 17
![Page 19: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/19.jpg)
Moments of measures + SDP
• If τ is a measure on [−1, 1], then for a polynomial p:∫p2(x)dτ(x) ≥ 0 and
∫(1− x2)p2(x)dτ(x) ≥ 0
• τ0, . . . , τ2m are the moments of a measure τ on[−1, 1] (i.e. τk =
∫xkdτ(x)) iff
τ0 τ1 τ2
τ1 τ2 τ3
τ2 τ3 τ4
� 0,
τ0 − τ2 τ1 − τ3τ1 − τ3 τ2 − τ4
� 0 (m = 2 case)
• Moments of measures on [−1, 1] can be described inan SDP
MIT Laboratory for Information and Decision Systems 18
![Page 20: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/20.jpg)
Moments of multivariate measures
• If τ is a measure on [−1, 1]k, then for a polynomialp(x1, . . . , xk):∫
p2(x)dτ(x) ≥ 0 and∫
(1− x2i )p
2(x)dτ(x) ≥ 0
• Requiring these conditions for all p up to a fixeddegree gives a necessary semidefinite condition thejoint moments of a measure must satisfy.
• Exact “in the limit”
MIT Laboratory for Information and Decision Systems 19
![Page 21: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/21.jpg)
Polynomial games
• Strategy space is Ci = [−1, 1] for all players i
• Utilities ui are multivariate polynomials
• Finitely supported equilibria always exist, withexplicit bounds on support size [1950s; SOP 2006]
• Minimax strategies and values can be computed bysemidefinite programming [Parrilo 2006]
MIT Laboratory for Information and Decision Systems 20
![Page 22: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/22.jpg)
Computing corr. equil. in poly.games: Naive attempt (LP)
• Intended as a benchmark to judge other techniques
• Ignore polynomial structure
• Restrict strategy choices (and deviations) to fixedfinite sets C̃i ⊂ Ci
• Compute exact correlated equilibria of approximategame
• This is a sequence of LPs which converges (slowly!)to the set of correlated equilibria as thediscretization gets finer.
MIT Laboratory for Information and Decision Systems 21
![Page 23: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/23.jpg)
Computing corr. equil. in poly.games by SDP relaxation
• Describe moments of measures π on [−1, 1]n with asequence of necessary SDP conditions which becomesufficient in the limit
• For fixed d, we want to use SDP to express∫p2(si)[ui(ti, s−i)− ui(s)]dπ ≤ 0
for all i, ti ∈ [−1, 1], and polys. p of degree < d
• Shown above: for each d this is a necessarycondition for π to be a correlated equilibrium whichbecomes sufficient as d→∞
MIT Laboratory for Information and Decision Systems 22
![Page 24: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/24.jpg)
Computing corr. equil. in poly.games by SDP relaxation (II)
• Define Sdi ∈ Rd×d[si] by[Sdi
]jk
= sj+ki , 0 ≤ j, k < d
• A polynomial is a square of a polynomial if andonly if it can be written as c>Sdi c for some c ∈ Rd
• Let Mdi (ti) =
∫Sdi [ui(ti, s−i)− ui(s)]dπ ∈ Rd×d[ti]
• Then we wish to constrain π to satisfyc>Md
i (ti)c ≤ 0 for all c ∈ Rd and ti ∈ [−1, 1], i.e.Mdi (ti) � 0 for ti ∈ [−1, 1]
MIT Laboratory for Information and Decision Systems 23
![Page 25: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/25.jpg)
Computing corr. equil. in poly.games by SDP relaxation (III)
• Mdi (ti) is a matrix whose entries are univariate
polynomials in ti with coefficients which are affinein the decision variables of the problem (the jointmoments of π)
• This can be expressed exactly as an SDP constraintfor any d (this is one of those special cases in whichthe condition is exact, even though there aremultiple variables, ti and c)
• Putting it all together we get a nested sequence ofSDPs converging to the set of correlated equilibria
MIT Laboratory for Information and Decision Systems 24
![Page 26: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/26.jpg)
0.5 1 1.5 2 2.5 3
−2.6
−2.4
−2.2
−2
−1.8
−1.6
−1.4
−1.2
−1
−0.8
−0.6
ux
u yComparison of Static Discretization, Adaptive Discretization, and SDP Relaxation
ux(x,y) = 0.554 + 1.36x − 1.2y + 0.596x2 + 2.072xy −0.394y2
uy(x,y) = −1.886 − 1.232x + 0.842y − 0.108x2 + 1.918xy − 1.044y2
SDP Relaxation
Static Discretization
Adaptive Discretization
MIT Laboratory for Information and Decision Systems 25
![Page 27: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/27.jpg)
Conclusions
• New characterizations of correlated equilibria ininfinite games
• First algorithms for computing correlated equilibriain any class of infinite games
Acknowledgements
• My advisors: Profs. Asuman Ozdaglar and PabloParrilo
• Prof. Muhamet Yildiz
Time for a better discretizationalgorithm?
MIT Laboratory for Information and Decision Systems 26
![Page 28: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/28.jpg)
Approximate correlated equilibria
• The probability measure π is an ε-correlatedequilibrium if∫
[ui(ζi(si), s−i)− ui(s)]dπ ≤ ε
for all i and all (measurable) departure functions ζi
• Same as definition of correlated equilibrium if ε = 0
• We will be interested in the case when the supportof π is a finite set
MIT Laboratory for Information and Decision Systems 27
![Page 29: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/29.jpg)
Characterizing finitely supportedε-correlated equilibria
• A probability measure π with finite supportcontained in C̃ =
∏C̃i is an ε-correlated
equilibrium if and only if∑{s−i∈C−i}
[ui(ti, s−i)− ui(s)]π(s) ≤ εi,si
for all i, si ∈ C̃i, and ti ∈ Ci and∑si∈C̃i
εi,si≤ ε
for all i.
MIT Laboratory for Information and Decision Systems 28
![Page 30: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/30.jpg)
Applying SOS / SDP
• Given a polynomial game and a finite support setC̃i ⊂ [−1, 1] for each player, the condition that π bea probability measure on C̃ and an ε-correlatedequilibrium can be written in an SDP
• First constraint says a univariate polynomial in tiwith coefficients linear in the π(s) and εi,si
is ≥ 0on [−1, 1], hence is expressible exactly in an SDP
• Second constraint is linear, so usable in SDP
• Conditions to make π a prob. measure also linear
• Always feasible if ε can vary
MIT Laboratory for Information and Decision Systems 29
![Page 31: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/31.jpg)
Adaptive discretization
• Given C̃ki , optimize the following (as an SDP)
min ε
s.t. π is an ε-correlated equilibrium
which is a correlated equilibrium
when deviations are restricted to C̃k
• Let εk and πk be an optimal solution
• If εk = 0 then halt
• Otherwise, compute C̃k+1i as described on next slide
and repeat
MIT Laboratory for Information and Decision Systems 30
![Page 32: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/32.jpg)
Adaptive discretization (II)
• Steps to compute C̃k+1
– For some player i, the ε-correlated equilibriumconstraints are tight
– Find values of ti making these tight (free withSDP duality), add these into C̃ki to get C̃k+1
i
– For j 6= i, let C̃k+1j = C̃kj
• By construction C̃k+1i ) C̃ki because if not then
ε = 0, a contradiction
MIT Laboratory for Information and Decision Systems 31
![Page 33: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/33.jpg)
Proof that εk → 0
• If not, there is a subseq. along which εkl ≥ ε > 0.
• There is some player i for whom C̃kl+1i ) C̃kl
i forinfinitely many l.
• Assume WLOG that C̃kl+1i ) C̃kl
i for all l.
• Cover Ci with finitely many open balls Bji such that|ui(si, s−i)− ui(ti, s−i)| ≤ ε
2 for all s−i ∈ C−iwhenever si and ti are in the same Bji .
• For some l, all the balls Bji which will ever containa point of some C̃kl
i already do.
MIT Laboratory for Information and Decision Systems 32
![Page 34: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/34.jpg)
Proof that εk → 0 (II)
• Let ti,si∈ C̃kl+1
i make the εkl -correlatedequilibrium constraints tight.
• Let ri,si ∈ C̃kli be in the same Bji as ti,si
for each si.
• Then we get a contradiction:
ε ≤∑s∈C̃kl
[ui(ti,si , s−i)− ui(s)]πkl(s)
−∑s∈C̃kl
[ui(ri,si , s−i)− ui(s)]πkl(s)
=∑s∈C̃kl
[ui(ti,si, s−i)− ui(ri,si
, s−i)]πkl(s) ≤∑s∈C̃kl
ε
2πkl(s) =
ε
2
MIT Laboratory for Information and Decision Systems 33
![Page 35: Characterization and Computation of Correlated Equilibria ...nstein/documents/RQE.pdf · LP characterization A probability distribution ˇ is a correlated equilibrium if and only](https://reader034.fdocuments.net/reader034/viewer/2022050610/5fb1431ce875fa4f5654583b/html5/thumbnails/35.jpg)
Adaptive discretization (III)
• We did not use the polynomial structure of the ui inthe convergence proof, just continuity
• Used polynomiality to convert the optimizationproblem into an SDP
• Can also do this conversion if the ui are rational oreven piecewise rational (and continuous)
• Solutions of such games are surprisingly complex –the Cantor measure arises as the unique Nashequilibrium of a game with rational ui [Gross 1952]
• Now we have a way to approximate these!
MIT Laboratory for Information and Decision Systems 34