A Projection Framework for Near-Potential Polynomial Games

Nikolai Matni (nmatni@caltech.edu)

Control and Dynamical Systems, California Institute of Technology

IEEE CDC Maui, December 13th 2012

Motivation – Potential Games

• Informal definition: local actions have predictable global consequences.

• Nice properties– Pure-strategy Nash Equilibria (NE)– Simple dynamics converge to these NE

• Applications to distributed control– Marden, Arslan & Shamma 2010– Candogan, Menache, Ozdaglar

& Parrilo 2009– Li & Marden, 2011

Motivation – Polynomial Games• Would like to consider general class of continuous games

– Finite players, continuous action sets.

• Why?– Goal is control: most systems of interest are analog.– Quantization leads to tradeoffs in granularity, performance and problem

dimension.

• Why not?– Potentially intractable to analyze (Parrilo 2006, Stein et al. 2006 for recent results).– Can lead to infinite dimensional optimization problems.

• Solution?– Restrict ourselves to polynomial cost functions and use Sum Of Squares (SOS)

methods.

Motivation – Near Potential Games

• O. Candogan, A. Ozdalgar, P.A. Parrilo, A Projection Framework for Near-Potential Games, CDC 2010 (and subsequent work)

• Basic idea: if a game is “close” to being a potential game, it behaves “almost as well.”

• Projection Framework – finite dimensional case– Potential games form a subspace.– Project onto this framework to find closest potential game.– If distance from subspace is small, original game inherits many nice

properties.

• Goal: Extend these ideas to polynomial games.

Outline• Motivation

– Potential games– Polynomial games– Near-Potential games

• Preliminaries– Game Theory– Algebraic Geometry/Sum of Squares (SOS)

• Projection Framework

• Properties– Static– Dynamic

• Example

• Conclusions and Future work

Outline• Motivation

– Potential games– Polynomial games– Near-Potential games

• Preliminaries– Game Theory– Algebraic Geometry/Sum of Squares (SOS)

• Projection Framework

• Properties– Static– Dynamic

• Example

• Conclusions and Future work

Prelims – Polynomial Game

• A polynomial game is given by: – A finite player set– Strategy spaces , where – Polynomial utility functions

,

• A polynomial game is:– Continuous if for all n, is a closed interval of the real

line – Discrete if for all n, – Mixed if some strategy sets are continuous, and some are

discrete.– Assume w.l.o.g.

Prelims – Potential Games

• A polynomial game G is a polynomial potential game if there exists a polynomial potential function such that, for every player n, and every

• Algebraic characterization (Monderer, Shapley ’96): A continuous game is a potential game iff

Prelims – Misc. Game Theory

• A strategy is an approximate Nash (or ε) Equilibrium if, for all n, we have that

Prelims – SOS and p(x)≥0

• Definition: a real polynomial p(x) admits a Sum Of Squares (SOS) decomposition if

• Why SOS?– Determining if p(x)≥0, is in general, NP-hard– Determining if p(x) is SOS tested through SDP

• Lemma [SOS relaxation]:If there exist SOS polynomials such that

then

Outline• Motivation

– Potential games– Polynomial games– Near-Potential games

• Preliminaries– Game Theory– Algebraic Geometry/Sum of Squares (SOS)

• Projection Framework

• Properties– Static– Dynamic

• Example

• Conclusions and Future work

Projection Framework – MPD & MDD

• Need a notion of distance in the space of games• Candogan et al. introduced Maximum Pairwise Distance

(MPD)

• Use the continuity of polynomials to define Maximum Differential Difference (MDD)

• Both capture how different two games are in terms of utility improvements due to unilateral deviations

Projection Framework

• Task: Given a polynomial game , find a nearby potential polynomial game

• Formulate as an optimization problem:

• Constraint ensures we get a Potential Game• Objective function minimizes MDD.• Intractable!

Projection Framework – Convexify

• Step 1: rewrite constraint in terms of algebraic characterization

• Step 2: introduce slack variable γ

Projection Framework – Convexify

• Step 3: apply Lemma [SOS relaxation]

• This is a finite dimensional SOS program, solvable in polynomial time. It yields a polynomial potential game satisfying

Projection Framework - Extensions

• Can extend this idea to mixed/discrete games

• Lemma [MPD]: If , then

• Continuous Relaxations: For a mixed or discrete game, set all strategy sets to [-1,1]– Apply previous SOS program and Lemma [MPD] to mixed

games or discrete games with– Allows us to apply algebraic characterization, which can

reduce number of constraints from O( ) to O(N)

Outline• Motivation

– Potential games– Polynomial games– Near-Potential games

• Preliminaries– Game Theory– Algebraic Geometry/Sum of Squares (SOS)

• Projection Framework

• Properties– Static– Dynamic

• Example

• Conclusions and Future work

Properties – Static

• Let and be such that . Then for every ε1-equilibrium y of ,

z(y) is an ε-equilibrium of , where

• For continuous games, D=0, z(y)=y, and local maxima of P are pure (ε=0) NE.

Properties – Static

• Let and be such that . Then for every ε1-equilibrium y of ,

z(y) is an ε-equilibrium of , where

• For continuous games, D=0, z(y)=y, and local maxima of P are pure (ε=0) NE.

Properties – Dynamic

• Definition: ε-better response dynamics– Round robin updates– Player updates only to improve utility by at least ε– Otherwise does not update

• Suppose there exists such that Then, under ε-better response dynamics, after a finite number of iterations, dynamics will be confined to the ε-equilibria set of , for

arbitrary.

Outline• Motivation

– Potential games– Polynomial games– Near-Potential games

• Preliminaries– Game Theory– Algebraic Geometry/Sum of Squares (SOS)

• Projection Framework

• Properties– Static– Dynamic

• Example

• Conclusions and Future work

Example – Distributed Power

• Consider the N player game defined by– – –

• Distributed power minimization interpretation

Example – Distributed Power

• Run through projection framework to find nearby potential game :

satisfying

Example – Distributed Power

• Potential function concave – can compute global maximum to identify .2-equilibria of G

• Alternatively, can run .2-better response dynamics to converge to a .2-equilibria of G.

• Quantify performance through cost function

Example – Distributed Power• Compare better-response (xbr) to

centralized (optimal x*) positions

• Better response comes within ~20% of centralized solution

• Completely decentralized

• Arbitrarily scalable

• Requires no a prioriknowledge of basestation locations

Conclusions & Future Work

• Introduce framework for analyzing polynomial games– Defined MDD and a tractable projection framework to find

nearby potential games– Related static and dynamic properties of polynomial games to

those of nearby potential games– Illustrated these methods on a distributed power problem

• Future work– Projecting onto weighted polynomial games– Additional static properties (mixed-equilibria)– Efficiency notions (price of anarchy, price of stability, etc.)