Repeated Games and Applications - Perfect Monitoring

Wouter Vergote (FUSL and CORE)

FUSL and CORE

December 2009

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 1 / 43

WW1: war in the trenches

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 2 / 43

WW1: war in the trenches

trench color

2.jpg

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 3 / 43

WW1: truce in the trenches

How to explain this? Robert Axelrod: "the evolution of cooperation"

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 4 / 43

Phases of the moon

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 5 / 43

Introduction: phases of the moon ?

Collusion through bid rigging

In one bid-rigging conspiracy rms General Electric and Westinghouse used the "phases of the moon" to take turns and determine which amongst them would submit the "low" bid to win the contracts.

Theoretical underpinning: McAfee and McMillan (AER 1992)

Is this e¢ cient if costs are private information and no transfers can be made?

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 6 / 43

Introduction: other areas

Why do countries wish to adhere to the rules of the WTO, even if there are no truly enforceable penalties from deviating?

Why do people voluntarily contribute to public goods?

Why do we observe informal insurance mechanisms in poor countries?

...

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 7 / 43

Canonical Stage Game

Players i : 1, ...,N Compact action sets Ai � Rk for some k,

ai 2 Ai a = (a1, ..., an) 2 A = ∏

i Ai

αi is mixed action for i , inducing a probability distribution over Ai :

αi (ai ) � 0 and ∑ ai2Ai

αi (ai ) = 1

set of mixed actions of player i : ∆(Ai ); ∆(A) = ∏i ∆(Ai ) is the set of mixed action proles

payo¤s are given by a continuous funtions u :

u : ∏i Ai ! Rn

how to extend u to mixed actions ?

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 8 / 43

Canonical Stage Game: Payo¤s

set of generated payo¤s: F = fv 2 Rn : 9a 2 A s.t. v = u(a)g set of feasible payo¤s: F 0=coF v belongs to the pareto frontier of F 0 if @v 0 2 F 0 s.t. v 0i > vi v 0 weakly dominates v if v 0i � vi for all i and v 0j > vj for some j v 2 coF is strongly e¢ cient if it is PO and not weakly dominated.

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 9 / 43

Canonical Stage Game: Assumptions and minmax

1 Ai is either nite or a continuum action space: a compact and convex subset of Rk for some k.

2 If Ai is a continuum action space, then u is continuous and ui is quasiconcave in ai .

player i 0s pure action minmax payo¤ is given by:

vpi � mina�i2A�i max ai2Ai

ui (ai , a�i )

is this payo¤ well dened? a (pure action) minmax prole for player i is bai = (baii ,bai�i ) How is it dened? is it unique?

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 10 / 43

Canonical Stage Game: Individual Rationality

v is weakly/strictly (pure action) individually rational if

vi � vpi /vi > v p i for all i .

Set of feasible SIR payo¤s:

F 0p = fv 2 coF : vi > vpi , i = 1, ..., ng

mixed action minmax payo¤ for player i :

v i � min α�i2∏j 6=i ∆(Aj )

max ai2Ai

ui (ai , α�i )

Set of feasible SIR payo¤s (relative to mixtures):

F � = fv 2 coF : vi > v i , i = 1, ..., ng

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 11 / 43

Canonical Stage Game: Public Correlation / Public Radomization

Why?

What? a probability space

Denition A public correlation device is a probability space ([0, 1] ,B,λ) , where is the Borel sigma-algebra and is the Lebesgue measure.

First a realization ω 2 [0, 1] is drawn, observed by all players before the choose actions: ai : [0, 1]! ∆(Ai ) expected payo¤: take expectations over ω 2 [0, 1] . (a1, ..., an) induces a joint distribution over ∏i ∆(Ai ) evaluation of deviation: ex post (after ω is drawn)

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 12 / 43

The Repeated Game

Time t 2 f0, 1, ...g nite or innite after each period all actions are observed and there is perfect recall.

Set of period t histories is: Ht � At with A0 � f∅g and At = ∏t�1s=0 A

Set of all possible histories: H � ∞S t=0 Ht

A pure strategy for player i : σi : H ! Ai . mixed strategy? behavior strategy for player i : σi : H ! ∆(Ai )

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 13 / 43

The Repeated Game: continuation game

for any ht 2 H the continuation game is the repeated game that begins in period t.

any strategy σ induces a continuation strategy σ jht where

σi jht (hτ) = σi (hthτ) for all hτ 2 Aτ

note that σi jht : H ! Ai ) subgame is strategically equivalent to original repeated game

Hence repeated games with perfect monitoring have a RECURSIVE structure

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 14 / 43

Outcome Path

An outcome path is an innite sequence of action proles a � (a0, a1, ...) 2 A∞. outcome path is di¤erent from a history

Denote at� (a0, a1, ..., at ) 2 At = history corresponding to a A pure strategy σ induces an outcome path (a0(σ), a1(σ), ...) as follows

a0(σ) � (σ1(∅); ..., σn(∅)) a1(σ) �

� σ1(a0(σ)); ..., σn(a0(σ))

� a2(σ) �

� σ1(a0(σ), a1(σ)); ..., σn(a0(σ), a1(σ))

� ...

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 15 / 43

Outcome Path

Behavioral strategy induce a path of play

σ(∅) = α0 2 ∏i ∆(Ai ) for each history a0 in the support of α0,

σ(a0) = α1 2 ∏i ∆(Ai ) ...

path of play at time t species a probability distribution over the histories at . the underlying behavioral strategy species for period t the mixed actions for each such history at , in turn inducing a probability distribution αt+1(at ) over period t + 1 action proles at+1, and over period t + 1 histories at+1

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 16 / 43

Payo¤s

A pure strategy σ induces an innite sequence of stage-game payo¤s (ui (a0(σ)), ui (a1(σ)), ui (a2(σ)), ...) 2 R∞

Payo¤s are discounted using a common discount factor δ 2 [0, 1) . If futi g is a sequence of stage game payo¤s, the average discounted utility is given by

(1� δ) ∞ ∑ t=0

δtuti

If Ai nite, the average discounted utility received from an action path is continuous with respect to the product topology.

Given a pure strategy prole σ we obtain

Ui (σ) = (1� δ) ∞ ∑ t=0

δtui (at (σ)).

Normalization ensures us that U(σ) 2 F 0

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 17 / 43

Nash Equilibrium and sequential rationality

Denition σ is a NE if 8i , 8σ0i :

Ui (σ) � Ui (σ0i , σ�i )

Lemma if σ is a NE then 8i

Ui (σ) � vpi (v i ) if σ is in pure (mixed) strategies

NE does not impose rational behavior out of equilibrium. We wish to rene NE by imposing sequential rationality: equilibrium behavior in every subgame.

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 18 / 43

Subgame perfect equilibrium

Denition σ is a SPE if 8ht 2 At σ jht is a NE of the repeated game

existence?

demanding concept: we need to check countably innite amount of histories and for every strategy there is an innite amount of deviations

we need to simplify 1 One shot deviation principle: limits the amount of alternative strategies to check

2 Automaton representation of strategies: allows us to organize subgames in equivalence classes

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 19 / 43

One shot deviation (OSD) principle

Denition

a OSD for player i from σi is a strategy bσi 6= σi such that 9! eht 2 At such that 8ht 6= eht : σi (ht ) = bσi (ht ).

example: one shot deviation from grim trigger

Denition

Fix σ�i . A OSD bσi from σi is PROFITABLE if, at eht s.t. σi (eht ) 6= bσi (eht ) : Ui (bσi jeht , σ�i jeht ) > Ui (σ jeht )

A NE can have protable one shot deviations (example see in class)

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 20 / 43

Subgame perfect equilibria and the one shot deviation principle

Lemma σ is a SPE i¤ @ protable OSDs

Proof. in pure strategies

What about NE? Does no protable OSDs on the equilibrium path imply a NE? No (see example)

Wouter Vergote (FUSL and CORE) Repeated Games December 2009 21 / 43

Automaton Representations of Strategy Proles

Denition

An automaton is a collection � W ,w0, f , τ

� where W is a set of states, w0

is the initial state, a decision function f : W !∏i