1

Chapter 13

Evolutionary Games

What happens when players no longer ‘choose’ their strategies but play pre-programmed actions?

Adopted with permission from course by Simon David Halliday

2

Introduction

• The basic idea behind evolutionary game theory (EGT) is that we can alter how we understand games so that we can do away with the assumption of rationality.

• We look at the types of games we have consistently investigated (PD, Chicken) in order to understand evolutionary dynamics and ‘hardwiring’ of strategies.

• We will also talk about several of the dynamics of EGT and interesting conclusions that can be drawn from them.

3

Framework

• A complex of one or more genes constitutes a genotype.

• The genotype governs specific physical outcomes called the phenotype.

• The success of one phenotype relative to another in a specific environment indicates its fitness.

• The process of selection therefore is the dynamic

4

Framework

• Chance mutations can change the proportions of the phenotypes in a given environment.

• If their fitness is higher than those present in the population, these mutants will invade the population.

• In general populations that express a specific phenotype will become more prevalent in society if they are fitter than other populations.

5

Framework

• A population is evolutionary stable if the

population cannot be successfully invaded by

any mutants.

• Remember that some populations may be more

successful than others in a given context, but

others may be successful in another context.

– Thus we need to be aware of how and why specific

phenotypes are successful.

• Many visible outcomes in biology can be seen to

be the result of the evolutionary process.

6

Framework

• Using evolutionary theory we can relax the assumptions of rationality!

• No longer are agents rationally bound by their preferences, but they are genetically predisposed to act in a specific manner (the phenotypic expression of their genotype).

• Individuals are ‘born’ to play one strategy.

• For humans this embeddedness may be a result not only of genetic programming, but also a result of cultural and social factors.

7

Framework

• From a population with a heterogeneous distribution of strategies, individuals are randomly paired and selected to interact.

• Those that come out with better payoffs on aggregate are said to have better fitness.

• Those with higher fitness are more able to find a compatible sexual mate and more likely to be able to reproduce.

• Thus fitter ‘strategies’ will become more prevalent in a population over time.

8

Framework

• In socioeconomic games (quite evidently different to biological interactions) the idea of ‘reproduction’ is somewhat different.

• Others observe the fitter strategies and adopt these strategies, or at least do their best to imitate them.

• In this way their reproduction is somewhat like Richard Dawkins conceptualisation of a ‘meme’ (rather than a ‘gene’)

• (Remember also that conscious experimentation with new strategies could substitute for mutation)

9

Framework

• Dawkins comments,

“Just as genes propagate themselves in the gene pool by leaping from body to body via sperm or eggs, so memes propagate themselves in the meme pool by leaping from brain to brain via a process which, in the broad sense, can be called imitation.”

Richard Dawkins, The Selfish Gene, 1986.

10

Framework

• There are two kinds of configurations in biological games:

1) Monomorphism: one phenotype has taken over the population, it was fitter than the others and dominates. The unique prevailing strategy is the evolutionary stable strategy.

2) Polymorphism: there are multiple phenotypes in a population, all reinforcing the presence of the others.

11

Framework

• What then is our idea of equilibrium?

– A configuration of strategies that is stable in the

population is our first idea of equilibrium.

– This mirrors the idea of an evolutionary stable

strategy configuration (ESS).

– A strategy is evolutionary stable if it has higher

average payoffs than other strategies.

– And if it is resistant to invasion by mutants.

– These are testable conditions and hence allow us to

come up with a stable idea of equilibrium.

12

Prisoner’s Dilemma

• We assume two types of players:

Cooperators and Defectors.

• Cooperators always cooperate.

• Defectors always defect.

• The payoff table represents the payoffs

that a given type receives when it meets

other players.

13

Column

20

(Defect)

26

(Cooperate)

Row

20

(Defect) (288, 288) (360, 216)

26

(Cooperate) (216, 360) (324, 324)

Figure 13.1. Prisoner’s Dilemma

14

Prisoner’s Dilemma

• The population consists only of cooperators and defectors

• Let x be the proportion of cooperators in the population.

• Hence the probability that any cooperator will meet another cooperator is approximately x.

• The probability that any cooperator will meet a defector is (1-x).

• Thus the payoff for a cooperator is:

π(C) = 324(x) + 216(1-x)

15

Prisoner’s Dilemma

• The probability that any defector will meet a cooperator is x.

• The probability that any defector will meet another defector is (1-x).

• Thus the average payoff for a defector is:

π(D) = 360(x) + 288(1-x)

• But: π(D) > π(C) always

• Why?

360(x) + 288(1-x) > 324(x) + 216(1-x) for all x

• Thus defectors will come to dominate the population! (in terms of fitness)

16

Prisoner’s Dilemma

• Is it possible for a mutant cooperator to invade a population of all defectors?

• Sadly not, even for a very small value of x, the cooperators will always be less fit than the defectors – they cannot invade!

• A 100% Defector population is an ESS.

• Thus we have a general result: if a game has a strictly dominant strategy in the rational behaviour analysis of the game, then this strategy should constitute an ESS.

17

The Repeated PD

• Now assume that we repeat the PD like we did in Chapter 11.

• We look at 2, 3 and n repetitions and see that if there is a phenotype for the Tit-for-Tat strategy then this phenotype could dominate the population depending on the initial population distribution.

• We assume two strategy phenotypes: – Defect Always (A)

– Tit-for-Tat (T) (Will cooperate, until opponent defects, then will defect until opponent cooperates)

18

Column

A T

Row

A (576, 576) (648, 504)

T (504, 648 ) (648, 648)

Figure 13.2. Twice Repeated Prisoner’s Dilemma

19

Column

A T

Row

A (864, 864) (936, 792)

T (792, 936) (972, 972)

Figure 13.3. Thrice Repeated Prisoner’s Dilemma

20

Repeated PD

• So, let’s assume that the players have to

interact 3 times whenever they get

matched with another player

(approximating our thrice-repeated PD).

• If there are x T types in the population, the

chance that any T will meet another T is x.

• The chance they will meet an A is (1-x).

π(T) = 972x + 792(1-x) = 792 + 180x

21

Repeated PD

• Now for the A types, the chance they will meet a T is x

• The chance they will meet an A is (1-x).

• Thus their payoffs are:

π(A) = 936(x) + 864(1-x) = 864 + 72x

To observe whether one strategy is better than another we must determine the population proportions required to sustain a strategy.

22

Repeated PD

• Thus:

π(T)> π(A) iff

792 + 180x > 864 + 72x

108x > 72

x > 2/3

• Thus if more than 2/3 of the initial starting population are Ts then Ts will come to dominate the population.

• If less than 2/3 of the population is made up of Ts then As will become the dominant species.

• The point of 66.6% Ts is thus the tipping point for this population, and any point beyond that forms the ‘basin of attraction’ for a 100% T equilibrium.

• Similarly any point below 66.6% will form the ‘basin of attraction’ for the all A equilibrium.

23

0 100%

Population Proportion of Ts (x)

Payoff

Figure 13.4 Fitness Graphs and Equilibria for the Thrice Repeated PD

A

T

E

972

864

936

792

66.6%

24

Repeated PD

• What if there are n rounds in the game?

• Well, the players simply get the payoff 288

or 324 modified by some amount (as per

the table).

• We go through the same process as

previously…

25

Column

A T

Row

A (288n,

288n)

(288n + 72,

288n - 72)

T (288n - 72,

288n + 72) (324n, 324n)

Figure 13.5. nth Repeated Prisoner’s Dilemma

26

Repeated PD

• If the proportion of T types is x then a T

will meet a T with probability x and an A

with probability (1-x) resulting in:

π(T) = x(324n) + (1-x)(288n – 72)

• If an A meets a T with probability x and an

A with probability (1-x) then this results in:

π(A) = x(288n + 72) + (1-x)(288n)

27

Repeated PD

• Thus the T type is fitter if:

π(T) > π(A) x(324n) + (1-x)(288n – 72) > x(288n + 72) + (1-x)(288n)

36xn > 72

x> 2/n

• Thus there are two monomorphic equilibria – 100% A, and 100% T

• There is an unstable point at x = 2/n

• An important point is that as the number of expected rounds increases, the population proportion of initial Ts to ensure a 100% T equilibrium diminishes.

28

Evolutionary Chicken

• We assume that players are distributed into the types Wimp and Macho.

• We allow x to be the proportion of Machos in the population.

• The interesting point about this game is that each strategy is relatively fitter when it is the minority in the population.

• The population will tend towards a polymorphic equilibrium.

29

Column

Wimp Macho

Row

Wimp 0, 0 -1, 1

Macho 1, -1 -2, -2

Figure 13.6. Evolutionary Chicken

30

Evolutionary Chicken

• The probability that a Macho will meet a Macho is x, and thus that a Macho will meet a Wimp is (1-x).

π(M) = -2(x) + 1(1-x) = 1 – 3x

• Similarly, for a Wimp to meet a Macho the probability is x and to meet a Wimp is (1-x)

π(W) = -1(x) + 0 (1-x) = -x

π(M) > π(W) if.f

1 – 3x > -x

x < 1/2

31

0 100%Population Proportion of Macho (x)

Payoff

Figure 13.7. Evolutionary Chicken

Wimp

Macho

0.5

32

Evolutionary Chicken

• This all means that if there is a proportion of less than half of Machos, the population will gain more Machos until there are 50% Machos and 50% Wimps.

• This 50/50 Split is the Stable polymorphic equilibrium (although it could slingshot around this like in our example previously until it reaches this stable point).

• This is very interesting when we look at the relationship between our mixed strategy NE from the rational behaviour version and this polymorphic ESS.

33

Evolutionary Assurance

• We can take a basic assurance game and once

more state that individuals will adopt specific

strategies depending on their genetic (social)

predisposition.

• For example we have two groups Women and

Men.

• Both groups prefer to meet up, but additionally

both strictly prefer meeting at one location V

(Vida) rather than S (Seattle)

34

Women

S V

MEN

S 1, 1 0, 0

V 0, 0 2, 2

Figure 13.8. Evolutionary Assurance Coffee Matching

35

Evolutionary Assurance

• If x is the proportion of S lovers (men or women) then (1-x) is the proportion of V types.

• Thus the expected payoff to any S is:

π(S) = x(1) + 0(1-x)

• And the payoff to any V is

π(V) = x(0) + 2(1-x)

• Thus π(S)> π(V) if.f

x> 2-2x => x>2/3

• There must be more than 66.6% of the population who are S lovers for S lovers to move to dominance.

36

0 100%

Population Proportion x of Seattle Lovers

Payoff

Figure 13.9. Evolutionary Assurance Game

V

S

66.6%

37

The Hawk-Dove Game

• Take two animals of the same species, some of them are innately aggressive, some of them are innately passive.

• These animals engage with one another when it comes to control and consumption of resources.

• We can also generalize these interactions to predispositions of individuals, populations or countries/governments.

38

The Hawk-Dove Game

• Hawks always fight for the whole value of the resource (V).

• Any Hawk is as likely to win and get V or lose and get –C, each of these occur with 50% probability.

• The expected payoff for any Hawk is thus (V-C)/2

• When two Doves meet they share the resource without a fight getting V/2 each.

• When a Hawk meets a Dove, the Dove retreats and the Hawk gets the whole resource V.

39

COLUMN

Hawk Dove

ROW

Hawk (V-C)/2, (V-C)/2 V, 0

Dove 0, V V/2, V/2

Figure 13.12. The Hawk-Dove Game

40

The Hawk-Dove Game

• There are 5 different cases for this game:

1. The Rational Strategic Choice Outcomes

2. Evolutionary Outcome for V>C

3. Evolutionary Outcome for V<C

4. Stable Polymorphism for V>C

5. Mixing between Hawk-Dove

We consider each case individually.

41

H-D: Rational Theory Outcomes

Two possibilities:

1. V>C: the game is a Prisoner’s Dilemma and

players will always choose to play Hawk.

2. V<C: the game is a game of Chicken and Hark

is no longer dominant. Instead there are two

pure strategy NEs of (Hawk; Dove) and (Dove;

Hawk) and a mixed strategy:

p(V-C)/2 + (1-p)V = p(0) + (1-p)V/2

=> p = V/C

42

H-D: Evolutionary V>C

• We check whether a mutant Dove could invade a population of Hawks.

• Let d be the population proportion of Doves and (1-d) the proportion of Hawks.

• Hawks/Doves meet Doves with probability d and Hawks with probability (1-d).

π(H) = d(V) + (1-d)(V-C)/2

π(D) = d(V/2) + (1-d)0

• π(H)> π(D) for all d.

• Thus the population cannot be invaded.

43

H-D: Evolutionary V>C

• The same holds true for any initial population distribution with any proportion d of Doves. Hawks will dominate and Doves will go to extinction.

• Moreover a 100% Dove population is vulnerable to invasion by mutant Hawks.

– The Hawks will get their higher payoffs.

• As in the rational version of the game, the NE is the ESS.

44

H-D: Evolutionary V< C

• In this case the game is evolutionary chicken.

• Thus we will have a polymorphic equilibrium.

• Once more we assume d as the population proportion of Doves and (1-d) that of the Hawks.

• At the margin then, a Dove could invade a population made up entirely of Hawks.

• Why? When d is small,

d(V/2) + (1-d)0 > d(V) + (1-d)(V-C)/2

• The Dove mutants have a higher payoff than the Hawks.

45

H-D: Evolutionary V< C

• Conversely for a population of all Doves, an

individual Hawk could also invade.

• Let h be the population of Hawks in a population

and (1-h) be this proportion for Doves.

• Hawks are fitter than Doves when h is small

because:

=> h(V-C/2) + (1-h)(V) > (h)0 + (1-h)(V/2)

• For any small h.

• We use this to find the polymorphic equilibrium.

46

H-D: Evolutionary V< C

Polymorphic Equilibrium

• Thus we must find the proportions for this polymorphic equilibrium:

π(H) = π(D)

h(V-C/2) + (1-h)(V) = (h)0 + (1-h)(V/2)

h(V-C/2) + (1-h)(V/2) = 0

V – hC = 0

h=V/C

• Which is the same probability in which the individuals played Hawk in the static/Rational version of the game.

47

H-D: Evolutionary Mixing

• We now assume a 3rd type of strategy (Maynard’s Cuckoo) M that mixes between Hawk and Dove.

• It plays Hawk with probability p and Dove with probability (1-p).

• We assume that p = V/C.

• When an H meets an M:

π(H) = p(V-C/2) + (1-p)V

= -1/2(V/C)(C-V) + (V/C)(C-V)

= V(C-V)/2C

48

H-D: Evolutionary Mixing

• When a D meets an M,

π(D) = p(0) + (1-p)(V/2)

= ((C-V)/C)(V/2)

= V(C-V)/(2C)

• These two payoffs are equal.

• Let V(C-V)/(2C) = K

• Now we check whether a mutant Hawk could enter a population entirely of M.

49

H-D: Evolutionary Mixing

• When a D meets an M, it is trivial to show that a mutant Hawk could not invade, because the M payoff will, on average, be better than the mutant Hawk.

• Similarly for a mutant Dove. Their payoffs are not high enough because when the M meets them as a ‘Dove’ type it does as well, and when it meets them as a ‘Hawk’ type it does strictly better.

• Thus a 100% M population is an ESS!