Carnival of Evolution #70: the games of evolution.

Bradly Alicea

Originally posted on Synthetic Daisies, April 1, 2014.

"It's not whether you win or lose, it's how you play the game" -- Grantland Rice, in an oddly

prescient and hypothetico-deductive capacity.

This month's Carnival of Evolution (#70) theme will be evolutionary games, broadly defined.

Games are generally associated with strategy and intentionality (e.g. having a brain). In fact,

formal game theory (the kind developed by John von Neumann and John Nash) arises from the

mathematical study of human decision-making and economic theory [1]. However, game theory

has also been applied to biological systems such as the dynamically stable behavioral states

exhibited by E. coli [2] and viruses [3]. Game theory can also be applied to many proximal

animal behaviors. While perhaps not the objects of selection themselves, these proximal

behaviors can be better understood in the context of an adaptive game. In this post, I will not

advocate for any one approach to evolutionary game theory, but will offer a guided tour

exploring the possibilities for this approach. The month's posts will be presented at various

points in this discussion.

The outcome of evolutionary games? TOP: tree of life (sensu Woese). BOTTOM: evolution of

complexity (sensu Gould).

Game theory traditionally quantifies the outcomes of intentional actions. In evolutionary

game theory, we are quantifying the discrete interactions between individuals. This does not

require formal cognitive mechanisms, only biological units (e.g. genes, organisms, or even

populations) that interact over time. Evolutionary game theory bears a striking conceptual

resemblance to population genetics. But instead of using a gene metaphor, the metaphor of

strategy is used. When these strategic interactions are shaped by natural selection and population

processes, the results are evolutionary dynamics. Evolutionary dynamics shape not only shape

microevolution, but have an influence on macroevolution as well.

Early game theory afficionados, in the pursuit of GOFAI.

The month's posts, part 1

In the post "Fixing on the Nitrogen fixation problem" at Mermaid's Tale, Anne

Buchanan presents a post on the biology of Nitrogen fixation in plants, and poses

it as an long-term scientific research problem with far-reaching

consequences. Holly Dunsworth, also writing at Mermaid's Tale, discusses the

challenges of teaching evolutionary concepts in her post "Are we removing the

wisdom along with the teeth?". John Wilkins from Evolving Thoughts enlightens

us about species concepts and the history of biology in "The Origins of

Speciation".PZ Myers presents a post at Pharyngula called "Pathways to Sex",

which is a comprehensive review on the evolution, diversity, and genetics of

sexual dimorphism. Jonathan Richardson highlights a new Trends in Ecology and

Evolution paper on the blog Eco-Evo-Evo-Eco (Eco-evolutionary Dynamics). As

one of the co-authors, he provides a discussion of local adaptation, or adaptation

at very small geographic scales [4].

Examples of evolutionary dynamics. COURTESY: Box 1, Figure 1 in [5].

How does game theory fit into evolutionary theory? Here are some definitions and their

broader implications in the context of evolutionary game theory:

Decision: Decision-making is not always a cognitive function. In evolutionary game theory,

decision-making can relate to the replication of genes or behaviors, which is a prime imperative

of life. Replicator dynamics, then, are the results of making a decision over and over again. After

each decision is made, a payoff can be assigned to the outcome. In evolutionary game theory, the

payoff (positive or negative) has a fitness consequence. These consequences provide feedback to

the player (e.g. organism) to act upon during subsequent decisions.

Outcome of the Hawk-Dove game in terms of population dynamics. COURTESY: Figure 1 in

[6]

Strategy Suite: The player (in this case, an organism) must choose a strategy to counter

responses by conspecifics, predators/prey, or even environmental stimuli. A pure strategy is a

single strategy played at a given time-point. A mixed strategy involves choosing from a number

of strategies at a given time point [7]. In the case of an evolutionary stable state (ESS), each

player's mixed strategy suite converges to a pure strategy over time [8]. These pure strategies are

optimal in the sense that established pure strategies cannot be beaten by upstart strategies that

might emerge in a population over time.

Outcome of a mutualistic relationship (legume-bacterium) modeled as a Prisoner's Dilemma

game and outcomes shown in terms of population dynamics. COURTESY: Figure 3 in [6].

Strategy as variation: The existence of pure or mixed strategies may be tied to genetic

variation. However, the evolution of these strategy suites (e.g. how they are deployed) is a

function of natural selection. One example of this is when a strategy becomes evolutionarily

stable in a population. Once a given strategy is fixed in the population, natural selection can

maintain its dominance, even as lower-frequency mutant strategies emerge [9]. In this sense,

strategies behave like loci in population genetics theory.

Maynard-Smith on the origins of Evolutionary Game Theory [10]. COURTESY: Web of

Stories.

In evolutionary games, strategies can be defined as heritable phenotypes [11], which

range from well-defined behaviors to morphological characters. A strategy is evolutionary

feasible if it is either an extant (current existing) variant within a population or a recurring

mutant in that population. The strategies themselves can range in frequency from very rare to

dominant. A player (organism) may be a carrier for latent strategies. Such strategies may have a

low payoff in one environment while having a much higher payoff in another environment.

But how can an organism not "show all of its cards" as it were? According to [12],

evolutionary game theory exists of an inner game and an outer game. The inner game is more

akin to classical (e.g. economic) game theory where there are payoffs for intentional strategies.

However, the evolutionary version also consists of an outer game that is dynamically linked to

the inner game. This linkage allows the outer game to take the form of translating these payoffs

into changes in phenotypic frequencies. This allows us to bridge proximal and ultimate causes of

adaptive change in a population.

The month's posts, part 2

Jeremy Yoder from Nothing in Biology Makes Sense! reviews the Festival of Bad

Ad-hoc Hypotheses. In "BAH! This looks amazing", Jeremy introduces us to the

quest to discover the best "well-argued and thoroughly researched but completely

incorrect evolutionary theory". Then, writing at Molecular Ecologist, Jeremy

discusses the occurrence of soft selective sweeps in bacterial populations of the

gut. Adam Goldstein of The Shifting Balance of Factors critiques scala

naturae views of evolution in "March of Progress, reloaded". Ed

Yong from Phenomena presents a new paper that highlights the role of doublesex,

which enables mimicry in the female common mormon butterfly (Papilio

polytes). Here's an interview of Baba Brinkman by Kylie Sturgess at CSI's

Curiouser and Curiouser blog. Baba Brinkman raps about evolution on a regular

basis. You will have to go to the post to find out more. And at the BEACON

Center blog, Danielle Whittaker introduces us to the work of Tyler Heather, who

works on the role of gene-phenotype interactions in the speed of adaptation, and

Raffica LaRosa, who measures natural selection in flowers.

"Time (evolution) is a game played beautifully by children (juveniles)" -- Heraclitus, perhaps

anticipating the rise of Evo-Devo

Hawk-Dove Games

Hawk-Dove games are the traditional two-player zero-sum games most people are

familiar with. A game with the simplest type of outcome, hawk-dove produces a winner and a

loser. Only winning is stable (e.g. winner-take-all), so such games often result in arms races and

necessitate conflict. While a pure "hawk" strategy is stable in the short term, it may not be

evolutionarily stable.

In an evolutionary context, Hawk-Dove can also be characterized as the well-known Red

Queen (a special instance of zero-sum game theory) [13]. The Red Queen, which characterizes

co-evolutionary arms races, provides a means for the emergence of complex evolutionary

dynamics between two species (e.g. players).

The month's posts, part 3

Razib Khan, writing at Unz Review, considers what can be learned from the re-

analysis of open-access genome data in a post entitled "Reanalyzing Data: it does

a mind good". Lesson: re-analysis is a highly fruitful endeavor. Dan Graur from

the Judge Starling tumblr brings us a discussion of the ENCODE project in

relation to the gene concept in "Mutons, Cistrons, Recons, & Nuons: News

Concerning the Death of “Gene” are Greatly Exaggerated". At Genealogical

World of Phylogenetic Networks, David Morrision leads a bibliometrics-based

discussion on the emergence and current state of phylogenetics research as a

subfield of evolutionary biology in "Has phylogenetics reached its apogee?".

Moving from analysis to modeling, Artem Kaznatcheev at the Theory, Evolution,

and Games group blog discusses a recent evolutionary-oriented theoretical

Computer Science conference in "Computational theories of evolution" and

"Algorithmic Darwinism".

Illustration of Hawk-Dove dynamics. COURTESY: Evolutionary Game Theory

Wikipedia page.

Two player games with complexity.

Rock-paper-scissor Games

Rock-paper-scissor games are defined by their non-transitive outcomes. In Sinervo and Lively

[14], male side-blotched lizard phenotypes give rise to three behavioral strategies. While

competitive, these behaviors do not result in a definitive winner. For example, while there is a

clearly dominant strategy (blue-throated guarders) that provides the highest payoff, alternate

strategies (yellow-throated sneakers and orange-throated usurpers) can also be stable. Rather

than converging to a pure strategy where competition would be winner-take-all, multiple

strategies can co-exist at varying frequencies indefinitely.

Example of Rock-Scissors-Paper in Side-blotched Lizard. COURTESY: Sinervo Lab (UCSC).

The month's posts, part 4 John Hawks, writing at his weblog, provides information and his own insights on

"A new early modern human genome from Siberia", which was isolated and

sequenced from a 45,000 year-old femur. Moving from ancient genomics to

theory, we have two posts on mechanisms and misunderstandings. The first

is Philip Ball's (Homunculus blog) take on the "Molecular mechanisms of

evolution". The second is from Larry Moran of Sandwalk blog, who introduces us

to "A chemist who doesn't understand evolution". Returning to human evolution,

but moving on to analysis, The Olduvai Gorge tumblr site provides a preview of

and link to the new article "The Doubly-Conditioned Frequency Spectrum does

not distinguish between ancient population structure and hybridisation". The

paper itself is a critique of a popular method used in studies of phylodemography.

Prisoner's Dilemma and Snowdrift Games

Most biologists are familiar with the Prisoners' Dilemma (PD) -- in fact, this is the

canonical game for demonstrating the evolution of cooperation [15]. A slightly less familiar

variant of the PD game is the snowdrift (or cooperation) game. In both PD and snowdrift games,

the maximal payoff results from cooperation and coordination between players rather than

competition.

Payoff matrix for the PD game using a generic example. A 2x2 payoff

matrix. COURTESY: Animalbehavioronline.com

Using the snowdrift game as an example, two players are confronted with the task of

clearing away a snowdrift. If completed, the work benefits them both. However, if only one

player decides to undertake the task, the second player can benefit without contributing (e.g.

free-riding). But since the first player is unlikely to put up with free-riding over repeated plays of

the game, the highest payoff for both players over repeated plays is attained from full

cooperation in performing the work. As this strategy is replicated over evolutionary time, it

becomes the dominant strategy. Thus, players converge upon this pure strategy through the

maximization of payoffs [16], and it becomes evolutionarily stable.

The month's posts, part 5

Bjorn Ostman from Pleiotropy presents a review of evolutionary dynamics in

holey fitness landscapes. Charles Goodnight from the excellent Evolution in

Structured Populations blog gives us three tutorial-esque posts the month:

"Mating structure, Interaction structure, and Selection Structure", "Griffing,

Associate Effects, and Heritability", and "Measuring the Heritability of

Contextual Traits". The population biology preprint blog Haldane's Sieve features

a new paper (now accepted at PLoS One) called "The Arrival of the Frequent:

how bias in genotype-phenotype maps can steer populations to local optima".

Using both simulation and genotype-phenotype maps, this paper demonstrates

that as rare variants, the fittest organisms in a population often do not survive to

be fixed or otherwise represented at evolutionary timescales. And in the spirit of

evolutionary computation, IEEE Spectrum has a feature on how bug-ridden

computer code is being refactored and otherwise fixed using genetic

algorithms derived from evolutionary theory.

“A mans friendships are one of the best measures of his worth” -- Charles Darwin

Stackelberg and Pursuit-Evasion Games

These types of games are not as familiar to biologists. However, in their instantiated

form, they appear to be quite useful to the evolution of biological complexity.

Stackelberg (or first-mover) games [17] might explain much about the emergence of

evolutionary constraints and biological complexity. One simple example of such a game is the

leader-follower game. The leader moves first by choosing from a mixed strategy suite, usually in

a way that maximizes the payoff. The second player must then continually respond to the actions

of the first move, as they are constrained from using a full set of possible strategies. While the

second player gains information from the first-mover's strategy, it only allows them to maximize

their payoff from a subset of strategies. This might explain the emergence of symbiotic

relationships, or perhaps the emergence of social dominance hierarchies.

Pursuit-evasion (or cops and robbers) games might explain the emergence of predator-prey

relationships. As is the case with Stackleberg games, the order in which turns are taken becomes

an important determinant of the payoff. In pursuit-evasion, however, the first mover (evader) is

constrained by what it takes to successfully avoid the pursuant (second mover) [18]. These types

of games are generally zero-sum, although they need not be.

Leader-follower dynamics, presented as an abstract model. COURTESY: Evolutionary Bilevel

Optimization.

Predator-prey dynamics in a two-state system. COURTESY: Wolfram Demonstrations Project.

The tic-tac-toe (a.k.a. naughts and crosses) game is an example of how leader-follower

dynamics can produce stable equilibria. In tic-tac-toe, there are first movers and second movers.

While optimal play by both players will result in a tie, the first move can often win the game if

the second mover makes a suboptimal move.

Tic, tac, toe! Sometimes learning how to play games are a matter of life and death.

Games Against Nature

Games against nature are 1-player games where the sole player implements a strategy

against a random process. The payoff is determined by how well the intentional player fares

against the random process. The obvious extension of this is an organism adapting in the face of

natural selection. One example of a game against nature can be found in cellular automata [19].

Cellular automata operate using simple rules imposed of a single cell by both its neighbors and

stochastic processes that lead to emergent patterns across a grid of cells. While such games do

not rely on competition nor cooperation, they do produce coordinated outcomes. In Conway's

Game of Life, each cell is "born" or "killed" based on the states of its neighbors. The result is not

a formal payoff matrix, but rather a set of patterns that persist or die off. Unlike zero-sum or

conventional cooperation games, the outcome of the game is non-deterministic.

A cellular automata game against nature, played during development.

The month's posts, part 6 Carlos Araya from CEHG Blog reviews the latest findings in the area of

experimental evolution in a post called "Dissecting the dynamics of adaptation

with experimental evolution". Henry Gee from The End of the Pier Show brings

us a paleontologically-inspired tale entitled "Careful with that Amphiooxus,

Eugene". Aeon Magazine has a feature this month on selfish gene theory as a

takeoff on the blogosphere kerfuffle started by David Dobbs with his article "Die,

Selfish Gene, Die!". Their roundtable includes David Dobbs, Robert Sapolsky,

Laura Hercher, Karen James, and John Dupre (a writer, a genetic counselor, two

biologists, and a philosopher). For further critical assessment of this roundtable,

see posts by Jerry Coyne at Why Evolution is True and Larry

Moran of Sandwalk.

John Conway, on the origins of his "game of life" (a game against

nature). COURTESY: Numberphile.

"There are no shortcuts in evolution" -- Louis D. Brandeis, who was not a biologist.

Many modern video games (such as first-person shooter games) are essentially games against

nature. In this conception, nature is an artificial agent that presents challenges to a player, which

can be overcome through either inherent skill or an adaptive solution. What if we could replace

the goal-directed behaviors of a player with evolutionary imperatives?

Evolutionary Simon: a plot device devised for this post, but does the model fit the data?

To model this possibility using a formal game model, I introduce something called

Evolutionary Simon. Simon is a programmed board game developed in the 1970s that might also

be used to model the proximate effects of behavioral selection. Recall that the Simon game

presents a sequence of lighted tiles (e.g. blue, blue, yellow, blue, red, green, red) that is generated

by a computer program. The player must then imitate this sequence by pressing the right buttons

in the correct order.

So far, this resembles a typical free recall (learning and memory) experiment. Now let us

introduce a diversity of players, some with greater innate recall capacity, some with less. This

innate capacity is improved upon by getting a correct answer. The payoff matrix for this 3x1

game:

Payoff matrix for Evolutionary Simon game. Payoffs are for strategies employed by a player (top

row). ε is used to distinguish minimally correct response from incorrect.

Incorrect

Partially Correct

Fully Correct

Simon Output

0

(1 – (1/cr)) + ε

1

Players respond to the output using either an entirely inappropriate response, a fully correct

response, or a partially correct response (which exposes the limitations of their memory).

Partially correct responses (cr) are scored by how many components of the original sequence

they were able to recall. For every turn, an agent receives a payoff. The length of a Simon

sequence can be used as a source of environmental selection.

In the end, the agents that end up with the largest payoffs over a wide range of generated patterns

are the fittest. But we can end up with quite interesting evolutionary dynamics. For example,

some agents might receive very high payoffs for specific patterns. And other agents might be

able to garner a sizeable payoff for nearly every pattern presented.

The month's posts, part 7

The Cosmos reboot hosted by Neil DeGrasse Tyson is coming along nicely.

Despite a few detail-oriented and denialism-related glitches, it has become a great

opportunity to make science accessible to a broader audience (episode 2 was

exclusively on evolution). I have been providing supplemental references on

selected topics from each episode here on Synthetic Daisies. Here are the

supplemental readings for the first episode (Section II of "Bits and

Starstuff"), second episode (Section II of "Futures of More Starstuff"), and third

episode (Section III of "Ancien Regimes, Google Grokking, and Starstuff"). Larry

Moran at Sandwalk has provided his own insights into the factual and conceptual

shortcomings of evolution, Cosmos-style. Greg Laden's Blog features a post

called "Will Neil DeGrasse Tyson's Cosmos be a turning point in science

denialism?", which considers the potential of the Cosmos reboot to combat

science denialism. In the spirit of combating bad scientific ideas, Alex B.

Berezow at Real Clear Science heeds us to "End the Hype over Epigenetics and

Lamarckian Evolution", and does so by highlighting a new paper in Cell [20].

While there have been many interesting recent findings regarding the potential for

short-term epigenetic heritability, it is also important to remember why Lamarck

fell into disrepute in the first place (HINT: it has to do with long-term

mechanisms). And finally, in the spirit of pop-science, here is an

infographic from Visual.ly and Juan Martinez on the History of Life.

"Evolution is all about survival of the (your most stable equilibrium here)" -- one possible moral

of our story

One lesson learned from modeling evolution as a game is that popular conceptions of

evolution such as "survival of the fittest" are fundamentally incorrect. Indeed, modeling mixed

strategy intra-specific competition using a rock-paper-scissors game [2] results in a "survival of

the weakest". Another lesson is that evolutionary games are more than simply a matter of zero-

sum competition or stable cooperation [21]. Despite the metaphor, evolutionary games are more

about capturing interactions than direct intentionality. However, contemporary models focus on

the role of natural selection in evolution. Yet due to their flexibility, evolutionary games could

also be used to model neutral processes and other contributors to evolutionary dynamics.

There are other lessons to be learned as well, including the linkages between micro- and

macroevolution and the evolution of sociality. Game-theory models can be combined with other

concepts at the intersection of economics and evolutionary biology to understand behavioral

signaling and other forms of informative communication. Examples include such as hedging

(managing trade-offs), biological markets [22], and handicapping [23]. So is life just one big

game? According to game theory and the application of game-inspired models, the answer is

"yes".

This month's Carnival is also available in printable form (on Figshare) for teaching

purposes. And don't forget to check out next month's Carnival of Evolution. Until then, enjoy

this month's posts. And remember, the game is not over until evolution has occurred.

NOTES: [1] von Neumann, J. and Morgenstern, O. Theory of Games and Economic Behavior. Princeton

Press (1947) AND Nash, J., Kuhn, J.W., Nasar, S. The Essential John Nash. Princeton Press

(2007).

[2] Kerr, B., Riley, M.A., Feldman, M.W., and Bohannan, B.J.M. Local dispersal promotes

biodiversity in a real-life game of rock–paper–scissors. Nature, 418, 171-174 (2002).

[3] Turner, P.E. Cheating Viruses and Game Theory. American Scientist, 93(5), 428-435

(2005).

[4] For more information, read the following paper: Richardson, J.L., Urban, M.C., Bolnick, D.I.,

and Skelly, D.K. Microgeographic adaptation and the spatial scale of evolution. Trends in

Ecology and Evolution, 29(3), 165-176 (2014).

[5] Nowak, M.A. and Sigmund, K. Evolution of Indirect Reciprocity. Nature, 437, 1291-1298

(2005).

[6] Cowden, C.C. Game theory, evolutionary stable strategies, and the evolution of biological

interactions. Nature Education Knowledge, 3(10), 6 (2012).

[7] Rasmussen, E. Games and Information. Blackwell Publishing (2006).

[8] Weibull, J.W. Evolutionary Game Theory. MIT Press (1995) AND Brown, J.S. and

Vincent, T.L. Evolutionary Game Theory, Natural Selection, and Darwinian Dynamics.

Cambridge University Press (2005).

[9] Nowak, M.A. Evolutionary Dynamics: exploring the equations of life. Belknap Press

(2006) AND Broom, M. and Rychtar, J. Game-Theoretical Models in Biology. Chapman-Hall

CRC Press (2013).

[10] Maynard-Smith, J. and Price, G.R. The Logic of Animal Conflict. Nature 246 (5427): 15

(1973).

[11] Brown, J.S. Fit of form and function, diversity of life, and procession of life as an

evolutionary game. In "Adaptationism and Optimality", S.H. Orzack and E. Sober eds., Chapter

4 (1999).

[12] Vincent, T.L. and Brown, J.S. Evolution of ESS Theory. Annual Review of Ecology and

Systematics, 19, 423-443 AND Charlesworth, B. Optimization Models, Quantitative Genetics,

and Mutation. Evolution, 44(3), 520-538 (1990).

[13] Cohen, J. and Newman, C.E. Host-parasite relations and random zero-sum games: the

stabilizing effect of strategy diversification. American Naturalist, 133(4), 533-552 (1989) AND

Perc, M. and Szolnoki, A. Coevolutionary games: a mini review. Biosystems, 99, 109-125

(2010).

[14] Sinervo, B. and Lively, C.M. The rock–paper–scissors game and the evolution of

alternative male strategies. Nature, 380, 240-243 (1996).

[15] Brembs, B. Evolution of Cooperation. Brembs.net Evolution section.

[16] Shutters, S.T. Punishment, Rational Expectations, and Relative Payoffs in a Networked

Prisoners Dilemma. In "Social Computing and Behavioral Modeling", H. Liu, J. Salerno, and

M.J. Young (eds.), pgs. 1-8 (2009).

[17] McNamara, J.M., Wilson, E.M.K., and Houston, A.I. Is it better to give information,

receive it, or be ignorant in a two-player game? Behavioral Ecology, 17(3), 441-451 (2006).

[18] Basar, T. and Olsder, G.J. Dynamic Noncooperative Game Theory. Academic Press

(1995).

[19] Sigmund, K. Games of Life: explorations in ecology, evolution, and behavior. Oxford

University Press (1993) AND Wolfram, S. A New Kind of Science. Wolfram Press (2002).

[20] Heard, E. and Martienssen, R. Transgenerational Epigenetic Inheritance: myths and

mechanisms. Cell, 157(1), 95–109 (2014).

[21] Bendor, J. and Swistak, P. Types of evolutionary stability and the problem of cooperation.

PNAS, 92, 3596-3600 (1995).

[22] Noe, R. and Hammerstein, P. Biological Markets: supply and demand determine the effect

of partner choice in cooperation, mutualism, and mating. Behavioral Ecology and Sociobiology,

35(1), 1-11 (1994).

[23] Grafin, A. Biological Signals as Handicaps. Journal of Theoretical Biology, 144, 517-546

(1990).