Carnival of Evolution #70: the games of evolution. Bradly Alicea … · 2015-12-04 · Adam...
Embed Size (px)
Transcript of Carnival of Evolution #70: the games of evolution. Bradly Alicea … · 2015-12-04 · Adam...
Carnival of Evolution #70: the games of evolution.
Originally posted on Synthetic Daisies, April 1, 2014.
"It's not whether you win or lose, it's how you play the game" -- Grantland Rice, in an oddly
prescient and hypothetico-deductive capacity.
This month's Carnival of Evolution (#70) theme will be evolutionary games, broadly defined.
Games are generally associated with strategy and intentionality (e.g. having a brain). In fact,
formal game theory (the kind developed by John von Neumann and John Nash) arises from the
mathematical study of human decision-making and economic theory . However, game theory
has also been applied to biological systems such as the dynamically stable behavioral states
exhibited by E. coli  and viruses . Game theory can also be applied to many proximal
animal behaviors. While perhaps not the objects of selection themselves, these proximal
behaviors can be better understood in the context of an adaptive game. In this post, I will not
advocate for any one approach to evolutionary game theory, but will offer a guided tour
exploring the possibilities for this approach. The month's posts will be presented at various
points in this discussion.
The outcome of evolutionary games? TOP: tree of life (sensu Woese). BOTTOM: evolution of
complexity (sensu Gould).
Game theory traditionally quantifies the outcomes of intentional actions. In evolutionary
game theory, we are quantifying the discrete interactions between individuals. This does not
require formal cognitive mechanisms, only biological units (e.g. genes, organisms, or even
populations) that interact over time. Evolutionary game theory bears a striking conceptual
resemblance to population genetics. But instead of using a gene metaphor, the metaphor of
strategy is used. When these strategic interactions are shaped by natural selection and population
processes, the results are evolutionary dynamics. Evolutionary dynamics shape not only shape
microevolution, but have an influence on macroevolution as well.
Early game theory afficionados, in the pursuit of GOFAI.
The month's posts, part 1
In the post "Fixing on the Nitrogen fixation problem" at Mermaid's Tale, Anne
Buchanan presents a post on the biology of Nitrogen fixation in plants, and poses
it as an long-term scientific research problem with far-reaching
consequences. Holly Dunsworth, also writing at Mermaid's Tale, discusses the
challenges of teaching evolutionary concepts in her post "Are we removing the
wisdom along with the teeth?". John Wilkins from Evolving Thoughts enlightens
us about species concepts and the history of biology in "The Origins of
Speciation".PZ Myers presents a post at Pharyngula called "Pathways to Sex",
which is a comprehensive review on the evolution, diversity, and genetics of
sexual dimorphism. Jonathan Richardson highlights a new Trends in Ecology and
Evolution paper on the blog Eco-Evo-Evo-Eco (Eco-evolutionary Dynamics). As
one of the co-authors, he provides a discussion of local adaptation, or adaptation
at very small geographic scales .
Examples of evolutionary dynamics. COURTESY: Box 1, Figure 1 in .
How does game theory fit into evolutionary theory? Here are some definitions and their
broader implications in the context of evolutionary game theory:
Decision: Decision-making is not always a cognitive function. In evolutionary game theory,
decision-making can relate to the replication of genes or behaviors, which is a prime imperative
of life. Replicator dynamics, then, are the results of making a decision over and over again. After
each decision is made, a payoff can be assigned to the outcome. In evolutionary game theory, the
payoff (positive or negative) has a fitness consequence. These consequences provide feedback to
the player (e.g. organism) to act upon during subsequent decisions.
Outcome of the Hawk-Dove game in terms of population dynamics. COURTESY: Figure 1 in
Strategy Suite: The player (in this case, an organism) must choose a strategy to counter
responses by conspecifics, predators/prey, or even environmental stimuli. A pure strategy is a
single strategy played at a given time-point. A mixed strategy involves choosing from a number
of strategies at a given time point . In the case of an evolutionary stable state (ESS), each
player's mixed strategy suite converges to a pure strategy over time . These pure strategies are
optimal in the sense that established pure strategies cannot be beaten by upstart strategies that
might emerge in a population over time.
Outcome of a mutualistic relationship (legume-bacterium) modeled as a Prisoner's Dilemma
game and outcomes shown in terms of population dynamics. COURTESY: Figure 3 in .
Strategy as variation: The existence of pure or mixed strategies may be tied to genetic
variation. However, the evolution of these strategy suites (e.g. how they are deployed) is a
function of natural selection. One example of this is when a strategy becomes evolutionarily
stable in a population. Once a given strategy is fixed in the population, natural selection can
maintain its dominance, even as lower-frequency mutant strategies emerge . In this sense,
strategies behave like loci in population genetics theory.
Maynard-Smith on the origins of Evolutionary Game Theory . COURTESY: Web of
In evolutionary games, strategies can be defined as heritable phenotypes , which
range from well-defined behaviors to morphological characters. A strategy is evolutionary
feasible if it is either an extant (current existing) variant within a population or a recurring
mutant in that population. The strategies themselves can range in frequency from very rare to
dominant. A player (organism) may be a carrier for latent strategies. Such strategies may have a
low payoff in one environment while having a much higher payoff in another environment.
But how can an organism not "show all of its cards" as it were? According to ,
evolutionary game theory exists of an inner game and an outer game. The inner game is more
akin to classical (e.g. economic) game theory where there are payoffs for intentional strategies.
However, the evolutionary version also consists of an outer game that is dynamically linked to
the inner game. This linkage allows the outer game to take the form of translating these payoffs
into changes in phenotypic frequencies. This allows us to bridge proximal and ultimate causes of
adaptive change in a population.
The month's posts, part 2
Jeremy Yoder from Nothing in Biology Makes Sense! reviews the Festival of Bad
Ad-hoc Hypotheses. In "BAH! This looks amazing", Jeremy introduces us to the
quest to discover the best "well-argued and thoroughly researched but completely
incorrect evolutionary theory". Then, writing at Molecular Ecologist, Jeremy
discusses the occurrence of soft selective sweeps in bacterial populations of the
gut. Adam Goldstein of The Shifting Balance of Factors critiques scala
naturae views of evolution in "March of Progress, reloaded". Ed
Yong from Phenomena presents a new paper that highlights the role of doublesex,
which enables mimicry in the female common mormon butterfly (Papilio
polytes). Here's an interview of Baba Brinkman by Kylie Sturgess at CSI's
Curiouser and Curiouser blog. Baba Brinkman raps about evolution on a regular
basis. You will have to go to the post to find out more. And at the BEACON
Center blog, Danielle Whittaker introduces us to the work of Tyler Heather, who
works on the role of gene-phenotype interactions in the speed of adaptation, and
Raffica LaRosa, who measures natural selection in flowers.
"Time (evolution) is a game played beautifully by children (juveniles)" -- Heraclitus, perhaps
anticipating the rise of Evo-Devo
Hawk-Dove games are the traditional two-player zero-sum games most people are
familiar with. A game with the simplest type of outcome, hawk-dove produces a winner and a
loser. Only winning is stable (e.g. winner-take-all), so such games often result in arms races and
necessitate conflict. While a pure "hawk" strategy is stable in the short term, it may not be
In an evolutionary context, Hawk-Dove can also be characterized as the well-known Red
Queen (a special instance of zero-sum game theory) . The Red Queen, which characterizes
co-evolutionary arms races, provides a means for the emergence of complex evolutionary
dynamics between two species (e.g. players).
The month's posts, part 3
Razib Khan, writing at Unz Review, considers what can be learned from the re-
analysis of open-access genome data in a post entitled "Reanalyzing Data: it does
a mind good". Lesson: re-analysis is a highly fruitful endeavor. Dan Graur from
the Judge Starling tumblr brings us a discussion of the ENCODE project in
relation to the gene concept in "Mutons, Cistrons, Recons, & Nuons: News
Concerning the Death of “Gene” are Greatly Exaggerated". At Genealogical
World of Phylogenetic Networks, David Morrision leads a bibliometrics-based
discussion on the emergence and current state of phylogenetics research as a
subfield of evolutionary biology in "Has phylogenetics reached its apogee?".
Moving from analysis to modeling, Artem Kaznatcheev at the Theory, Evolution,
and Games group blog discusses a recent evolutionary-oriented theoretical
Computer Science conference in "Computational theories of evolution" and
Illustration of Hawk-Dove dynamics. COURTESY: Evolutionary Game Theory
Two player games with complexity.
Rock-paper-scissor games are defined by their non-transitive outcomes. In Sinervo and Lively
, male side-blotched lizard phenotypes give rise to three behavioral strategies. While
competitive, these behaviors do not result in a definitive winner. For example, while there is a
clearly dominant strategy (blue-throated guarders) that provides the highest payoff, alternate
strategies (yellow-throated sneakers and orange-throated usurpers) can also be stable. Rather
than converging to a pure strategy where competition would be winner-take-all, multiple
strategies can co-exist at varying frequencies indefinitely.
Example of Rock-Scissors-Paper in Side-blotched Lizard. COURTESY: Sinervo Lab (UCSC).
The month's posts, part 4 John Hawks, writing at his weblog, provides information and his own insights on
"A new early modern human genome from Siberia", which was isolated and
sequenced from a 45,000 year-old femur. Moving from ancient genomics to
theory, we have two posts on mechanisms and misunderstandings. The first
is Philip Ball's (Homunculus blog) take on the "Molecular mechanisms of
evolution". The second is from Larry Moran of Sandwalk blog, who introduces us
to "A chemist who doesn't understand evolution". Returning to human evolution,
but moving on to analysis, The Olduvai Gorge tumblr site provides a preview of
and link to the new article "The Doubly-Conditioned Frequency Spectrum does
not distinguish between ancient population structure and hybridisation". The
paper itself is a critique of a popular method used in studies of phylodemography.
Prisoner's Dilemma and Snowdrift Games
Most biologists are familiar with the Prisoners' Dilemma (PD) -- in fact, this is the
canonical game for demonstrating the evolution of cooperation . A slightly less familiar
variant of the PD game is the snowdrift (or cooperation) game. In both PD and snowdrift games,
the maximal payoff results from cooperation and coordination between players rather than
Payoff matrix for the PD game using a generic example. A 2x2 payoff
matrix. COURTESY: Animalbehavioronline.com
Using the snowdrift game as an example, two players are confronted with the task of
clearing away a snowdrift. If completed, the work benefits them both. However, if only one
player decides to undertake the task, the second player can benefit without contributing (e.g.
free-riding). But since the first player is unlikely to put up with free-riding over repeated plays of
the game, the highest payoff for both players over repeated plays is attained from full
cooperation in performing the work. As this strategy is replicated over evolutionary time, it
becomes the dominant strategy. Thus, players converge upon this pure strategy through the
maximization of payoffs , and it becomes evolutionarily stable.
The month's posts, part 5
Bjorn Ostman from Pleiotropy presents a review of evolutionary dynamics in
holey fitness landscapes. Charles Goodnight from the excellent Evolution in
Structured Populations blog gives us three tutorial-esque posts the month:
"Mating structure, Interaction structure, and Selection Structure", "Griffing,
Associate Effects, and Heritability", and "Measuring the Heritability of
Contextual Traits". The population biology preprint blog Haldane's Sieve features
a new paper (now accepted at PLoS One) called "The Arrival of the Frequent:
how bias in genotype-phenotype maps can steer populations to local optima".
Using both simulation and genotype-phenotype maps, this paper demonstrates
that as rare variants, the fittest organisms in a population often do not survive to
be fixed or otherwise represented at evolutionary timescales. And in the spirit of
evolutionary computation, IEEE Spectrum has a feature on how bug-ridden
computer code is being refactored and otherwise fixed using genetic
algorithms derived from evolutionary theory.
“A mans friendships are one of the best measures of his worth” -- Charles Darwin
Stackelberg and Pursuit-Evasion Games
These types of games are not as familiar to biologists. However, in their instantiated
form, they appear to be quite useful to the evolution of biological complexity.
Stackelberg (or first-mover) games  might explain much about the emergence of
evolutionary constraints and biological complexity. One simple example of such a game is the
leader-follower game. The leader moves first by choosing from a mixed strategy suite, usually in
a way that maximizes the payoff. The second player must then continually respond to the actions
of the first move, as they are constrained from using a full set of possible strategies. While the
second player gains information from the first-mover's strategy, it only allows them to maximize
their payoff from a subset of strategies. This might explain the emergence of symbiotic
relationships, or perhaps the emergence of social dominance hierarchies.
Pursuit-evasion (or cops and robbers) games might explain the emergence of predator-prey
relationships. As is the case with Stackleberg games, the order in which turns are taken becomes
an important determinant of the payoff. In pursuit-evasion, however, the first mover (evader) is
constrained by what it takes to successfully avoid the pursuant (second mover) . These types
of games are generally zero-sum, although they need not be.
Leader-follower dynamics, presented as an abstract model. COURTESY: Evolutionary Bilevel
Predator-prey dynamics in a two-state system. COURTESY: Wolfram Demonstrations Project.
The tic-tac-toe (a.k.a. naughts and crosses) game is an example of how leader-follower
dynamics can produce stable equilibria. In tic-tac-toe, there are first movers and second movers.
While optimal play by both players will result in a tie, the first move can often win the game if
the second mover makes a suboptimal move.
Tic, tac, toe! Sometimes learning how to play games are a matter of life and death.
Games Against Nature
Games against nature are 1-player games where the sole player implements a strategy
against a random process. The payoff is determined by how well the intentional player fares
against the random process. The obvious extension of this is an organism adapting in the face of
natural selection. One example of a game against nature can be found in cellular automata .
Cellular automata operate using simple rules imposed of a single cell by both its neighbors and
stochastic processes that lead to emergent patterns across a grid of cells. While such games do
not rely on competition nor cooperation, they do produce coordinated outcomes. In Conway's
Game of Life, each cell is "born" or "killed" based on the states of its neighbors. The result is not
a formal payoff matrix, but rather a set of patterns that persist or die off. Unlike zero-sum or
conventional cooperation games, the outcome of the game is non-deterministic.
A cellular automata game against nature, played during development.
The month's posts, part 6 Carlos Araya from CEHG Blog reviews the latest findings in the area of
experimental evolution in a post called "Dissecting the dynamics of adaptation
with experimental evolution". Henry Gee from The End of the Pier Show brings
us a paleontologically-inspired tale entitled "Careful with that Amphiooxus,
Eugene". Aeon Magazine has a feature this month on selfish gene theory as a
takeoff on the blogosphere kerfuffle started by David Dobbs with his article "Die,
Selfish Gene, Die!". Their roundtable includes David Dobbs, Robert Sapolsky,
Laura Hercher, Karen James, and John Dupre (a writer, a genetic counselor, two
biologists, and a philosopher). For further critical assessment of this roundtable,
see posts by Jerry Coyne at Why Evolution is True and Larry
Moran of Sandwalk.
John Conway, on the origins of his "game of life" (a game against
nature). COURTESY: Numberphile.
"There are no shortcuts in evolution" -- Louis D. Brandeis, who was not a biologist.
Many modern video games (such as first-person shooter games) are essentially games against
nature. In this conception, nature is an artificial agent that presents challenges to a player, which
can be overcome through either inherent skill or an adaptive solution. What if we could replace
the goal-directed behaviors of a player with evolutionary imperatives?
Evolutionary Simon: a plot device devised for this post, but does the model fit the data?
To model this possibility using a formal game model, I introduce something called
Evolutionary Simon. Simon is a programmed board game developed in the 1970s that might also
be used to model the proximate effects of behavioral selection. Recall that the Simon game
presents a sequence of lighted tiles (e.g. blue, blue, yellow, blue, red, green, red) that is generated
by a computer program. The player must then imitate this sequence by pressing the right buttons
in the correct order.
So far, this resembles a typical free recall (learning and memory) experiment. Now let us
introduce a diversity of players, some with greater innate recall capacity, some with less. This
innate capacity is improved upon by getting a correct answer. The payoff matrix for this 3x1
Payoff matrix for Evolutionary Simon game. Payoffs are for strategies employed by a player (top
row). ε is used to distinguish minimally correct response from incorrect.
(1 – (1/cr)) + ε
Players respond to the output using either an entirely inappropriate response, a fully correct
response, or a partially correct response (which exposes the limitations of their memory).
Partially correct responses (cr) are scored by how many components of the original sequence
they were able to recall. For every turn, an agent receives a payoff. The length of a Simon
sequence can be used as a source of environmental selection.
In the end, the agents that end up with the largest payoffs over a wide range of generated patterns
are the fittest. But we can end up with quite interesting evolutionary dynamics. For example,
some agents might receive very high payoffs for specific patterns. And other agents might be
able to garner a sizeable payoff for nearly every pattern presented.
The month's posts, part 7
The Cosmos reboot hosted by Neil DeGrasse Tyson is coming along nicely.
Despite a few detail-oriented and denialism-related glitches, it has become a great
opportunity to make science accessible to a broader audience (episode 2 was
exclusively on evolution). I have been providing supplemental references on
selected topics from each episode here on Synthetic Daisies. Here are the
supplemental readings for the first episode (Section II of "Bits and
Starstuff"), second episode (Section II of "Futures of More Starstuff"), and third
episode (Section III of "Ancien Regimes, Google Grokking, and Starstuff"). Larry
Moran at Sandwalk has provided his own insights into the factual and conceptual
shortcomings of evolution, Cosmos-style. Greg Laden's Blog features a post
called "Will Neil DeGrasse Tyson's Cosmos be a turning point in science
denialism?", which considers the potential of the Cosmos reboot to combat
science denialism. In the spirit of combating bad scientific ideas, Alex B.
Berezow at Real Clear Science heeds us to "End the Hype over Epigenetics and
Lamarckian Evolution", and does so by highlighting a new paper in Cell .
While there have been many interesting recent findings regarding the potential for
short-term epigenetic heritability, it is also important to remember why Lamarck
fell into disrepute in the first place (HINT: it has to do with long-term
mechanisms). And finally, in the spirit of pop-science, here is an
infographic from Visual.ly and Juan Martinez on the History of Life.
"Evolution is all about survival of the (your most stable equilibrium here)" -- one possible moral
of our story
One lesson learned from modeling evolution as a game is that popular conceptions of
evolution such as "survival of the fittest" are fundamentally incorrect. Indeed, modeling mixed
strategy intra-specific competition using a rock-paper-scissors game  results in a "survival of
the weakest". Another lesson is that evolutionary games are more than simply a matter of zero-
sum competition or stable cooperation . Despite the metaphor, evolutionary games are more
about capturing interactions than direct intentionality. However, contemporary models focus on
the role of natural selection in evolution. Yet due to their flexibility, evolutionary games could
also be used to model neutral processes and other contributors to evolutionary dynamics.
There are other lessons to be learned as well, including the linkages between micro- and
macroevolution and the evolution of sociality. Game-theory models can be combined with other
concepts at the intersection of economics and evolutionary biology to understand behavioral
signaling and other forms of informative communication. Examples include such as hedging
(managing trade-offs), biological markets , and handicapping . So is life just one big
game? According to game theory and the application of game-inspired models, the answer is
This month's Carnival is also available in printable form (on Figshare) for teaching
purposes. And don't forget to check out next month's Carnival of Evolution. Until then, enjoy
this month's posts. And remember, the game is not over until evolution has occurred.
NOTES:  von Neumann, J. and Morgenstern, O. Theory of Games and Economic Behavior. Princeton
Press (1947) AND Nash, J., Kuhn, J.W., Nasar, S. The Essential John Nash. Princeton Press
 Kerr, B., Riley, M.A., Feldman, M.W., and Bohannan, B.J.M. Local dispersal promotes
biodiversity in a real-life game of rock–paper–scissors. Nature, 418, 171-174 (2002).
 Turner, P.E. Cheating Viruses and Game Theory. American Scientist, 93(5), 428-435
 For more information, read the following paper: Richardson, J.L., Urban, M.C., Bolnick, D.I.,
and Skelly, D.K. Microgeographic adaptation and the spatial scale of evolution. Trends in
Ecology and Evolution, 29(3), 165-176 (2014).
 Nowak, M.A. and Sigmund, K. Evolution of Indirect Reciprocity. Nature, 437, 1291-1298
 Cowden, C.C. Game theory, evolutionary stable strategies, and the evolution of biological
interactions. Nature Education Knowledge, 3(10), 6 (2012).
 Rasmussen, E. Games and Information. Blackwell Publishing (2006).
 Weibull, J.W. Evolutionary Game Theory. MIT Press (1995) AND Brown, J.S. and
Vincent, T.L. Evolutionary Game Theory, Natural Selection, and Darwinian Dynamics.
Cambridge University Press (2005).
 Nowak, M.A. Evolutionary Dynamics: exploring the equations of life. Belknap Press
(2006) AND Broom, M. and Rychtar, J. Game-Theoretical Models in Biology. Chapman-Hall
CRC Press (2013).
 Maynard-Smith, J. and Price, G.R. The Logic of Animal Conflict. Nature 246 (5427): 15
 Brown, J.S. Fit of form and function, diversity of life, and procession of life as an
evolutionary game. In "Adaptationism and Optimality", S.H. Orzack and E. Sober eds., Chapter
 Vincent, T.L. and Brown, J.S. Evolution of ESS Theory. Annual Review of Ecology and
Systematics, 19, 423-443 AND Charlesworth, B. Optimization Models, Quantitative Genetics,
and Mutation. Evolution, 44(3), 520-538 (1990).
 Cohen, J. and Newman, C.E. Host-parasite relations and random zero-sum games: the
stabilizing effect of strategy diversification. American Naturalist, 133(4), 533-552 (1989) AND
Perc, M. and Szolnoki, A. Coevolutionary games: a mini review. Biosystems, 99, 109-125
 Sinervo, B. and Lively, C.M. The rock–paper–scissors game and the evolution of
alternative male strategies. Nature, 380, 240-243 (1996).
 Brembs, B. Evolution of Cooperation. Brembs.net Evolution section.
 Shutters, S.T. Punishment, Rational Expectations, and Relative Payoffs in a Networked
Prisoners Dilemma. In "Social Computing and Behavioral Modeling", H. Liu, J. Salerno, and
M.J. Young (eds.), pgs. 1-8 (2009).
 McNamara, J.M., Wilson, E.M.K., and Houston, A.I. Is it better to give information,
receive it, or be ignorant in a two-player game? Behavioral Ecology, 17(3), 441-451 (2006).
 Basar, T. and Olsder, G.J. Dynamic Noncooperative Game Theory. Academic Press
 Sigmund, K. Games of Life: explorations in ecology, evolution, and behavior. Oxford
University Press (1993) AND Wolfram, S. A New Kind of Science. Wolfram Press (2002).
 Heard, E. and Martienssen, R. Transgenerational Epigenetic Inheritance: myths and
mechanisms. Cell, 157(1), 95–109 (2014).
 Bendor, J. and Swistak, P. Types of evolutionary stability and the problem of cooperation.
PNAS, 92, 3596-3600 (1995).
 Noe, R. and Hammerstein, P. Biological Markets: supply and demand determine the effect
of partner choice in cooperation, mutualism, and mating. Behavioral Ecology and Sociobiology,
35(1), 1-11 (1994).
 Grafin, A. Biological Signals as Handicaps. Journal of Theoretical Biology, 144, 517-546