Games of multicellularity

Kamran Kaveh a,n, Carl Veller a,b,c, Martin A. Nowak a,b,ca Program for Evolutionary Dynamics, Harvard University, Cambridge, MA 02138, USAb Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138, USAc Department of Mathematics, Harvard University, Cambridge, MA 02138, USA

H I G H L I G H T S

! Simple multicellular organisms arise by cells staying together after division.! Staying together generates a particular population structure.! We study deterministic evolutionary dynamics in that population structure.! We derive conditions for natural selection to favor one strategy over another.! Simple multicellularity promotes cooperation among cells.

a r t i c l e i n f o

Article history:Received 16 June 2015Received in revised form21 April 2016Accepted 29 April 2016Available online 12 May 2016

Keywords:Evolution of multicellularityEvolutionary game theoryCooperationComplexity

a b s t r a c t

Evolutionary game dynamics are often studied in the context of different population structures. Here wepropose a new population structure that is inspired by simple multicellular life forms. In our model, cellsreproduce but can stay together after reproduction. They reach complexes of a certain size, n, beforeproducing single cells again. The cells within a complex derive payoff from an evolutionary game byinteracting with each other. The reproductive rate of cells is proportional to their payoff. We consider alltwo-strategy games. We study deterministic evolutionary dynamics with mutations, and derive exactconditions for selection to favor one strategy over another. Our main result has the same symmetry as thewell-known sigma condition, which has been proven for stochastic game dynamics and weak selection.For a maximum complex size of n¼2 our result holds for any intensity of selection. For ≥n 3 it holds forweak selection. As specific examples we study the prisoner's dilemma and hawk-dove games. Our modeladvances theoretical work on multicellularity by allowing for frequency-dependent interactions withingroups.

& 2016 Elsevier Ltd. All rights reserved.

1. Introduction

The emergence of multicellular life forms is an important stepin the evolutionary history of life on earth (Grosberg and Strath-mann, 2007; Bell and Mooers, 1997; Knoll, 2011; Bonner, 1998,2009a, 2009b; Rokas, 2008; Carroll, 2001; Rainey, 2007; Michod,1997, 1996; Michod and Roze, 2001; Hanschen et al., 2015).Multicellularity arose numerous times in prokaryotes, including incyanobacteria, actinomycetes, and myxobacteria (Grosberg andStrathmann, 2007; Bell and Mooers, 1997; Schirrmeisteret al., 2011). Complex multicellular organisms evolved in six eu-karyotic groups: animals, plants, fungi as well as brown, green andred algae.

A comparison between simple multicellular and their relativeunicellular organisms indicates multiple evolutionary transitions.These include increase in genetic complexity, cell differentiation,cell adhesion and cell-to-cell communication (Rokas, 2008). Divi-sion of labor, efficient dispersal, improved metabolic efficiency,and limiting interaction with non-cooperative individuals havebeen suggested as advantageous traits offered by multicellularity(Michod and Roze, 2001; Michod, 2007; Bonner, 1998; Pfeifferet al., 2001; Pfeiffer and Bonhoeffer, 2003; Kirk, 2005; Mora VanCauwelaert et al., 2015) (see also Grosberg and Strathmann, 2007and references therein.)

Multicellular organisms are usually formed by single cellswhose daughter cells stay together after division (Bonner, 1998;Koschwanez et al., 2011; Maliet et al., 2015; Rossetti et al., 2011). Incontrast, multicellular organisms via aggregation are formed byseparate cells coming together. Staying together and coming

Contents lists available at ScienceDirect

journal homepage: www.elsevier.com/locate/yjtbi

Journal of Theoretical Biology

http://dx.doi.org/10.1016/j.jtbi.2016.04.0370022-5193/& 2016 Elsevier Ltd. All rights reserved.

n Corresponding author.E-mail address: kkavehma@gmail.com (K. Kaveh).

Journal of Theoretical Biology 403 (2016) 143–158

www.sciencedirect.com/science/journal/00225193www.elsevier.com/locate/yjtbihttp://dx.doi.org/10.1016/j.jtbi.2016.04.037http://dx.doi.org/10.1016/j.jtbi.2016.04.037http://dx.doi.org/10.1016/j.jtbi.2016.04.037http://crossmark.crossref.org/dialog/?doi=10.1016/j.jtbi.2016.04.037&domain=pdfhttp://crossmark.crossref.org/dialog/?doi=10.1016/j.jtbi.2016.04.037&domain=pdfhttp://crossmark.crossref.org/dialog/?doi=10.1016/j.jtbi.2016.04.037&domain=pdfmailto:kkavehma@gmail.comhttp://dx.doi.org/10.1016/j.jtbi.2016.04.037

together lead to very different evolutionary dynamics (Tarnitaet al., 2013), and pose different challenges for the problem ofevolution of cooperation (Nowak, 2006b; Nowak et al., 2010a;Olejarz and Nowak, 2014). The same two modes for the evolutionof complexity are also observed in the context of eusocialityamong insects (Wilson, 1971; Gadagkar and Bonner, 1994; Ga-dagkar, 2001; Hunt, 2007). A common route to eusociality isdaughters staying with their mothers (Nowak et al., 2010b), butthere is also the coming together of different individuals in theformation of new colonies (Wilson, 1971; Gadagkar, 2001).

Here, we carry out a theoretical study of the dynamics under-lying the evolution of multicellularity. Previous studies of suchdynamics, both theoretical and experimental, have often beencarried out under the assumption that within-group fitnessesderive from a simple, additive cooperative dilemma. For example,cells producing ATP from an external energy resource might do sowith high yield but low rate, or with low yield but high rate(Pfeiffer et al., 2001). In the context of a group of cells trying tomake use of an energy resource, the former behavior characterizescooperators, and the latter defectors, because the benefits of a highrate of resource use accrue to the individual cell, while the costs ofinefficient resource use accrue more broadly within the group(Pfeiffer and Bonhoeffer, 2003). If the costs accrue equally to allgroup members, the strategic problem within the group can beconceptualized as an additive public goods game. Many othermodels of the evolution of multicellularity can be conceptualizedin the same way (Penn et al., 2012). For example, the aggregationof biofilms in Pseudomonas bacteria involves the production, costlyto individual providers, of the components of an extracellularmatrix and other substances (Davies and Geesey, 1995; Matsukawaand Greenberg, 2004; Diggle et al., 2006).

This assumption reduces the strategic conflicts within eachmulticellular unit to a very simple, frequency-independent form(Michod, 1999). Because a group's reproductive success is sharedequally among its constituents (no matter their type), the onlywithin-group conflict involves the constant cost to cooperation.

This is not realistic in many scenarios. In the example of ATPproduction described above, if the benefits of efficient resourceuse accrue more locally than to the whole group (for example, topairs of interacting cells within the group), then the strategic in-teractions among cells are more complicated than a linear publicgoods game (Fig. 1). Without taking this into account (i.e., as-suming that the benefits produced by cooperators are sharedevenly among group members), it would seem that defectorsshould always be at an advantage within the group. But once thestrategic complexity of local interactions is taken into account,then cooperators can have a within-group advantage if most oftheir interactions within the group are with fellow cooperators(Fig. 1).

Another example where strategic interaction within the groupis important is when certain cell types are preferentially found inthe reproductive propagules emitted by the group. Thus, in mul-ticellular clusters of the yeast Saccharomyces cerevisiae, experi-mentally selected for by gravity-based methods, some cells (co-operators) undergo apoptosis to destabilise the multicellular unitand create new propagules; having apoptosed, they cannotthemselves be in these propagules (Ratcliff et al., 2012; Pentz et al.,2015).

Another example involves cells that either aggressively orpassively try to sequester resources for themselves; if the presenceof many aggressive types involves a destructive cost to them, thenthe within-group conflict resembles a hawk-dove game. Becausethe within-group conflicts are frequency-independent in this ex-ample, their effects in the context of the evolution of multi-cellularity cannot be understood under a linear public goodsconceptualization.

To put it concisely, the evolution of multicellularity is oftenstudied in a framework that does not adequately account for theinteractions of cells within a group. In this paper, we place theevolution of multicellularity into an explicitly game-theoreticframework. Evolutionary game dynamics is the study of frequ-ency dependent selection (Maynard Smith, 1982; Hofbauer andSigmund, 1998; Nowak, 2006a). The success of a genotype (orphenotype or strategy) depends on the frequency of differentgenotypes in the population. Evolutionary game dynamics wasinitially studied in well-mixed and infinitely large populationsusing deterministic differential (Hofbauer and Sigmund, 1998;Maynard Smith, 1982; Weibull, 1997). More recently it has movedto finite population sizes using stochastic dynamics (Nowak,2006a; Taylor et al., 2004; Traulsen and Hauert, 2009). Evolu-tionary games are also studied in structured populations (Nowakand May, 1992; Page et al., 2000; Hauert and Doebeli, 2004;Ohtsuki et al., 2006; Szabó et al., 2000; Tarnita et al., 2009a,2009b; Hauert and Imhof, 2012; Langer et al., 2008; Antal et al.,2009b; Allen and Nowak, 2015; Cooney et al., 2016).

A game-theoretic approach to the evolution of multicellularityallows us to generalize the traditional framework by accountingfor frequency-dependent competition within multicellular units.

The primary goal of our paper is to understand how the po-pulation structure of simple multicellularity affects the outcome ofbiological games. Previous studies have explored the evolutionaryemergence of staying together (Tarnita et al., 2013) in the contextof diffusible public goods (Olejarz and Nowak, 2014) and in sto-chastic dynamics (Ghang and Nowak, 2014). Here we study de-terministic evolutionary dynamics in a population where stayingtogether has already evolved.

In our model, single cells divide, but the two daughter cells canstay together after cell division. These cells may undergo further

Fig. 1. Growth of a simple multicellular complex containing two competing me-tabolic phenotypes (Pfeiffer et al., 2001). The cooperator phenotype (blue) uses thelimited food resource to produce ATP with high efficiency but low rate; the defectorphenotype produces ATP with low efficiency but high rate. If the benefits of effi-cient resource use are shared equitably among the whole group, and the benefits ofa high rate of resource use are enjoyed by individual cells, then the within-groupconflict is a linear public goods game. In this case, the results of interactions withinthe group are frequency-independent, and defectors always grow as a proportion ofthe group. On the other hand, if the benefit of efficient resource use is shared morelocally, then within-group strategic interactions are more complex. Now, co-operators can increase as a proportion of the group if they typically interact withcooperators, which can occur, for example, if a viscous spatial structure governsinteractions (pictured). (For interpretation of the references to color in this figurecaption, the reader is referred to the web version of this paper.)

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158144

division until the complex reaches a specified maximum size.Thereafter, the complex does not grow further but produces sin-gle-cellular offspring, which subsequently form new complexes.Within a complex, cells interact according to a biological game.This means that they derive payoffs which affect their re-productive rate. We consider natural selection acting on two typesof cells (or strategies), determined by their genotype.

We include mutation between the two types, assumed tooccur during cell division. Each offspring adopts its parent's typewith probability − u1 and changes to the other type withprobability u. We shall be interested both in low rates of mu-tation (corresponding, for example, to nucleotide substitutions)and in very high rates of mutation (for example, geneticswitches, epigenetic marking, or structural mutations derivingfrom a modular genetic architecture – a fuller discussion ofthese is provided in the Discussion section). In the absence ofmutation, u¼0, one of the two types is bound to take over thewhole population (fixation). With mutation, <

= ( + )( − )= ( + )( − ) + ( + )= ( + )= ( + )= ( + ) + ( + )( − )= ( + )( − ) ( )

P wa u

P wb u wc u

P wd u

Q wa u

Q wb u wc u

Q wd u

2 1 1

1 1 1

2 1

2 1

1 1 1

2 1 1 2

00

01

11

00

01

11

The parameters a b c d, , , are the elements of the 2#2 payoffmatrix

( )⎛⎝⎜

⎞⎠⎟

a bc d

0 101 3

In each complex a type 0 cell obtains payoff a from anothertype 0 cell, and b from a type 1 cell. Similarly, a type 1 cell obtainspayoff c from a type 0 cell, and d from a type 1 cell. The gameinteraction occurs only between cells within the same complex.The intensity of selection is denoted byw and measures howmuchthe payoff of the game contributes to the fitness.

Note that the reproductive rate of a cell, which multiplies themutation rate, must always be non-negative. Therefore we require

+ { } ≥w a b c d1 min , , , 0. If some entries of the payoff matrix arenegative then these conditions limit the maximum intensity ofselection.

The average fitness, ϕ, is obtained from the constraintthat the relative abundances sum up to unity,

+ + ( + + ) =x x x x x2 10 1 00 01 11 . We haveϕ = ( + ) + ( + ) + ( + ) + + ( )P Q x P Q x P Q x x x 400 00 00 01 01 01 11 11 11 0 1

The equilibrium abundances are obtained by setting all timederivatives in Eq. (1) equal to zero. The solutions can be expressedin terms of ratio of type 1 to type 0 singlets η =⋆ ⋆ ⋆x x/1 0 and thevalue of average fitness at equilibrium, ϕ⋆:

ϕϕ η

ϕϕ

ηη

ϕ η

ϕ

ϕη

η

= + +

= + +

= −+ += +

= −+ + ( )

⋆⋆

⋆ ⋆

⋆⋆

⋆

⋆

⋆

⋆⋆ ⋆

⋆⋆

⋆⋆

⋆

⋆

x

x

x u

x u

x u

21

1

2 11

21

1

21

2 1 5

0

1

00

01

11

Eq. (5) and the values of ϕ⋆ and η⋆ in terms of game payoffs andmutation rate are derived in Appendix A (Eqs. (A.5)–(A.7)). Thetotal equilibrium abundances of type 0 and type 1 cells are

η ϕ ϕ η

η ϕ ϕ η η η

≡ + +

= + + + ( − ) + ( + )

≡ + +

= + + + ( − ) + ( + ) ( )

⋆ ⋆ ⋆ ⋆

⋆ ⋆⋆ ⋆

⋆ ⋆ ⋆ ⋆

⋆ ⋆⋆ ⋆ ⋆ ⋆

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

x x x x

u u

x x x x

u u

2

11

12

2 1 1

2

11

12

2 1 16

tot,0 0 00 01

tot,1 1 11 01

A strategy is favored by selection, if its equilibrium frequency isgreater than what it is in the neutral case. Here the neutralabundance of type 0 and type 1 is 1/2. Thus, the condition for type0 to be selected over type 1 is that the total number of type 0 cellsis larger than total number of type 1 cells at equilibrium

> ( )⋆ ⋆x x 7tot,0 tot,1

Substituting from Eq. (6), and using the fact that the average fit-ness is always positive, ϕ >⋆ 0, we arrive at the conditionη < ( )⋆ 1 8

Substituting for η⋆ from Eqs. (A.6) and (A.5), Eq. (8) becomes

− − + − − − < ( )⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟u

uu

a b c uu

d12

1 1 09

There are two zeros for the equality. We denote them u1 and u2.We have =u 1/22 and

= −− + − ( )ua d

a d c b 101

If u is outside the interval of ( )u u,1 2 (or ( )u u,2 1 if ^ + ^ + ^ ( )⋆ ⋆ ⋆ ⋆ ⋆ ⋆P x P x P x Q x Q x Q x 1400 00 01 01 11 11 00 00 01 01 11 11for ⋆ ⋆x x0 1condition as in Eq. (8). In the above discussion we have assumed apositive mutation rate >u 0. It was implied that the s-conditionholds for some mutation rate < a d. Asshown in Appendix B this is the condition for type 0 to be anevolutionary stable strategy (ESS) at weak selection. If

condition for risk dominance, + > +a b c d, and the condition forevolutionary stability is the same as the Pareto efficiency, >a d.

3. Examples

3.1. Cooperation

Consider the payoff matrix for a simplified game of cooperation

− −( )

⎛⎝⎜

⎞⎠⎟

C DCD 0 15

Here and indicate benefit and cost values respectively. Type0 cells denote the cooperator strategy, C. Type 1 cells denote thedefector strategy, D. Inside a complex, a type 0 cell pays a cost, ,and provides a benefit, , to the other cell. A type 1 cell pays nocost and provides no benefit.

At zero mutation and for > the dynamics is driven by purecomplexes, 00 and 11. Here cooperator complexes are advantagedover defector complexes. Mixed complexes, 01, are bound to be-come extinct. In this population structure, cooperators are evolu-tionarily stable. This is in contrast to evolution of cooperation inunstructured populations where defectors are stable.

For < − ( )u

u1

16

The critical mutation rate is

= − < =( )

⎛⎝⎜

⎞⎠⎟u u

12

1 1/

12 17

1 2

Values of ⋆xtot,0 and ⋆xtot,1 as functions of u are plotted in Fig. 3 forparameters = 5 and = 1. For these parameter values we ob-serve =u 0.41 and =u 0.52 , in agreement with Eq. (17). In Fig. 4frequencies of all populations (singlets and complexes) are plottedas a function of u for the same cost and benefit values and w¼0.1.We see that the condition >⋆ ⋆x xtot,0 tot,1 coincides with >⋆ ⋆x x0 1 inagreement with Eq. (8). Eq. (17) describes the phase boundary inthe space of mutation and benefit-to-cost ratio. This is depicted inFig. 5. The region between the curves ( )u , /1 and ( )u , /2 iswhere the defector strategy is selected. The two phase boundariesnever meet as = =u u 1/21 2 does not have a positive finite solutionfor / . In fact =u u2 is the vertical asymptote of the criticalbenefit-to-cost ratio as a function of u.

3.2. Hawk-dove

Now consider a hawk-dove game given by the payoff matrix

−

( )

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

H D

HD

2

02 18

Type 0 is hawk (H) and type 1 is dove (D). Inside 00 complexeseach hawk gains payoff /2 and pays the cost /2. In a 01 complex,a hawk gains payoff while a dove does not gain from the in-teraction. For 11 complexes, each dove gains /2. From Eq. (9) andfor − ( )u

u1

2 19

Therefore, we have

= + ( )u 1

1 2 / 201

As discussed before we expect the evolutionarily stable strategy(ESS) to dominate for small values of the mutation rate

3.3. Average fitness

We now study how the average fitness at equilibrium, ϕ⋆, de-pends on the mutation rate, u. The average fitness at equilibrium iscalculated in Appendix A, Eq. (A.4)

( )ϕ = − + + ( − ) + ( )⋆ D z D z1 4 1 2112 12 0 1Here η η≡ ( + ) = ( + )⋆ ⋆ ⋆ ⋆ ⋆z x x x/ 1 /1 0 1 is the fraction of type 1 sing-lets. Note that z is a function of u and of the payoff values. From Eq.(A.5), the coefficients D0 and D1 are

( ) ( )( ) ( )

= + + + −= + + + − ( )

D wa w b c a u

D wd w b c d u

2 1 2

2 1 2 22

0

1

If >a d then type 0 is ESS and as →u 0, we have →z 0. In thiscase z is an increasing function near u¼0 and therefore′( = ) >z u 0 0. If

= ′ + ( ′ − ′ ) + ( − ) ′

= ( + − ) + ( − ) − ( − )( − ) ′ ( )

Gu

D D D z D D z

w b c a w a d z w a d u z

dd

2 2 2 1 24

0 1 0 1 0

The prime symbol represents the derivative with respect to u. Atu¼0 we have ( = ) =z u 0 1 and ′( = ) ( )

=

=

Gu

w z u

Gu

w z u

dd

0 0

dd

1 1 025

u

u

0

1

The inequalities are true for > 0. Thus for the hawk-dove, ϕ ⋆0 andϕ ⋆1 are local maxima. The average fitness at u¼1 mutation is aglobal maximum. The signs of derivatives at Eq. (25) imply thatthere is a global minimum for ⋆u at < . We have also used( = ) = ′( = ) >z u z u0 0, 0 0 and ( = ) < ′( = )

solutions, −⋆xi k i, are given by

( )

∑

∑ϕ

ϕ

ϕ

ϕ

= +

= +

⋮= + + ×

+ ( < < ≤ ≤ )

⋮

= + ( ≤ ≤ )( )

⋆⋆

=− −

⋆

⋆⋆

=− −

⋆

−⋆ ⋆ − −

−

− − − −⋆

− − − −⋆

−⋆

⋆ − − − −⋆

− − − −⋆

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

x P x

x Q x

x P Q

P x Q x k n i k

x P x Q x i n

11

11

0 , 0

1 031

l

n

l n l l n l

l

n

l n l l n l

i k i i k i i k i

i k i i k i i k i i k i

i n i i n i i n i i n i i n i

1,00

, ,

0,10

, ,

, , ,1

1, 1, , 1 , 1

, 1, 1, , 1 , 1

To solve Eq. (31) analytically, we can, in principle, follow thesame method as used in Section 2 (and in Appendix A). Theabundances −⋆xi k i, can be expressed in terms of η ≡⋆ ⋆ ⋆x x/0,1 1,0 and theaverage fitness ϕ⋆. The system of equations (31) can be reduced totwo equations for η⋆ and ϕ⋆. We have presented a sketch of thismethod in Appendix C for n¼3. A general approach for arbitrary nis similar but the solutions become cumbersome as n increases.

At w¼0 the dynamics is neutral. In this limit we have ϕ =⋆ 1.Eq. (31) becomes a system of linear recurrence equations

∑

∑

^ = ^ ^

^ = ^ ^

⋮

^ = +^ ^ + ^ ^ ( ≤ <

≤ ≤ )

⋮

^ = ^ ^ + ^ ^ ( ≤ ≤ )( )

⋆

=− −

⋆

⋆

=− −

⋆

−⋆

− − − −⋆

− − − −⋆

−⋆

− − − −⋆

− − − −⋆

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

x P x

x Q x

xk

P x Q x k n

i k

x P x Q x i n

12

12

11

1 , 0

032

l

n

l n l l n l

l

n

l n l l n l

i k i i k i i k i i k i i k i

i n i i n i i n i i n i i n i

1,00

, ,

0,10

, ,

, 1, 1, , 1 , 1

, 1, 1, , 1 , 1

Here x̂ denotes the w¼0 limit. Similarly, ^ = ( − ) +P u i uj1i j, and^ = + ( − )Q ui u j1i j, are zero selection limits of Pi j, and Q i j, . Thesolutions x̂i j, have the following properties: (i) =⋆ ⋆x xi j j i, , . (ii) Thefrequencies of type 0 and type 1 cells are the same:∑ = ∑ =−⋆ −⋆ix ix 1/2i k i k i i k k i i, , , , . (iii) The total abundance of cells incomplexes of the same size, k, is independent of the mutation rate.

Fig. 7. A general model for games of multicellularity. The figure depicts some of the possible reactions and the corresponding rates for =n 4. The abundance of a complex ofsize k with i many type 0 cells is denoted by −xi k i, . The coefficients Pi j, denote the rates at which complexes with i type 0 cells and j type 1 cells generate type 0 offsprings.Similarly, Q i j, denote the rate at which the same complex generates type 1 cells. (See Eq. (30).)

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158150

5. The r-condition for ≥n 2

In this section we derive the s-condition for the generalizedmodel introduced in previous section at the weak selection limit.For n¼2, we showed that σ ( ) = ( − )u u u1 /2 . It turns out that for

>n 2 the parameter sn in Eq. (11), is now given by a ratio of twopolynomials of degree −n 1 in u. In fact the general form for σ ( )unis ( − ( )) ( )c h u h u/n n n . Here hn(u) is a polynomial in u, and cn is anumerical constant. The symmetry of the s-condition is preservedas we increase the maximum complex size. We will later comparenumerical solutions for strategy selection in the general model andanalytical prediction of s-condition. We observe that our predicteds-condition holds well above the weak selection limit as well.

We rewrite Eq. (31) in the following form

( )

( )

∑

∑

ϕ

ϕ

ϕ

ϕ

= −

= −

⋮

= +

− + ( ≤ < ≤ ≤ )

⋮

= + ( ≤ ≤ )( )

⋆⋆

=− −

⋆ ⋆

⋆⋆

=− −

⋆ ⋆

−⋆

⋆ − − − −⋆

− − − −⋆

− − −⋆

−⋆

⋆ − − − −⋆

− − − −⋆

⎪

⎪

⎪

⎪

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

⎧⎨⎩

⎫⎬⎭

⎛⎝⎜⎜

⎞⎠⎟⎟

x P x x

x Q x x

x P x Q x

P Q x k n i k

x P x Q x i n

1

1

1

1 , 0

1 033

l

n

l n l l n l

l

n

l n l l n l

i k i i k i k i i k i i k i

i k i i k i i k i

i n i i n i i n i i n i i n i

1,00

, , 1,0

0,10

, , 0,1

, 1, 1 1, , 1 , 1

, , ,

, 1, 1, , 1 , 1

Total number of type 0 and type 1 cells are written in terms ofsolutions of Eq. (33) as

∑ ∑

∑ ∑

= ·

= ( − )·( )

⋆

= =−

⋆

⋆

= =−

⋆

x i x

x k i x34

k

n

i

k

i k i

k

n

i

k

i k i

tot,01 0

,

tot,11 0

,

The condition for type 0 selection, Eq. (7), is >⋆ ⋆x xtot,0 tot,1. Tohave a first order estimate of abundances in powers of selectionintensity we substitute xi j, in left-hand side of Eq. (33) with w¼0

limit solutions, x̂i j, . This way −xi k i, from left-hand side of Eq. (33) isexpressed to zeroth and first order in w. Substituting solutions intoEqs. (34) and (7) gives rise to a closed form generalized s-condi-tion. After some straightforward algebra we get

∑ ∑ ∑ ∑^ > ^( )= =

− −⋆

= =− −

⋆P x Q x

35k

n

i

k

i k i i k ik

n

i

k

i k i i k i1 0

, ,1 0

, ,

This is a generalization of Eq. (14). We can write this in terms offitness gains of either of type 0 and 1 strategies as we did for n¼2case. Denoting fitness gains for type 0 and type 1 in weak selectionby δf n0, and δf n1, , respectively, we can write:

∑ ∑

∑ ∑

δ

δ

= ( − ) + ( − ) ·^

= ( − ) + ( − ) ·^( )

= =−

⋆

= =−

⋆

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

f i i a k i b x

f i i d k i c x

1

136

nk

n

i

k

i k i

nk

n

i

k

k i i

0,2 1

,

1,2 1

,

Then Eq. (35) can be rewritten as

δ δ− − < ( )⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟u f f

12

037n n0, 1,

The zeros of the term δ δ−f fn n0, 1, determine u1 whereas u2 is zeroof −u 1/2.

Let us consider n¼3 as an example. In this case Eq. (35) iswritten as

^ + ^ + ^ + ^ + ^ + ^

+ ^ + ^ + ^

> ^ + ^ + ^ + ^ + ^ + ^

+ ^ + ^ + ^ ( )

⋆ ⋆ ⋆ ⋆ ⋆ ⋆

⋆ ⋆ ⋆

⋆ ⋆ ⋆ ⋆ ⋆ ⋆

⋆ ⋆ ⋆

P x P x P x P x P x P x

P x P x P x

Q x Q x Q x Q x Q x Q x

Q x Q x Q x 38

3,0 3,0 2,1 2,1 1,2 1,2 0,3 0,3 2,0 2,0 1,1 1,1

0,2 0,2 1,0 1,0 0,1 0,1

3,0 3,0 2,1 2,1 1,2 1,2 0,3 0,3 0,2 0,2 1,1 1,1

2,0 2,0 1,0 1,0 0,1 0,1

The terms corresponding to singlets ^ ^x x,1,0 0,1 can be dropped fromboth sides of the inequality since they do not confer any fitnessgain or loss. Similarly Eq. (37) is

( )( )δ δ− − < ( )u f f 0 3912 0,3 1,3The fitness gains δf0,3 and δf1,3 are

( )( )

δ

δ

= ·^ + ·^ + ·^ + + ·^ + ·^

= ·^ + ·^ + ·^ + + ·^ + ·^ ( )

⋆ ⋆ ⋆ ⋆ ⋆

⋆ ⋆ ⋆ ⋆ ⋆

f a x b x a x a b x b x

f d x c x a x d c x c x

2 6 2 2

2 6 2 2 40

0,3 2,0 1,1 3,0 2,1 1,2

1,3 0,2 1,1 0,3 1,2 2,1

The n¼3 solutions for Eq. (32) are

^ = ^ =

^ = ^ = −

^ =

^ = ^ = ( − )

^ = ^ = ( − ) ( )

⋆ ⋆

⋆

⋆ ⋆

⋆ ⋆

x x

x x u

x u

x x u

x x u u

3221

22

111

11211 41

1,0 0,1

2,0, 3,0

1,1

3,0 0,3

2

2,1 1,2

Substituting the above results back into Eq. (40) we can writethe s-condition, Eq. (39), as

− ( − ) − +− + + ( − ) < ( )⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟u a d

u uu u

b c12

4 9 74 9

042

2

2

Now u1 is the solution of ( − )a d ( − + )u u4 9 72 ( − )u u/ 9 4 2+( − ) =b c 0 .

We have numerically solved the model for n¼3 and = 5 and= 1 for the cooperation game and the hawk-dove game. Pre-

dicted values for u1 and u2 are in excellent agreement for variousselection intensities (Fig. 8). The phase diagrams for these gamesare depicted in Fig. 9 for w¼0.1 and 0.3. The phase boundariesmatch very well with the results from Eq. (42).

The same method can be used for larger maximum complexsizes at weak selection. Substituting for ^ −xi k i, from solutions of Eq.(32) into Eq. (36) we can write

( ))

δδ

= − ( ) + ( )= ( ) + ( − ( ) ( )

f c h u a h u b

f h u c c h u d 43

n n n n

n n n n

0,

1,

From Eq. (36), the polynomial hn(u) and the constant cn are ex-pressed in terms of ^ −xi k i, solutions

∑ ∑

∑ ∑

( ) = ( − )^

= ( − )^( )

= =−

⋆

= =−

⋆

h u i k i x

c i k x144

nk

n

i

k

i k i

nk

n

i

k

i k i

1 0,

1 0,

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158 151

Using properties of Eq. (32) solutions, we can show that cn is infact a constant. For n¼2, we have ( ) =c u 12 and ( ) =h u u2 . For n¼3,from Eq. (42) we have =c 73 and ( ) = − +h u u u4 93 2 (up to aconstant common factor). Repeating the same steps for highercomplex sizes we have computed sn for = …n 2, , 20. To avoid longformulas, below we write sn for only = …n 2, , 7

( )

σ

σ

σ

σ

σ

σ

= −

= − +− +

= − + − +− +

= − + − +− + − +

= − + − + − +− + − +

= − + + − + − +− + − + − + 45

uu

u uu u

u u uu u u

u u u uu u u u

u u u u uu u u u u

u u u u u uu u u u u u

1

4 9 74 9

8 28 35 238 28 35

32 160 308 281 16332 160 308 281

64 432 1176 1640 1215 63964 432 1176 1640 1215

256 2240 8160 5968 17980 11439 5553256 2240 8160 15968 17980 11439

2

32

2

43 2

3 2

54 3 2

4 3 2

65 4 3 2

5 4 3 2

76 5 4 3 2

6 5 4 3 2

The results are plotted for = …n 2, , 20 in Fig. 10. As can be seenfrom above results, hn(u) always has a zero at u¼0. Also cn is al-ways a positive constant. The value of the critical mutation, u1, isobtained from the equality

σ ( ) = −− ( )uc ba d 46n 1

which can be solved for u1 using Eq. (45). These estimated valuesof u1 are compared with numerical solutions of the model forvarious n, for the game of cooperation (Table 1) and the hawk-dove game (Table 2). Numerical values of u1 are calculated for twoselection intensities, w¼0.01 and w¼0.1 up to n¼7 while theo-retical estimate of u1, Eq. (46), is presented up to n¼20.

We can now ask, for the general model, if the condition for evo-lutionary stability of a strategy is the same as in the case n¼2, whichis >a d. Proving a strategy is ESS by calculating the eigenvalues of theJacobian is a cumbersome procedure for arbitrary n. Instead we as-sume that limit of →u 0 of s-condition leads to type 0 being ESS. Weproved this in Appendix B for n¼2 in weak selection. For →u 0 and

≤ ≤n2 7, sn's from Eq. (45) have the limiting forms

σ

σ

σ

σ

σ

σ

( → ) = −

( → ) ∼ ( ) −

( → ) ∼ ( ) −

( → ) ∼ ( ) −

( → ) ∼ ( ) −

( → ) ∼ ( ) − ( )

u uu

u uu

u uu

u uu

u uu

u uu

0 1

0 7/9

0 23/35

0 163/281

0 639/1215

0 5553/11439 47

2

3

4

5

6

7

The leading term coefficients …1, 7/9, 23/35, are all positive and thusin the limit →u 0 lead to >a d.

Substituting the above results into Eq. (46), one can see that nomatter how large n is, the value of u1 always remains positive. Inother words, the phase boundary ( )u , /1 does not hit the y-axisfor some large-n. If type 0 is ESS for a given game at a small valueof n, it will remain ESS also for large n. Interestingly the plot ofsigma versus the mutation rate also represents part of the phaseboundary in the hawk- dove game. From Eq. (46) and substitutingpayoff values for the hawk-dove game, we have σ ( ) = ( )u 2 /1 . Forlarge-n, as can be seen from Fig. 10, the phase boundary seems tobe reaching an asymptote. We can confer from this observationthat for large complex sizes we still expect dove to be the domi-nant strategy.

6. Discussion

The evolution of multicellularity is often treated in a frame-work that distinguishes (‘decouples’) the fitness of an individualwithin a group and the overall fitness of the group, with thesequantities deriving from a simple cooperative dilemma such as alinear public goods game. While this conceptual framework hasbeen very useful in understanding many aspects of the transitionto multicellularity (Rainey and Kerr, 2010), it can be restrictive inmany cases. In particular, it often does not adequately account forthe interactions of cells within a group, which are likely to be

Fig. 8. Numerical solutions of ⋆xtot,0 and⋆xtot,1 for n¼3 for the cooperation game

(top) and the hawk-dove game (bottom). Total abundances of each strategy areplotted as a function of u. Parameters are = =5, 1 and =w 0.1, 0.2, 0.5 (thecurvature increases with w.) Notice that the main difference between n¼2 andn¼3 is the slight change in the value of u1. While for n¼2 the value of u1 wasindependent of the selection intensity, for n¼3 it is weakly dependent on w, butthis is almost impossible to see from these plots.

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158152

game-theoretic (i.e., frequency-dependent). One particularly clearexample is when some cell types are more likely than others to actas reproductive propagules. In the demonstration of Ratcliff et al.(2012) that some cells in multicellular yeast clusters undergoapoptosis (ostensibly to help break the group up into severalsmaller multi-cellular propagules), apoptosed cells obviouslycannot propagate further (Libby et al., 2014). If the likelihood of acell's apoptosing depends on its genotype, then a model that is notgame-theoretic cannot account for this important phenomenon.

Another example of explicitly game-theoretic interactionswithin multicellular groups is the competition between ‘co-operator’ and ‘cheater’ strains of yeast which use different glucosemetabolism (MacLean and Gudelj, 2006; Pfeiffer et al., 2001). Insuch setups, selection dynamics between different phenotypes

with different rates of ATP yield follow a prisoner's dilemma(MacLean and Gudelj, 2006). If staying together (and the con-comitant ability to form groups) has already evolved in the po-pulation, a model that accounts for this frequency dependence isrequired if we are adequately to model the evolutionary dynamicsof the population.

For these reasons, the construction of a more general, fre-quency-dependent framework is desirable. In the model devel-oped in this paper, cells divide and stay together until they reach acomplex of a certain size. Subsequent cell divisions lead to singlecells that leave and start their own complexes. The reproductive

Fig. 10. The value of s versus the mutation rate, u, is shown for various maximumcomplex sizes =n 2, 3, 4, 5, 6, 7, 10, 15, 20.

Table 1The critical mutation rate, u1, for the game of cooperation. Cooperators are favoredif

rate of a cell is determined by a game based on interactions of cellswithin a complex, and is therefore explicitly frequency-dependent.This game can represent competition for resources, exchange ofnutrients, cellular communication, or energy sharing mechanisms.

Studying the evolutionary dynamics of this setup, we de-termined how this population structure affects the outcome ofevolutionary games in the presence of mutation. In particular, weshowed that the condition for one strategy to be more abundantthan the other strategy in the mutation-selection equilibrium hasthe same symmetry as the s condition of stochastic evolutionarydynamics (Tarnita et al., 2009b). We calculated the value of s andexamined it for the game of cooperation and the hawk-dove game.

An important feature of our model is the suppression of thecheater phenotype during the evolution of multicellularity. Sincethe selection dynamics inside each complex is frequency-depen-dent (for example, a prisoner's dilemma), a cheater phenotype thatappears inside a cooperating colony has a lower chance of be-coming abundant inside the group. In other words, game-theoreticinteractions at all levels, individuals and groups, can lead to sup-pression of the cheater strategy without the need for evolution ofnew mechanisms of conflict mediation. This is the case when thecooperative phenotype is evolutionarily stable for small mutationrates. Our model also allows us to characterize the critical level ofgroup diversity, the mutation rate uc,1, above which a cheaterphenotype destabilizes cooperation inside the group.

We see our study primarily as a contribution toward under-standing how population structure affects evolutionary dynamics(Nowak et al., 2010a). We have quantified to what extent the po-pulation structure of simple multicellularity, perhaps as found atthe various origins of multicellularity, is conducive to favoringcooperation. Cooperation is thought to be crucially involved inevolutionary transitions such as the emergence of multicellularity(Maynard Smith and Szathmáry, 1997; Nowak, 2006a; Nowak andHighfield, 2011). Conversely, the somatic evolution of cancer isseen as a breakdown of cooperation among the cells of anorganism.

We have allowed throughout for the possibility of very highmutation rates. This is no mere theoretical fancy. While mutationsby nucleotide substitution and gene conversion are relatively rare,there are many other sources of frequent mutation.

Mutation in our model is perfectly consistent, for example, withepigenetically-induced heritable phenotype switching, such asthat observed in experimental populations of Pseudomonas bac-teria (Beaumont et al., 2009), a model organism in the field ofexperimental multicellularity (Rainey and Rainey, 2003; Nikolaevand Plakunov, 2007; Hammerschmidt et al., 2014). Epigeneticmutations in general, so long as they are heritable, are consistentwith our model, and are known in some cases to occur far morefrequently than sequence mutations (van der Graaf et al., 2015).

Another source of frequent mutation involves unstable geneticarchitectures. In Pseudomonas, for example, modularity of the ge-netic architecture underlying the group-forming phenotype allowsit to arise often and in multiple different ways (McDonald et al.,2009; Rainey and Kerr, 2010).

The directed gene transposition and epigenetic control under-lying mating-type switching in yeast (Klar and Fogel, 1979; Klar,1987, 2007), another model organism in experimental multi-cellularity (Koschwanez et al., 2011; Ratcliff et al., 2012), are alsoconsistent with our model. In fission yeast (Schizosaccharomycespombe), mating-type switching is more regular than mutation is inour model (Miyata and Miyata, 1981; Egel, 1984), but its highfrequency and heritability (Klar, 1987, 2007) suggest that geneticmechanisms of the sort underlying it could justify the considera-tion of very high mutation rates in our model.

These examples all suggest that high mutation rates might besufficiently common in organisms undergoing the transition to

multicellularity to justify the importance of high mutation rates inour model and its results. It is important to note that mutations inour model must be heritable; for example, non-heritable pheno-type switching and cell differentiation in response to environ-mental cues are ruled out.

Extensions of our model to include cases where staying togetheris stochastic, which means that cells can leave a given complex witha certain probability, more complicated life cycles, as well as asym-metric mutations are subjects of future works. While our currentmodel is deterministic, the extension to stochastic dynamics and fi-nite total population size should be straightforward.

Our model does not study important questions such as thedifferent implications of staying together versus coming together(Tarnita et al., 2013) or the evolution of germ line soma separation(Michod and Nedelcu, 2003; Michod and Roze, 2001) or the evo-lution of simple versus complex multicellularity (Knoll, 2011) forwhich we refer to the existing literature.

Acknowledgments

Support from the John Templeton Foundation is gratefullyacknowledged.

Appendix A. Exact solutions for n¼2

In this appendix we present derivation of equilibrium solutionsfor n¼2. Putting time derivatives to zero in Eq. (1), we have

ϕ

ϕ

ϕ

ϕ

ϕ

= + + +

= + + +

= ( − )

= ( + )

= ( − )( )

⋆⋆

⋆ ⋆ ⋆

⋆⋆

⋆ ⋆ ⋆

⋆⋆

⋆

⋆⋆ ⋆

⋆

⋆⋆

⋆

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

x P x P x P x

x Q x Q x Q x

xu x

xu x x

xu x

11

11

1

1A.1

0 00 00 01 01 11 11

1 00 00 01 01 11 11

000

010 1

111

Substituting for complex abundances from last three equationsinto first two equations in (A.1) and dividing them we obtain thefollowing relation for η =⋆ ⋆ ⋆x x/1 0

( ) ( )( ) ( )η

ηη

= ( − ) + + ( − ) +( − ) + + ( − ) + ( )

⋆⋆

⋆Q u Q u Q u Q u

P u P u P u P u1 11 1 A.2

00 01 11 01

00 01 11 01

Average fitness function at equilibrium, ϕ⋆, can be expressed interms of η⋆ as well

( ) ( )( )( ) ( )

ϕ ϕ

η

η

( ) +

= + + ( − ) + +

+ + ( − ) + +( )

⋆ ⋆

⋆

⋆

⎧⎨⎩⎫⎬⎭

P Q u P Q u

P Q u P Q u

11

1

1A.3

2

00 00 01 01

11 11 01 01

The above equations, (A.2) and (A.3), can be rewritten as quadraticequations

η η

ϕ ϕ η η

( ) + − =

( ) + − + ( + ) = ( )

⋆ ⋆

⋆ ⋆⋆

⋆

A B C

D D

01

10

A.4

2

20 1

Coefficients A B C D, , , 0 and D1 are expressed in terms of payoffcoefficients and mutation rate

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158154

( )

( )(

( )( )

( )( )

( )( )

( ) ( )≡ ( − ) += − + ( − − ) + ( + ) +≡ ( − )( − ) + ( − )= ( − + −

+ ( − ) + − ) + ( − )≡ ( − ) + = ( − − ) −

+ ( + )) +≡ ( + )( − ) + ( + )

= ( + )( − ) + + ( + )≡ ( + )( − ) + ( + )

= ( + )( − ) + + ( + ) ( )

A u P uP

w c b d u b d w u

B P Q u P Q u

w a b c d u

w d a b c u w a d

C Q u Q u b a c w u

a c w u

D P Q u P Q u

wa u w b c u

D P Q u P Q u

wd u w b c u

1

2 2 2 3

1

2

4 2

1 2 2

2 3

1

2 1 1 2

1

2 1 1 2 A.5

11 01

00 11 01 01

2

00 012

0 00 00 01 01

1 11 11 01 01

Values of η⋆ and ϕ⋆ are thus given by

η

ϕ

ηη

= − + +

=− + + ( + )+

( )

⋆

⋆

⋆

⋆

B B ACA

D D

42

1 1 41

2 A.6

2

0 1

Coefficients A C D, , 0 and D1 in Eq. (A.5) are positive sincecoefficients P P,00 01, P11 and Q Q Q, ,00 01 11 are positive for < / 1 for the game of cooperation. As canbe seen at value = , λ( )i ii1

. changes sign. For the hawk-dove game however, dovestrategy is always ESS at =u 0.

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158 155

δ

δ

δ

δ

= ⟶ −

= ⟶

= ⟶ −

= ⟶

= ⟶ ( )

⋆ ⋆

⋆ ⋆

x x x x

x x

x x x x

x

x x

3

03

30 0

03 B.3

0 0 0

1

00 00 00

01

11

This corresponds to introducing a small fraction δx of type 1 cellsto the system: δ→x xtot,1 , δ→ −x x1tot,0 . Substituting into Eq. (1)(or Eq. (B.1)) and keeping terms up to lowest order in w we obtainlinearized equation around the fixed point( = = = )⋆ ⋆ ⋆ ⋆ ⋆x x x x x, 0, , 0, 00 1 00 01 11

δ δ≈ − ( − ) + ( ) ( )xt

w a d x wdd

23 B.4

2

which indicates corresponding eigenvalue λ ≈ − ( )( − )( ) w a d2 /3i1 .For >a d and independent of off-diagonal payoff coefficients, band c, a fixed point of ( )⋆ ⋆x x, 0, , 00 00 is ESS. Similar condition can beobtained by linearizing time operator around type 1 fixed point,( )⋆ ⋆x x0, , 0,1 11 . This leads to eigenvalue, λ ≈ ( )( − )( ) w a d2 /3ii1 ((ii)denotes type 1 fixed point). Thus for the Jacobian matrix, Jij, de-fined as

= ∂∂→ = ( )

( )→=→⋆J F

xx x x x x, , , ,

B.5ij

i

j x x0 1 00 11

all the eigenvalues can be calculated to the leading order of se-lection intensity. For type 0 fixed point we obtain

( ) ( )( )

( ) ( )( ) ( )

λ

λ

λ

λ

≈ − − +

≈ − − +

≈ − + − + | − | +

≈ − + − − | − | + ( )

( )

( )

( )

( )

a d w w

aw w

a d a d w w

a d a d w w

1

3

3 B.6

i

i

i

i

123

2

223

2

313

2

413

2

We numerically calculated all the eigenvalues around bothfixed-points at =u 0. For =w 0.01 results matched very well withEq. (B.6). For larger intensities of selection there are deviationsfrom Eq. (B.6). However it seems that the ESS condition, >a d, stillholds away from weak selection as well. Fig. B1 shows numericalresults for eigenvalues around both fixed points at =u 0 for thegame of cooperation and the hawk-dove game. Eigenvalues areplotted as a function of / and for =w 0.5. For the game of co-operation as the benefit to cost ratio passes unity, =/ 1, theeigenvalue λ( )i1 switches sign thus indicting the type 0 is evolu-tionary stable for > .

For >n 2 the above analysis can be tedious but in principle thesame steps can be done. The same result, however, can be in-tuitively understood. For finite u there is a single equilibrium statethat is globally attractive inside the multi-dimensional simplex ofstates. As →u 0 this fixed point moves approaches to the ESS fixedpoint among the two fixed point for =u 0. This is indicated by thes-condition. The strategy that is favored by s-condition as →u 0 isthe ESS.

Appendix C. Exact solutions for n¼3

Here we present a sketch of derivation of exact solutions formodel with maximum three-cell complexes. The equilibriumabundances for n¼3 satisfy the coupled system of equation:

ϕ

ϕ

ϕ

ϕ

ϕ

ϕ

ϕ

ϕ

ϕ

= + + + +

= + + + +

= + +

= + + +

= + +

=

= +

= +

=( )

⋆⋆

⋆ ⋆ ⋆ ⋆

⋆⋆

⋆ ⋆ ⋆ ⋆

⋆⋆

⋆

⋆⋆

⋆ ⋆

⋆⋆

⋆

⋆⋆

⋆

⋆⋆

⋆ ⋆

⋆⋆

⋆ ⋆

⋆⋆

⋆

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

x P x P x P x P x

x Q x Q x Q x Q x

xP Q

P x

xP Q

P x Q x

xP Q

Q x

x P x

x P x Q x

x P x Q x

x Q x

11

11

1

1

1

1

1

1

1C.1

1,0 3,0 3,0 2,1 2,1 1,2 1,2 0,3 0,3

0,1 3,0 3,0 2,1 2,1 1,2 1,2 0,3 0,3

2,02,0 2,0

1,0 1,0

1,11,1 1,1

0,1 0,1 1,0 1,0

0,20,2 0,2

0,1 0,1

3,0 2,0 2,0

2,1 1,1 1,1 2,0 2,0

1,2 0,2 0,2 1,1 1,1

0,3 0,2 0,2

Coefficients Pi j, and Q i j, are given by Eq. (30). We also have= = − = =P Q u P Q u1 ,1,0 0,1 0,1 1,0 . Similar to n¼2 case, solutions

−⋆xi i,3 can be expressed in terms of ratio of singlets η =⋆ ⋆ ⋆x x/1 0 and

total fitness ϕ⋆. Upon dividing the first two equations in Eq. (C.1)and substituting for abundances from rest of the equations weobtain a quadratic equation for η⋆

η ϕ

ϕ η ϕ

ϕ η ϕ η

ϕ η

ϕ

ϕ η ϕ

ϕ η ϕ η

ϕ η

= + + ( − )

+ + + ( + )+( − )

+ +

+ ( − )+ + + + + ( + )

+ ( − )+ +

+ + ( − )

+ + + ( + ) +( − )

+ +

+ ( − )+ + + + + ( + )

+ ( − )+ + ( )

⋆⋆

⋆⋆

⋆

⋆⋆

⋆⋆

⋆⋆

⋆

⋆⋆

⋆

⋆⋆

⋆⋆

⋆⋆

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎧⎨⎩

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜

⎞⎠⎟

⎫⎬⎭⎧⎨⎩

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜⎜

⎞⎠⎟⎟

⎛⎝⎜

⎞⎠⎟

⎫⎬⎭

QP

P Qu

Q P uP Q

Q uP Q

QP u

P QQ uP Q

QQ u

P Q

PP

P Qu

P P uP Q

Q uP Q

PP u

P QQ uP Q

PQ u

P Q

1

11

11

1

1

11

11

1

C.2

3,02,0

2,0 2,0

2,1 1,11,1 1,1

2,0

2,0 2,0

1,20,2

0,2 0,2

1,1

1,1 1,1

0,30,2

0,2 0,2

3,02,0

2,0 2,0

2,1 1,11,1 1,1

2,0

2,0 2,0

1,20,2

0,2 0,2

1,1

1,1 1,1

0,30,2

0,2 0,2

singlet and complex abundances, up to the common factor,+⋆ ⋆x x0,1 1,0, are expressed in terms of η⋆ and ϕ⋆ from Eq. (C.1),

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158156

ϕ η

ϕ

ϕη

η

ϕ ϕ η

ϕ ϕ ϕη

η

ϕ ϕη

η ϕ

ϕ ϕη

η

= −+ + + +

= + + +

= −+ + + +

= −+ + + +

= + + +−

+ + +

+

= −+ + + + + +

+

= −+ + + + ( )

⋆⋆ ⋆

⋆ ⋆

⋆⋆

⋆ ⋆

⋆⋆

⋆

⋆⋆ ⋆

⋆⋆ ⋆ ⋆

⋆ ⋆

⋆⋆ ⋆ ⋆

⋆

⋆

⋆ ⋆

⋆⋆ ⋆

⋆

⋆ ⋆

⋆ ⋆

⋆⋆ ⋆

⋆

⋆⋆ ⋆

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

x uP Q

x x

x uP Q

x x

x uP Q

x x

x P uP Q

x x

x P uP Q

Q uP Q

x x

x P uP Q

Q uP Q

x x

x Q uP Q

x x

1 11

11

1 1 11

1 11

1 11

1 11 C.3

2,02,0 2,0

1,0 0,1

1,11,1 1,1

1,0 0,1

0,20,2 0,2

1,0 0,1

3,0 2,02,0 2,0

1,0 0,1

2,1 1,11,1 1,1

2,02,0 2,0

1,0 0,1

1,2 0,20,2 0,2

1,11,1 1,1

1,0 0,1

0,3 0,20,2 0,2

1,0 0,1

Substituting these into condition +⋆ ⋆x x1,0 0,1( ) )+ + + + ( + + + =⋆ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆x x x x x x x2 3 12,0 1,1 0,2 3,0 2,1 1,2 0,3 , we obtain+⋆ ⋆x x1,0 0,1 in terms of η⋆ and ϕ⋆

( )

( )

( )( )

ϕ ϕ η

ϕ ϕ

ϕ ηϕ

ϕ ηϕ

+ = + +

× ( + + −+ +

+ ( + + ( + )+ +

+ ( + + ( − )+ +

⋆ ⋆ ⋆ ⋆⋆

⋆⋆

⋆⋆

⋆

⋆⋆

⋆⎪

⎪

⎪⎪⎧⎨⎩

⎡⎣⎢⎢

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎤⎦⎥

⎛⎝⎜

⎞⎠⎟

⎞⎠⎟⎟

⎞⎠⎟⎟

⎫⎬⎭ C.4

x x

P Q uP Q

P Qu

P Q

P Qu

P Q

11

3 2 1

3 2 1

3 2 1

1,0 0,1

2,0 2,02,0 2,0

1,1 1,11,1 1,1

0,2 0,20,2 0,2

ϕ⋆ is obtained from substituting above solutions, Eqs. (C.3) and(C.4) into

( ) ( ) ( )( ) ( ) ( )( )

ϕ = + + + + ++ + + + + ++ + + + ( )

⋆ ⋆ ⋆ ⋆

⋆ ⋆ ⋆

⋆ ⋆ ⋆

P Q x P Q x P Q x

P Q x P Q x P Q x

P Q x x x C.5

3,0 3,0 3,0 2,1 2,1 2,1 1,2 1,2 1,2

0,3 0,3 0,3 2,0 2,0 2,0 1,1 1,1 1,1

0,2 0,2 0,2 1,0 0,1

Eqs. (C.2) and (C.5) combined with Eq. (C.4) can be solved toobtain closed form solutions for ϕ⋆ and η⋆, which upon sub-stituting into Eq. (C.3) gives all abundances in terms of payoffvalues and mutation. This approach in principle is generalized ton-cell complex model as well. We always get a quadratic equationin η⋆ coupled with a degree-n equation for ϕ⋆. The condition forthe strategy selection, however, can be answered in weak selectionlimit without exact knowledge of individual abundance of differ-ent complexes as discussed in Section 5.

Appendix D. Well-mixed games with mutation

In this appendix we briefly review unstructured game in thepresence of mutations and compare the condition for neutralitywith the one we obtained for game of multicellularity, Eq. (1).Derivations for finite populations are done in the literature (Tar-nita et al., 2009b; Traulsen et al., 2008; Antal et al., 2009a). Con-sider a well-mixed population of two populations 0 and 1 withabundances x0 and x1. Each individual can replicate based on itspayoff values determined by the game and the offspring can

mutate with probability u. The dynamics for two populations iswritten as,

( ) ( )( ) ( )

ϕϕ

̇ = + ( − ) + + − −̇ = + + + ( − ) − − ( )

x x P x P x u x Q x Q x u x x

x x P x P x u x Q x Q x u x x

1

1 D.1

0 0 0 0 1 1 1 0 0 1 1 0 0

1 0 0 0 1 1 1 0 0 1 1 1 1

where fitness functions, P Q,0,1 0,1, are given in terms of payoffcoefficients a b c d, , , , and intensity of selection w

= ( + )= ( + )= ( + )= ( + ) ( )

P wa

P wb

Q wc

Q wd

1

1

1

1 D.2

0

1

0

1

ϕis given by

( ) ( )ϕ = + + + − ( )x P x P x x Q x Q x 1 D.30 0 0 1 1 1 0 0 1 1Enforcing the condition that total frequencies add up to unity,

+ =x x 10 1 . Calling = ( )x x t0 , we have

( ) = ( − − )( + ( − ))

+ ( − )( − )( + ( − )) ( )t

x t x u x P x P x

x u x Q x Q x

dd

1 1

1 1 , D.4

0 1

0 1

Putting LHS of the above equation to zero for equilibrium solutions( )⋆x and assuming type 0 is selected, i.e., >⋆x 1/2, after somestraightforward algebra we get

+ > + ( )a b c d D.5

which is the s-condition with σ = 1. Notice that this relationship isindependent of u.

References

Allen, B., Nowak, M.A., 2015. Games among relatives revisited. J. Theor. Biol. 378,103–116.

Allen, B., Tarnita, C.E., 2014. Measures of success in a class of evolutionary modelswith fixed population size and structure. J. Math. Biol. 68 (1–2), 109–143.

Allen, B., Traulsen, A., Tarnita, C.E., Nowak, M.A., 2012. How mutation affects evo-lutionary games on graphs. J. Theor. Biol. 299, 97–105.

Antal, T., Nowak, M.A., Traulsen, A., 2009a. Strategy abundance in 2#2 games forarbitrary mutation rates. J. Theor. Biol. 257 (2), 340–344.

Antal, T., Ohtsuki, H., Wakeley, J., Taylor, P.D., Nowak, M.A., 2009b. Evolution ofcooperation by phenotypic similarity. Proc. Natl. Acad. Sci. 106 (21), 8597–8600.

Beaumont, H.J., Gallie, J., Kost, C., Ferguson, G.C., Rainey, P.B., 2009. Experimentalevolution of bet hedging. Nature 462 (7269), 90–93.

Bell, G., Mooers, A.O., 1997. Size and complexity among multicellular organisms.Biol. J. Linn. Soc. 60 (3), 345–363.

Bonner, J.T., 1998. The origins of multicellularity. Integr. Biol. Issues News Rev. 1 (1),27–36.

Bonner, J.T., 2009a. First Signals: The Evolution of Multicellular Development.Princeton University Press, Princeton.

Bonner, J.T., 2009b. The Social Amoebae: The Biology of Cellular Slime Molds.Princeton University Press, Princeton.

Carroll, S.B., 2001. Chance and necessity: the evolution of morphological com-plexity and diversity. Nature 409 (6823), 1102–1109.

Cooney, D., Allen, B., Veller, C., 2016. Assortment and the evolution of cooperation ina Moran process with exponential fitness, arXiv preprint arxiv:1509.05757.

Davies, D.G., Geesey, G.G., 1995. Regulation of the alginate biosynthesis gene algC inPseudomonas aeruginosa during biofilm development in continuous culture.Appl. Environ. Microbiol. 61 (3), 860–867.

Diggle, S.P., Stacey, R.E., Dodd, C., Cámara, M., Williams, P., Winzer, K., 2006. Thegalactophilic lectin, LecA, contributes to biofilm development in Pseudomonasaeruginosa. Environ. Microbiol. 8 (6), 1095–1104.

Egel, R., 1984. The pedigree pattern of mating-type switching in Schizosacchar-omyces pombe. Curr. Genet. 8 (3), 205–210.

Gadagkar, R., 2001. The Social Biology of Ropalidia marginata: Toward Under-standing the Evolution of Eusociality. Harvard University Press, Cambridge.

Gadagkar, R., Bonner, J.T., 1994. Social insects and social amoebae. J. Biosci. 19 (2),219–245.

Ghang, W., Nowak, M.A., 2014. Stochastic evolution of staying together. J. Theor.Biol. 360, 129–136.

van der Graaf, A., Wardenaar, R., Neumann, D.A., Taudt, A., Shaw, R.G., Jansen, R.C.,Schmitz, F., Colomé-Tatché, M., Johannes, F., 2015. Rate, spectrum, and

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158 157

http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref1http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref1http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref1http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref2http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref2http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref2http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref3http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref3http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref3http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref4http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref4http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref4http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref4http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref5http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref5http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref5http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref6http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref6http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref6http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref7http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref7http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref7http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref8http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref8http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref8http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref9http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref9http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref10http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref10http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref11http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref11http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref11http://arxiv:1509.05757/http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref13http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref13http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref13http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref13http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref14http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref14http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref14http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref14http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref15http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref15http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref15http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref16http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref16http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref17http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref17http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref17http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref18http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref18http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref18http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref19http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref19

evolutionary dynamics of spontaneous epimutations. Proc. Natl. Acad. Sci. U. S.A. 112 (21), 6676–6681.

Grosberg, R.K., Strathmann, R.R., 2007. The evolution of multicellularity: a minormajor transition?. Annu. Rev. Ecol. Evol. Syst., 621–654.

Hammerschmidt, K., Rose, C.J., Kerr, B., Rainey, P.B., 2014. Life cycles, fitness de-coupling and the evolution of multicellularity. Nature 515 (7525), 75–79.

Hanschen, E.R., Shelton, D.E., Michod, R.E., 2015. Evolutionary transitions in in-dividuality and recent models of multicellularity, Evolutionary Transitions toMulticellular Life. Springer, pp. 165–188.

Harsanyi, J.C., Selten, R., 1988. A General Theory of Equilibrium Selection in Games.MIT Press Books, Cambridge.

Hauert, C., Doebeli, M., 2004. Spatial structure often inhibits the evolution of co-operation in the snowdrift game. Nature 428 (6983), 643–646.

Hauert, C., Imhof, L.A., 2012. Evolutionary games in deme structured, finite popu-lations. J. Theor. Biol. 299, 106–112.

Hofbauer, J., Sigmund, K., 1998. Evolutionary Games and Population Dynamics.Cambridge University Press, Cambridge.

Hunt, J.H., 2007. The Evolution of Social Wasps. Oxford University Press, New York.Kirk, D.L., 2005. A twelve-step program for evolving multicellularity and a division

of labor. BioEssays 27 (3), 299–310.Klar, A.J., 1987. Differentiated parental DNA strands confer developmental asym-

metry on daughter cells in fission yeast. Nature 326 (6112), 466–470.Klar, A.J., 2007. Lessons learned from studies of fission yeast mating-type switching

and silencing. Annu. Rev. Genet. 41, 213–236.Klar, A.J., Fogel, S., 1979. Activation of mating type genes by transposition in Sac-

charomyces cerevisiae. Proc. Natl. Acad. Sci. U. S. A. 76 (9), 4539–4543.Knoll, A.H., 2011. The multiple origins of complex multicellularity. Annu. Rev. Earth

Planet. Sci. 39, 217–239.Koschwanez, J.H., Foster, K.R., Murray, A.W., 2011. Sucrose utilization in budding

yeast as a model for the origin of undifferentiated multicellularity. PLoS Biol. 9(8), e1001 122.

Langer, P., Nowak, M.A., Hauert, C., 2008. Spatial invasion of cooperation. J. Theor.Biol. 250 (4), 634–641.

Libby, E., Ratcliff, W.C., et al., 2014. Ratcheting the evolution of multicellularity.Science 346 (6208), 426–427.

MacLean, R.C., Gudelj, I., 2006. Resource competition and social conflict in experi-mental populations of yeast. Nature 441 (7092), 498–501.

Maliet, O., Shelton, D.E., Michod, R.E., 2015. A model for the origin of group re-production during the evolutionary transition to multicellularity. Biol. Lett. 11(6), 20150 157.

Matsukawa, M., Greenberg, E.P., 2004. Putative exopolysaccharide synthesis genesinfluence Pseudomonas aeruginosa biofilm development. J. Bacteriol. 186 (14),4449–4456.

Maynard Smith, J., 1982. Evolution and the Theory of Games. Cambridge UniversityPress, Cambridge.

Maynard Smith, J., Szathmáry, E., 1997. The Major Transitions in Evolution. OxfordUniversity Press, Oxford, UK.

McAvoy, A., Hauert, C., 2015. Structure coefficients and strategy selection in mul-tiplayer games. J. Math. Biol., 1–36.

McDonald, M.J., Gehrig, S.M., Meintjes, P.L., Zhang, X.X., Rainey, P.B., 2009. Adaptivedivergence in experimental populations of Pseudomonas fluorescens. IV. Ge-netic constraints guide evolutionary trajectories in a parallel adaptive radiation.Genetics 183 (3), 1041–1053.

Michod, R.E., 1996. Cooperation and conflict in the evolution of individuality. II.Conflict mediation. Proc. R. Soc. Lond. B: Biol. Sci. 263 (1372), 813–822.

Michod, R.E., 1997. Cooperation and conflict in the evolution of individuality. I.Multilevel selection of the organism. Am. Nat., 607–645.

Michod, R.E., 1999. Darwinian Dynamics: Evolutionary Transitions in Fitness andIndividuality. Princeton University Press, Princeton.

Michod, R.E., 2007. Evolution of individuality during the transition from unicellularto multicellular life. Proc. Natl. Acad. Sci. 104 (Suppl 1), 8613–8618.

Michod, R.E., Nedelcu, A.M., 2003. On the reorganization of fitness during evolu-tionary transitions in individuality. Integr. Comp. Biol. 43 (1), 64–73.

Michod, R.E., Roze, D., 2001. Cooperation and conflict in the evolution of multi-cellularity. Heredity 86 (1), 1–7.

Miyata, H., Miyata, M., 1981. Mode of conjugation in homothallic cells of Schizo-saccharomyces pombe. J. Gen. Appl. Microbiol. 27 (5), 365–371.

Mora Van Cauwelaert, E., Del Angel, A., Antonio, J., Benítez, M., Azpeitia, E.M., 2015.Development of cell differentiation in the transition to multicellularity: a dy-namical modeling approach. Front. Microbiol. 6, 559.

Nathanson, C.G., Tarnita, C.E., Nowak, M.A., 2009. Calculating evolutionary dy-namics in structured populations. PLoS Comput. Biol. 5 (12), e1000 615.

Nikolaev, Y.A., Plakunov, V.K., 2007. Biofilm–“City of microbes” or an analogue ofmulticellular organisms? Microbiology 76 (2), 125–138.

Nowak, M., Highfield, R., 2011. SuperCooperators: Altruism, Evolution, and Why WeNeed Each Other to Succeed. Simon and Schuster, New York.

Nowak, M.A., 2006a. Evolutionary Dynamics. Harvard University Press, Cambridge.Nowak, M.A., 2006b. Five rules for the evolution of cooperation. Science 314 (5805),

1560–1563.Nowak, M.A., May, R.M., 1992. Evolutionary games and spatial chaos. Nature 359

(6398), 826–829.Nowak, M.A., Tarnita, C.E., Antal, T., 2010a. Evolutionary dynamics in structured

populations. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 365 (1537), 19–30.Nowak, M.A., Tarnita, C.E., Wilson, E.O., 2010b. The evolution of eusociality. Nature

466 (7310), 1057–1062.Ohtsuki, H., Hauert, C., Lieberman, E., Nowak, M.A., 2006. A simple rule for the

evolution of cooperation on graphs and social networks. Nature 441 (7092),502–505.

Olejarz, J.W., Nowak, M.A., 2014. Evolution of staying together in the context ofdiffusible public goods. J. Theor. Biol. 360, 1–12.

Page, K.M., Nowak, M.A., Sigmund, K., 2000. The spatial ultimatum game. Proc. R.Soc. Lond. Ser. B: Biol. Sci. 267 (1458), 2177–2182.

Penn, A.S., Conibear, T.C.R., Watson, R.A., Kraaijeveld, A.R., Webb, J.S., 2012. CanSimpson's paradox explain co-operation in Pseudomonas aeruginosa biofilms?FEMS Immunol. Med. Microbiol. 65 (2), 226–235.

Pentz, J., Taylor, B.P., Ratcliff, W.C., 2015. Apoptosis in snowflake yeast: novel trait, orside effect of toxic waste? bioRxiv, p 029918.

Pfeiffer, T., Bonhoeffer, S., 2003. An evolutionary scenario for the transition to un-differentiated multicellularity. Proc. Natl. Acad. Sci. 100 (3), 1095–1098.

Pfeiffer, T., Schuster, S., Bonhoeffer, S., 2001. Cooperation and competition in theevolution of ATP-producing pathways. Science 292 (5516), 504–507.

Rainey, P.B., 2007. Unity from conflict. Nature 446 (7136) 616–616.Rainey, P.B., Kerr, B., 2010. Cheats as first propagules: a new hypothesis for the

evolution of individuality during the transition from single cells to multi-cellularity. Bioessays 32 (10), 872–880.

Rainey, P.B., Rainey, K., 2003. Evolution of cooperation and conflict in experimentalbacterial populations. Nature 425 (6953), 72–74.

Ratcliff, W.C., Denison, R.F., Borrello, M., Travisano, M., 2012. Experimental evolu-tion of multicellularity. Proc. Natl. Acad. Sci. 109 (5), 1595–1600.

Rokas, A., 2008. The origins of multicellularity and the early history of the genetictoolkit for animal development. Annu. Rev. Genet. 42, 235–251.

Rossetti, V., Filippini, M., Svercel, M., Barbour, A., Bagheri, H.C., 2011. Emergentmulticellular life cycles in filamentous bacteria owing to density-dependentpopulation dynamics. J. R. Soc. Interface 8 (65), 1772–1784.

Schirrmeister, B.E., Antonelli, A., Bagheri, H.C., 2011. The origin of multicellularity incyanobacteria. BMC Evol. Biol. 11 (1), 45.

Szabó, G., Antal, T., Szabó, P., Droz, M., 2000. Spatial evolutionary prisoner's di-lemma game with three strategies and external constraints. Phys. Rev. E 62 (1),1095.

Tarnita, C.E., Antal, T., Ohtsuki, H., Nowak, M.A., 2009a. Evolutionary dynamics inset structured populations. Proc. Natl. Acad. Sci. 106 (21), 8601–8604.

Tarnita, C.E., Ohtsuki, H., Antal, T., Fu, F., Nowak, M.A., 2009b. Strategy selection instructured populations. J. Theor. Biol. 259 (3), 570–581.

Tarnita, C.E., Wage, N., Nowak, M.A., 2011. Multiple strategies in structured popu-lations. Proc. Natl. Acad. Sci. 108 (6), 2334–2337.

Tarnita, C.E., Taubes, C.H., Nowak, M.A., 2013. Evolutionary construction by stayingtogether and coming together. J. Theor. Biol. 320, 10–22.

Taylor, C., Fudenberg, D., Sasaki, A., Nowak, M.A., 2004. Evolutionary game dy-namics in finite populations. Bull. Math. Biol. 66 (6), 1621–1644.

Traulsen, A., Hauert, C., 2009. Stochastic evolutionary game dynamics. Rev. Non-linear Dyn. Complex. 2, 25–61.

Traulsen, A., Shoresh, N., Nowak, M.A., 2008. Analytical results for individual andgroup selection of any intensity. Bull. Math. Biol. 70 (5), 1410–1424.

Weibull, J.W., 1997. Evolutionary Game Theory. MIT Press, Cambridge.Wilson, E.O., 1971. The Insect Societies. Harvard University Press (Distributed by

Oxford University Press), Cambridge, Massachusetts, USA.

K. Kaveh et al. / Journal of Theoretical Biology 403 (2016) 143–158158

http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref19http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref19http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref19http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref20http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref20http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref20http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref21http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref21http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref21http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref22http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref22http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref22http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref22http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref23http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref23http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref24http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref24http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref24http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref25http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref25http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref25http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref26http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref26http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref27http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref28http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref28http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref28http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref29http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref29http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref29http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref30http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref30http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref30http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref31http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref31http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref31http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref32http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref32http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref32http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref33http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref33http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref33http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref34http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref34http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref34http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref35http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref35http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref35http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref36http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref36http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref36http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref37http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref37http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref37http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref38http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref38http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref38http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref38http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref39http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref39http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref40http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref40http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref41http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref41http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref41http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref42http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref42http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref42http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref42http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref42http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref43http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref43http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref43http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref44http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref44http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref44http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref45http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref45http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref46http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref46http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref46http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref47http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref47http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref47http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref48http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref48http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref48http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref49http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref49http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref49http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref50http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref50http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref50http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref51http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref51http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref52http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref52http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref52http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref53http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref53http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref54http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref55http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref55http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref55http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref56http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref56http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref56http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref57http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref57http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref57http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref58http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref58http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref58http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref59http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref59http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref59http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref59http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref60http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref60http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref60http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref61http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref61http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref61http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref62http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref62http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref62http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref62http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref64http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref64http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref64http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref65http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref65http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref65http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref66http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref67http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref67http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref67http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref67http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref68http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref68http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref68http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref69http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref69http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref69http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref70http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref70http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref70http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref71http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref71http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref71http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref71http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref72http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref72http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref73http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref73http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref73http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref74http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref74http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref74http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref75http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref75http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref75http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref76http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref76http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref76http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref77http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref77http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref77http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref78http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref78http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref78http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref79http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref79http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref79http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref80http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref80http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref80http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref81http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref82http://refhub.elsevier.com/S0022-5193(16)30071-6/sbref82

Games of multicellularityIntroductionModel and results for maximum complex size nequal2ExamplesCooperationHawk-doveAverage fitness

Model for larger complexes, n≥3The sigma-condition for n≥2DiscussionAcknowledgmentsExact solutions for nequal2Evolutionary stability for nequal2Exact solutions for nequal3Well-mixed games with mutationReferences