Impact of Selection Strength on the Evolution of ...mitchenerg.people.cofc.edu/Mitchener - Impact of...
Transcript of Impact of Selection Strength on the Evolution of ...mitchenerg.people.cofc.edu/Mitchener - Impact of...
Impact of SelectionStrength on the Evolutionof Regulatory Networks
W. Garrett MitchenerCollege of Charleston Mathematics [email protected]://mitchenerg.people.cofc.edu
Introduction
The problem that led to this project was a result from Plotkin and Nowak [2000], inwhich an inequality from information theory puts an upper bound on the fitness ofa communication system with a one-to-one mapping of signal to meaning. If thereare enough topics of conversation, switching to a sequential coding system (string-of-signals to one meaning) yields a fitness advantage that overcomes that bound.See also Zuidema and de Boer [2009]. Evolutionary game theory can explain thesurvival and spread of such a system, but not its origin from organisms that do nothave it.
The basis of biological computation is the reaction or regulatory network. How aresuch networks discovered by selection-mutation processes?
The Utrecht Machine (UM) is a discrete abstraction of a gene regulatory network.For this project, an evolutionary simulation is used to discover UM-based agentsthat solve a data encoding problem. Details of the selection process have a signifi-cant impact on population dynamics: Weak selection leads to shorter overall timesto first perfect solution, shorter genomes, fewer repairs to auxiliary mechanismsafter an improvement to the main mechanism, and fewer atypical synaptic codes,and is more likely to discover improved partial solutions by breeding outliers. Thetimes at which key innovations appear and spread depends on the population statein surprising ways.
FYI: I call it the Utrecht Machine because the conference where I first presented it was
EvoLang 2010 in Utrecht.
Traditional population genetics
Abstracts away the generegulatory network.Models the genotype as alist of alleles, mapped to anumerical fitness for thephenotype. Markovian &memoryless.
A1 A1
B1 B1
C2 C2
♂ ♀Parents
A2 A2
B1 B2
C3 C3
F[...B1B1...]=4.5F[...B1B2...]=4.8
Fitness table
...
P[B1⇝B2]=0.01
Mutation table
...
A1 A2
B1 B1
C2 C3
A1 A2
B1 B2
C2 C3
A1 A2
B1 B1
C2 C3
A1 A2
B1 B2
C2 C3
B2B1B1
B1B1
B1B1
♂
♀Offspring
Systems biology
Genes produce functionalproducts (enzymes, etc.)but also regulatoryproducts that promote orrepress other genes.Complex networks ofinteracting molecules.
lacIrepressor
lacZenzyme lacY lacA
lactose
glucose
galactose
lacI disabledby lactose
lac operon
Population game dynamics: replicator-mutator
(1) xj =
K∑k=1
xkFkQkj − xjφ
• k = 1, 2, . . .K: types of individuals• xk = fraction of population of type k• Bkj = payoff for type k interacting with type j• F = Bx: vector of average payoffs by type• φ = xTF: overall average payoff• Qkj = probability offspring of type k is type j
Population genetics: Moran-Mendel
The population is finite. Dynamics form a Markov chain: Kill one agent uni-formly, replace by cloning another chosen in proportion to fitness.
• A1,A2, . . . ;B1,B2 . . . : alleles at loci A & B
• Phi, . . . : prob of mutation from Ah to Ai . . .• Fj1j2k1k2 = fitness of genotype Aj1Aj2Bk1Bk2
How to combine selection-mutation populationdynamics with relevant molecular details?
• Genome = inherited record of instructions:Should be a string of letters, subject to mutation and recombination like DNA
• Virtual machine = how to build the phenotype:Should be an artificial regulatory network
The Utrecht Machine or UM is designed to meet those requirements. The state ofthe UM is a table mapping patterns p to integer activation levels Ap. Each patternis metaphorically a protein that binds to a promoter or inhibitor sequence on DNA.The activation level of a pattern is how many units of it are present.
Each UM reaction instruction [As > θ =⇒ incAp, decAq] is parameterized by fourintegers: a switch pattern s, a threshold θ, a pattern p to activate called p-up, anda pattern q to inhibit called p-down. An instruction is active if the activation level ofits switch pattern As satisfies As > θ. An active instruction adds 1 to the activationof its p-up pattern and subtracts 1 from the activation of its p-down pattern. Allactive instructions are performed simultaneously.
A bit of input is provided by increasing the activation of a particular pattern duringeach time step the input bit is set.
Example: UM for binary NAND
Output
0
NAND
6
4
5
0
1
2
3
1
1
4
Input
0
1
Network representation, essentials only b0 b1 ¬(b0 ∧ b1)
F F TF T TT F TT T F
A0 > 0 =⇒ incA0, decA7
A1 > 0 =⇒ incA6, decA7
A2 > 0 =⇒ incA6, decA7
A4 > 1 =⇒ incA7, decA6
A5 > 1 =⇒ incA7, decA6
Two bits of input supplied through patterns 4 and 5, one bit of output read as1(A6 > 3). Stop when A0 > 4 or after a maximum of 10 time steps.
UM reaction instructions are just tuples of 4 integers, and their effects are indepen-dent of the order in which they are listed. They can therefore be encoded as a binarygenome and subject to biologically realistic mutation and recombination.
Building the network representation
Output
0
06
4
3
6
1
7
6
2
7
7
4
6
0 7
7
5
6
0
0
0
0
1
1
0
0
1
1
4
5
Input
0
1
One link for each. . .
• input bit
• output bit
• stop signal
Each instructiongenerates two linksfrom the switchpattern node: one forp-up, one for p-down.
Links for p-up, input, & output are solid with black arrowhead. Links for p-down are dashed with white arrowhead. Pattern nodes are circular for mostpatterns, keystones for those involved with input and output. Numbers on linksindicate a threshold. A link with no number is activated by an input bit insteadof a threshold.
Reduction from full to essential network
Network representation, all details
Input
0
1
NAND
6
4
5
0
7
1
2
1
1
0
0
1
10
0
00
Output
03
4
Merge pattern nodes, layout so that infor-mation mostly flows left to right.
Trim non-essential details...
does not affect output
switch pattern irrelevant
Remove pattern nodes that don’t mat-ter, like 7, which is used as a no-op or bitbucket in this machine. Turn nodes wherethe pattern doesn’t really matter into dia-monds. This happens to patterns 0, 1 and 2because all instructions with these for theswitch pattern happen to always be active.Then neaten up the layout.
Example: NAND with genome and step-by-step interface
0 0 0 7
1 0 6 7
2 0 6 7
4 1 7 6
5 1 7 6
genome bits
instruction bits
instruction integers
input / output bits
activation levels
pointer to next instruction
time
switch pattern
threshold p-upp-down
Triples of genome bits are decoded using majority-of-three into instruction bits,which are read left-to-right as the binary representations of integers.
Why the extra genome encoding layer, majority-of-three?
GA
A
A
A
A
G
G
G
G
C
C
C
C
U
U
U
U
UC AG UCAGUCAGUCAGUCAGU
CAG
UCAGUCAGUCAGUCAGU
CA
GUCAGUCAGUCAG
UCAG UCAG
P
S
U
nG
nG
oG
oG
oG
G
P
P
P
P
P
nM
nM
M
M
nM
nM
nM
Phenylalanine
Phe
Leucine
Leu
Leucine
Leu
Proline
Pro
Histidine
His
Glutamine
Gln
Isoleucine
Ile
Methionine
Met
Threonine
Thr
Asparagine
Asn
Lysine
Lys
Arginine
Arg
Arginine
Arg
Valine
Val
Alanine
Ala
Glutamic acid
Glu
Aspartic acid
Asp
Glycine
Gly
Serine
Ser
Serine
Ser
Tyrosine
Tyr
Cysteine
Cys
Tryptophan
Trp
Stops
Stop
E G F LS
S
Y
C
WL
P
H
R
R
QIM
TN
K
V
A
D89.09
75.07
174.20
174.20
146.19
165.19
133.11
117.15
147.13
146.15155.16
115.13
105.09
105.09
131.18
132.12
MW
= 14
9.21 D
a
131.18
119.12
204.23
131.18
181.19
121.16
HN
NH2
NH
H2N
OH
O
H2N
CH3 OH
O
H2N
O
H2N
OH
O
O
HO
H2N
OH
O
HS
H2N
OH
O
H2N
O
NH2
OH
O
O
OH
H2N
OH
OH2N
OH
O
NH
H2N
OH
O
N
CH3 CH3
H2N
OH
O
CH3
CH3
H2N
OH
O
CH3
CH3
H2N
OH
O
H2N
H2N
OH
O
CH3 S
H2N
OH
O
H2N
OH
O
NH
OH
O
H2N
HO OH
O
H2N
HO OH
O
H2N
HO
CH3
OH
O
NH
H2N
OH
O
HO
H2N
OH
O
H2N
CH3
CH3
OH
O
BasicAcidicPolarNonpolar(hydrophobic)
S -M - P - U - nM -oG - nG -
SumoMethyl
PhosphoUbiquitinN-Methyl
O-glycosylN-glycosyl
Modification
am
ino a
cid
2nd1st position 3rdUC
Picture by Kosi Gramatikoff and Seth Miller, posted on Wikipedia
The natural genetic code maps 64three-letter codons to 20 or so amino acidsplus a few control codes. Most amino acidsare represented by several synonymouscodons. Some of these codons are robust inthat substitution mutations are likely toresult in another synonymous codon,leaving the resulting protein unchanged.Ex: CUU CUC, both code for leucine.Other codons are fragile in that they aremore likely to mutate to a different aminoacid. Ex: UUA UUC switches fromleucine to phenylalanine. An excess offragile codons is thought to be evidencethat a gene has recently been subject toselection [Plotkin and Kudla, 2011]. Themajority-of-three encoding in the simulationis so that a future project can address thisphenomenon.
Recombination via haploid crossover
Agents in the simulation have a single chro-mosome. They reproduce in pairs. Thegenomes are aligned at the beginning andsplit at a random location. The first part ofone (blue) is attached to the second part ofthe other (red) to form the offspring genome.
Supported mutations
1 0 0 1 → 1 1 0 1single bit substitutionsprob 0.005 per genome bit
→ ✂
gene (instruction) deletionprob 0.001 per gene
→gene (instruction) duplicationprob 0.001 per gene
Experimental task: Transmit 2 bits over time
We use a selection-mutation process to evolve solutions to a sequential coding prob-lem. A genome is built into an agent follows: Two UMs are built from the genome,one sender and one receiver. The sender gets two bits of input, plus constant inputinto its role pattern 1. The receiver must generate two bits of output that reproducethe input, and signal when to stop. The receiver gets input from the sender througha single synapse, plus constant input into its role pattern 2.
Sketch of agent & scoring details
Input
0
1
Sender
3
8
9
1
Receiver
4
12
13
02
10
Output
0
1
10
10
10
Each agent is presented with allfour possible input words 〈00〉, 〈10〉,〈01〉, 〈11〉, starting at a zero statefor each and running for up to 100time steps. It earns 10, 000 points foreach bit correctly transmitted, plus100 − max(20, t) each time it stops af-ter t steps. Maximum possible score:10, 000× 4× 2 + 80× 4 = 80, 320.
Selection protocols
...
Agents are sorted in descending order by score. Then 300 newagents are created as follows: k1 new agents are created by breed-ing pairs selected uniformly at random from among the top h1, thenk2 are bred from the top h2, . . . . New agents are added to the top*of the list, and agents on the bottom are killed to maintain a con-stant population of 500. The list is re-sorted, preserving the originalorder* in case of a tie.
*These details prevent stagnation.
Selection strengths:
Protocol name (h1,k1), (h2,k2) . . .
Very weak (300, 500)—all agents breed equallyWeak (100, 200), (500, 100)
Strong (100, 300)—only top 100 breedVery strong (10, 300)—only top 10 breed
Even under very weak selection there is still some selection because agents withlow ratings are more likely to end up at the bottom of the list and die. All of theseprotocols eventually yield maximum-scoring solutions starting from an initial popu-lation of 500 genomes each with 32 genes generated uniformly at random. Patternshave six bits (0-63) and thresholds have four bits (0-15). Each gene has 22 instruc-tion bits, which are encoded by majority-of-three as 66 genome bits.
Example solution to bit transmission problem
Output
0
1
Receiver
0
2
3
4
12
13
22
42
47 10
15
3
10
610
7
6
4
3 4
5
4
012
Sender
1
0
3
5
8
9
47
10
7
3
3
1
3
9
0
12
6
Input
0
1
Timing
Pre-boost
Early activation
Late activation
cruft
00100111
0 5 10 15 20
input words stop signal
synapse turned on
synapse turned off
Synaptic code
time
Typical synaptic code:
〈00〉 is represented bynever activating thesynapse. 〈11〉 isrepresented by veryearly activation of thesynapse. 〈10〉 and 〈01〉are represented byintermediate activationtimes. This code istypical of solutionsevolved by thesimulation.
• Chicken-and-egg problem solved incrementally
• Timing mechanism entangled with other mechanisms
• Spurious but harmless activation of receiver mechanism in sender
• Recombination assembles new combinations of genes, discovers better net-works, often from sub-optimal parents
Rating vs. time
0k 50k 100k 150k 200k0
20 000
40 000
60 000
80 000
æ
ææ
æ ææææ æ æ ææ æ
0 12k
60 000
60 320
æ
æææ ææ æ
200k 230k80 160
80 320
Maximum rating as a func-tion of agent ID number.Agents are numbered asthey are born, so ID numbermeasures time. Large jumpsoccur at each major innova-tion, that is, when an agentappears that transmits anadditional bit correctly. Re-ductions in time spent yieldsmaller jumps, shown asinserts.
Typical trajectory is as follows: (1) Initially, most agents always output 〈00〉, scoring40, 000 = half credit. (2) Recombination joins a link from one input bit directly tothe synapse with a link from the synapse directly to the corresponding output bit,scoring 60, 000. (3) A sender mechanism appears that generates a fully informativesynaptic code, but no benefit is realized until. . . (4) A mechanism appears that con-nects to the other output bit, scoring 70, 000. (5) Some sort of complex mechanismis finally discovered that transmits all 8 bits correctly. After each major innovation,the timing mechanism may need to be repaired to get the other 320 points.
Distribution of ratings by generation
10 20 30 40 50
100
200
300
400
500
250 260 270 280
100
200
300
400
500
690 700 710 720 730
100
200
300
400
500
20 40 60 80 100
1
5
10
50
100
500
240 260 280 300 320 340
1
5
10
50
100
500
700 720 740 760 780
1
5
10
50
100
500
Color scheme:
1005 1010
100
200
300
400
500
0
10000
20000
30000
40000
50000
60000
70000
80000
Upper row: Stacked bar chart of counts of agents whosescore is between 10, 000n and 10, 000(n + 1) in each genera-tion for 50 generations after each major innovation.Lower row: Log-scale plot of count of agents whose scoreis nearly maximal in each generation for 50 generations af-ter each major innovation. (That is, the count in the highestcorresponding sub-bar in the chart above it.)
Each major innovation is more difficult than the previous ones. When an agent thatcan transmit an additional bit appears, that ability spreads exponentially, but forhigher-scoring innovations, the initial delay is longer, the growth rate is lower, andthe equilibrium fraction of high-scoring agents is lower.
Varying selection strength
Increasing selection strength reduces the number of agents that can reproduce,which has some counter-intuitive consequences. Aggregating observations from10, 000 runs of the simulation:
Log length of genome
1.4 1.6 1.8 2.0 2.2 2.4
V. Weak
Weak
Strong
V. Strong
Log agent ID
4.5 5.0 5.5 6.0 6.5 7.0
V. Weak
Weak
Strong
V. Strong
Distributions of the log10 length of the genome (left) and ID number (right) of thefirst agent in each sample run to achieve the maximum score. Bands indicatedeciles. The correlation is imperfect, but in general stronger selection leads tolonger genomes, and more agents searched (= more search time).
Unscaled distribution charts
Length of genome
50 100 150 200
V. Weak
Weak
Strong
V. Strong
Agent ID
0 2.0 ´106 4.0 ´106 6.0 ´106 8.0 ´106 1.0 ´107 1.2´107
V. Weak
Weak
Strong
V. Strong
Distributions of unscaled genome length and agent ID number. Bands indicatedeciles. A few especially long genomes and long-running samples draw out theuppermost decile.
Innovations through outliers
During the long equilibrium phases, most of thepopulation hovers near the peak of a ridge inthe fitness landscape. Genetic diversity ensuresthat there are always a few outliers that don’tachieve the highest score present in the popu-lation at that time. But they are more likely tobe near the edge of the zone of attraction of theridge, in which case their offspring are morelikely to jump to another ridge—an innovation.
In weaker protocols, there aremore samples with at least onemajor innovation where at leastone parent was an outlier. Thisphenomenon appears to be aconsequence of the fact thatweaker selection maintainsgreater diversity. V. Weak Weak Strong V. Strong
0.2
0.4
0.6
0.8
1.0
With
Without
Typical vs. atypical synaptic codes
Example atypical synaptic codes:00
10
01
11
0 5 10 15 20
00
10
01
11
0 5 10 15 20
Most solutions use typicalsynaptic codes something likethe example solution. But 5-20% of solutions end up withsomething very different, andoften with much more complexmechanisms.
In stronger protocols, a greaterfraction of runs settle on atyp-ical synaptic codes. Possibly,the reduced diversity associ-ated with stronger selectionincreases the likelihood thatthe population gets trapped ina ridge with a smaller zone ofattraction, which yields moreatypical results.
V. Weak Weak Strong V. Strong
0.2
0.4
0.6
0.8
1.0
Atypical
Typical
Timing repairs
V. Weak6 7 8
Weak6 7 8
Strong6 7 8
V. Strong6 7 8
20
40
60
80
100 Percent of runs that required repair or tweak-ing of the timing mechanism after the inno-vation that enables the transmission of anadditional bit: The height of the bar over n isthe percent of runs that jump from a rating of10, 000n + 320 to between 10, 000(n + 1) and10, 000(n+ 1) + 319.
Frequently, a mutation that improves the bit transmission mechanism disruptsthe timing mechanism, sometimes stealing part of its structure. Such an eventyields a jump of just under 10, 000 in score, followed by minor jumps that re-store the additional 320 timing points. Almost all runs have to repair (or ini-tially discover) the timing mechanism after the first major innovation, but only10-40% do so after the others. The very strong selection protocol has to makethe most repairs, but the trend is imperfect: the very weak protocol makesmore repairs than the weak and strong protocols after the innovation at n = 7.This means that it is not at all unusual (1) for evolution to sacrifice one func-tional mechanism to build another, and (2) for the resulting mechanisms to beentangled.
Time for last major innovation
Very weak Weak Strong Very strong
2.0 ´106 4.0 ´106 6.0 ´106 8.0 ´106 1.0 ´107 1.2´107
1.´10-9
1.´10-8
1.´10- 7
1.´10-6
0.00001
2´106 4´106 6 ´106 8 ´106
1.´10-9
1.´10-8
1.´10- 7
1.´10-6
0.00001
2´106 4´106 6 ´106 8 ´106 1´107
1.´10-9
1.´10-8
1.´10- 7
1.´10-6
0.00001
2.0 ´106 4.0 ´106 6.0 ´106 8.0 ´106 1.0 ´107 1.2´107
1.´10-9
1.´10-8
1.´10- 7
1.´10-6
0.00001
0 500 000 1.0 ´106 1.5´106 2.0 ´1060
1.´10-6
2.´10-6
3.´10-6
4.´10-6
5.´10-6
6.´10-6
0 500 000 1.0 ´106 1.5´106 2.0 ´1060
1.´10-6
2.´10-6
3.´10-6
4.´10-6
5.´10-6
6.´10-6
0 500 000 1.0 ´106 1.5´106 2.0 ´1060
1.´10-6
2.´10-6
3.´10-6
4.´10-6
5.´10-6
6.´10-6
0 500 000 1.0 ´106 1.5´106 2.0 ´1060
1.´10-6
2.´10-6
3.´10-6
4.´10-6
5.´10-6
6.´10-6
Upper row: Log-scale PDF histogram of number of agents searched betweennext-to-last and last major innovation. Lower row: Hazard function histogramof the same.
In traditional population dynamics, mutations are modeled with a Markovchain: An allele Aj in a parent mutates to Ak in a child with a probability thatdepends only on j and k. Starting from a population of all Aj, the time to firstappearance of Ak is approximately exponentially distributed. However, in thesimulation, the time required for the last major innovation (jump from a maxi-mum rating of 70, 000± to 80, 000) is not exponentially distributed: Its PDF isnot linear on a log plot, and its hazard function is not constant.
Hazard function
For a random variable T with density function f and cumulative distributionfunction F, the hazard function is
h(t) = lim∆t→0
P (t 6 T 6 T + ∆t | T > t)∆t
=f(t)
P (T > t)=
f(t)
1 − F(t)
The name comes from the idea that T is the time at which a device fails, so thehazard function is the instantaneous failure rate of the device at time t giventhat it has survived to time t. If T has an exponential distribution, a calculationshows that its hazard function is a constant, equal to the reciprocal of its mean.
Discussion
These simulations show that the origin story must be complicated. What seemsto be happening: The population goes through periods of neutral drift and diver-sification. Recombination tries many combinations of genetic variants of partiallysuccessful genes, including those from parents that score lower than the currentmaximum (epistasis). Innovations frequently take the form of entangled mecha-nisms that often must be tuned or repaired.
Stronger selection results in generally longer genomes and greater search time.There are also indications that it traps the population tightly near ridges in thefitness landscape with small zones of attraction (fewer innovations through outliers,more atypical synaptic codes). A possible explanation is that strong selection lowersgenetic diversity (founder effect).
The time between innovations is not exponentially distributed, which indicatessome memory effect: Shortly after one innovation, the population is more likelyto have another. Possibly during a selective sweep (time during which an innova-tion is spreading) the population retains some diversity which increases the chanceof another innovation. If the sweep runs to completion, the process is reset andbecomes memoryless.
Next steps
• Find ways to measure diversity directly and check these possibilities. Thegenomes are all different lengths with different amounts of space betweenuseful genes, so an alignment algorithm is needed to measure genotypic dis-tances.
• Implement fully diploid genomes. This seems to be an important detail: Re-combination may be more likely to find useful combinations of genes whenthey are at a distance on the genome. Under haploid recombination, it islikely that none of the offspring of such an individual will inherit both partsbecause crossover will occur in between them. But with diploid recombina-tion, roughly 1/4 of the offspring will inherit both parts.
• Model regulatory network formation more formally. This will require Markovchains with huge state space.
References
W. Garrett Mitchener. A discrete artificial regulatory network for simulating the evolutionof computation. In EvoNet2012: Evolving Networks, from Systems/Synthetic Biology toComputational Neuroscience, East Lansing, Michigan, USA, 2012.
Joshua B. Plotkin and Grzegorz Kudla. Synonymous but not the same: the causes and conse-quences of codon bias. Nat Rev Genet, 12(1):32–42, January 2011.
Joshua B. Plotkin and Martin A. Nowak. Language evolution and information theory. Journalof Theoretical Biology, 205(1):147–159, July 2000.
Willem Zuidema and Bart de Boer. The evolution of combinatorial phonology. Journal ofPhonetics, 37(2):125–144, April 2009.