A Hitchikers Guide To Parallel G As
Transcript of A Hitchikers Guide To Parallel G As
![Page 1: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/1.jpg)
Parallel GAs
A Hitchiker’s guide to parallel GAs
3 key papers by Eric Cantu-Paz and David Golberg
Presented by :
Yann SEMETUniversite de Technologie de Compiegne
![Page 2: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/2.jpg)
Parallel GAs
Our 3 papers
A survey of Parallel Genetic Algorithms, E. Cantu-Paz, 1998
On the scalability of Parallel GAs, E. Cantu-Paz & David Golberg, 1999
Markov Chain Models of Parallel GAs, E. Cantu-Paz, 2000
![Page 3: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/3.jpg)
Parallel GAs
Roadmap
General knowledge Toward parallel GAs Categorization Key issues
Theory run for parameter sizing Bounding cases Gambler’s Ruin model Markov Model
![Page 4: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/4.jpg)
Parallel GAs
A good nest
Illinois and parallelism
Illinois and GAs
An ideal nest for parallel GAs
![Page 5: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/5.jpg)
Parallel GAs
Flynn’s taxonomy
SISD : your PC
SIMD : not really relevant
MISD : multi processors
MIMD : super computers
![Page 6: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/6.jpg)
Parallel GAs
GAs : 2 levels of parallelism
Population-wise
Computation-wise
![Page 7: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/7.jpg)
Parallel GAs
Taxonomy
Master-slave GAs
Fine-grained GAs
Coarse-grained GAs
Hierarchical GAs
![Page 8: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/8.jpg)
Parallel GAs
Master and Slaves
Similar to a simple GA
Fitness/Communication tradeoff
Fitness distribution
Operator distribution
Efficient speedup
![Page 9: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/9.jpg)
Parallel GAs
Fine-grained
One population
Limited spatial interaction
Critical parameter : radius
Matches massively parallel computer
A potential alternative to Coarse-Grained
![Page 10: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/10.jpg)
Parallel GAs
Multiple-Deme
Key factors : Demes Migration Topology
![Page 11: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/11.jpg)
Parallel GAs
Hierarchical
Three possibilities
The engineer’s choice
![Page 12: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/12.jpg)
Parallel GAs
Non-Traditional GAs
ECO
GENITOR
mGAs, fmGAs
GP
![Page 13: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/13.jpg)
Parallel GAs
Engineering Summary 1
Categorization
Market Map
![Page 14: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/14.jpg)
Parallel GAs
Engineering Summary 2
Key parameters : Demes Migration Topology
Goal : remain panmictic
![Page 15: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/15.jpg)
Parallel GAs
Milestone1
Up to now : Categorization Key issues
To come : Theoretical scaling Markov models
![Page 16: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/16.jpg)
Parallel GAs
Theory Roadmap
Goal : calculate optimal parameters
From Master-Slave to : Multiple deme High Migration Dense topology
Markov models
![Page 17: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/17.jpg)
Parallel GAs
Paramaters to be tuned
Populations : Size Number
Migration : Rate Frequency
Topology : Density Shape
![Page 18: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/18.jpg)
Parallel GAs
Single Population 1
Assumptions Distributed population Modified operators to ensure panmictism
Why ? Straightforward and intuitive Close to Multiple-deme bounding case
![Page 19: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/19.jpg)
Parallel GAs
Single Population 2
Computation time :
Optimal chunk size :
cf
p TPkP
nTT )1(
c
f
kT
nTP *
![Page 20: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/20.jpg)
Parallel GAs
Multi Populations 1
2 Bounding cases : Lower Bound on migration and connectivity Upper bound : “deja vu)” but :
At most one migration per generation Picking the migrants
![Page 21: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/21.jpg)
Parallel GAs
The Gambler’s ruin model
A random walk to absorbative barriers
Predicts solution quality
Yields population sizing
A conservative model
0
0
1
1
1 x
n
n
bb p
q
pq
pq
P
![Page 22: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/22.jpg)
Parallel GAs
Multiple demes
Relaxation :
m
r
m
Q
mm
QP rr
2
ln
2
^
:
^^
![Page 23: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/23.jpg)
Parallel GAs
Regular topologies
Over two epochs
1
^
1^
x
p
qP
bbbb
bbx
PP
PP
11
^
1
![Page 24: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/24.jpg)
Parallel GAs
Optimal parameters
Deme size :
Optimal connectivity :c
xczz
nkk
d
2
^
12^^
2 22
3/2
0*
2
c
f
T
Tgn
![Page 25: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/25.jpg)
Parallel GAs
Topology considerations
Efficiency depends on connectivity
Extented Neighborhoods
After several epochs :
nPnrcP bbdmbb
![Page 26: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/26.jpg)
Parallel GAs
Derivation…
Dimensional analysis gives :
The GRM then gives :
1
1'
1'
mc
1'ln
1ln2
1'
1 0
^
n
qp
Pn
k
d
![Page 27: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/27.jpg)
Parallel GAs
Derivation…
Optimal connectivity :
Optimal number of epochs obtained similarly
3/2
0*
'2
c
f
T
Tgn
![Page 28: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/28.jpg)
Parallel GAs
The long run
At the end :
drnn
dnn 1'
11
max
r
![Page 29: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/29.jpg)
Parallel GAs
Finally
Solving the time equation :
Similar to single population !
c
f
T
nTgr
1*
![Page 30: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/30.jpg)
Parallel GAs
Markov Chains
Transient and closed states
M : transition matrix
N :fundamental matrix : T : time V : distribution A : Pbb on the long run
![Page 31: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/31.jpg)
Parallel GAs
Upper Bounding case
Full migration
Dense topology
V is a binomial distribution
![Page 32: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/32.jpg)
Parallel GAs
Arbitrary Migration
Different rates : might not converge
Yields more state
V is again binomial but with a disjunction
![Page 33: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/33.jpg)
Parallel GAs
Arbitrary Topologies
Even more states !
Which demes actually converged ?
Each state is a binary string
![Page 34: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/34.jpg)
Parallel GAs
Conclusions on Markov Chains
Predictive models
Panmictism on the long run
Prefer : High migration rates Dense topologies
![Page 35: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/35.jpg)
Parallel GAs
General Summary
Categorization
Key issues
Parameter sizing
Accurate Predictive Models
Tradeoff Practical Guidelines
![Page 36: A Hitchikers Guide To Parallel G As](https://reader033.fdocuments.net/reader033/viewer/2022060116/557e0794d8b42ae6458b4ae2/html5/thumbnails/36.jpg)
Parallel GAs
Discussion