25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1....
-
Upload
thomas-millam -
Category
Documents
-
view
215 -
download
1
Transcript of 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1....
![Page 1: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/1.jpg)
11 April 2023 Copyright Edward Tsang 2
Evolutionary Computation:Model-based Generate & Test
Model (To Evolve)
Candidate Solution
Observed Performance
Test
Feedback
(To update m
odel)
A Candidate Solution could be a vector of variables, or a tree
A Model could be a population of solutions, or a probability model
The Fitness of a solution is application-dependent, e.g. drug testing
Generate: select, create/mutate vectors / trees
![Page 2: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/2.jpg)
11 April 2023 Copyright Edward Tsang 3
Motivation
• Idea from natural selection
• To contain combinatorial explosion– E.g. the travelling salesman problem
• To evolve quality solutions– E.g. to find hardware configurations
![Page 3: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/3.jpg)
11 April 2023 Copyright Edward Tsang 4
Terminology in GA
• Example of a candidate solution
• in binary representation (vs real coding)
• A population is maintained
1 0 0 0 1 1 0 Chromosome with Geneswith alleles
String withBuilding blockswith values
fitness
evaluation
![Page 4: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/4.jpg)
11 April 2023 Copyright Edward Tsang 5
Example Problem For GA
• maximize f(x) = 100 + 28x – x2
– optimal solution: x = 14 (f(x) = 296)
• Use 5 bits representation– e.g. binary 01101 = decimal 13– f(x) = 295
Note: representation issue can be tricky Choice of representation is crucial
![Page 5: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/5.jpg)
11 April 2023 Copyright Edward Tsang 6
Example Initial Population
No. String Decim. f(x) weight Accum
1 01010 10 280 28 28
2 01101 13 295 29 57
3 11000 24 196 19 76
4 10101 21 247 24 100
Accumaverg
1,018254.5
To maximize f(x) = 100 + 28x – x2
![Page 6: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/6.jpg)
11 April 2023 Copyright Edward Tsang 7
Selection in Evolutionary Computation
• Roulette wheel method– The fitter a string,
the more chance it has to be selected
24%10101 28%
01010
29%01101
19%11000
![Page 7: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/7.jpg)
11 April 2023 Copyright Edward Tsang 8
Crossover
1 0 1 0 1
0 1 0 1 0
1 0 0 1 0
0 1 1 0 1
18
13
280
295
F(x)
0 1 1 0 1
0 1 0 1 0
0 1 1 1 0
0 1 0 0 1
14
9
296
271
1,142
285.5
Sum of fitness:
Average fitness:
![Page 8: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/8.jpg)
11 April 2023 Copyright Edward Tsang 9
A Control Strategy For GA
• Initialize a population• Repeat
– Reproduction• Select strings randomly but reflecting fitness
– Crossover– Mutation– Replace old population with offspring
• Until termination condition
![Page 9: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/9.jpg)
11 April 2023 Copyright Edward Tsang 10
Discussion
• New population has higher average fitness– Is this just luck?– Fundamental theorem
• Is an offspring always legal?– Epistasis
• A fundamental question: can candidate solutions be represented by string?
![Page 10: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/10.jpg)
11 April 2023 Copyright Edward Tsang 11
Claimed advantages of GA
• Simple to program
• General -- wide range of applicability
• Efficient -- finding solutions fast
• Effective -- finding good solutions
• Robust -- finding solutions consistentlyAre these claims justified?Different people have different opinions
![Page 11: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/11.jpg)
Epistasis Epistasis (interaction between Genes)(interaction between Genes)
Constraints
Penalties, Repair
Guided Genetic Algorithm
![Page 12: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/12.jpg)
11 April 2023 Copyright Edward Tsang 23
Constraints
• Task: find optimal solution in constrained satisfaction
• Difficulties: – Before one can attempt to optimization, one
needs to find legal solutions– Crossover may generate illegal solutions– Sometimes satisfying constraints alone is hard
• When problem is tightly constrained
![Page 13: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/13.jpg)
11 April 2023 Copyright Edward Tsang 24
Epistasis, Example in TSP
• Travelling Salesman Problem
• After crossover, offspring may not be legal tours– some cities may be
visited twice, others missing
1 3 684 5 2 7
6 8 514 2 7 3
1 3 684
6 8 514
2 7 3
5 2 7
crossover
![Page 14: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/14.jpg)
11 April 2023 Copyright Edward Tsang 25
Penalty Method
• If a string violates certain constraints
• Then a penalty is deducted from the fitness– E.g. in TSP, if penalties are high, GA may
attempt to satisfy constraints before finding short tours
• Problem with tightly constrained problems:– most strings are illegal
![Page 15: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/15.jpg)
11 April 2023 Copyright Edward Tsang 26
Repair Method
• If a string violates certain constraints
• Then attempt to repair it– E.g. in TSP, replace duplicated cities with
missing cities– possibly with complete search or heuristics
• Make sure that a population only contains legal candidate solutions– One can then focus on optimization
![Page 16: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/16.jpg)
GA for Machine LearningGA for Machine Learning
Classifiers
Bucket Brigade
![Page 17: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/17.jpg)
11 April 2023 Copyright Edward Tsang 31
Production System Architecture
Working MemoryProduction Rules
Scheduler
Retrieve data
Change data( firing )
Conditions Actions
Conditions Actions
Conditions Actions
…..
Facts
Sensor inputs
Internal states
…..
![Page 18: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/18.jpg)
11 April 2023 Copyright Edward Tsang 32
Classifier System Components
• Classifiers: Condition-Action rules– Special type of production system
• A Credit System– Allocating rewards to fired rules
• Genetic Algorithm– for evolving classifiers
![Page 19: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/19.jpg)
11 April 2023 Copyright Edward Tsang 33
Classifier System Example
Message List011100001100
Classifiers
01##:000000#0:1100
….
Detectors0111
Effectors1100
Info
Payoff
Action
Classifiers bid to fire
Classifiers have fixed length
conditions and actions
![Page 20: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/20.jpg)
11 April 2023 Copyright Edward Tsang 34
Apportionment of Credit
• The set of classifiers work as one system– They react to the environment– The system as a whole gets feedback
• How much to credit each classifier?– E.g. Bucket Brigade method
• Each classifier i has a strength Si
– which form the basis for credit apportionment– as well as fitness for evolution
![Page 21: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/21.jpg)
11 April 2023 Copyright Edward Tsang 35
Bucket Brigade, Basics
• A classifier may make a bid to fire
Bi = Si * Cbid
– where Cbid is the bid coefficient
• Effective bid: EBi = Si * Cbid + N(bid)– where N(bid) is a noise function – with standard deviation bid
• Winner of an auction pays the bid value to source of the activating message
![Page 22: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/22.jpg)
11 April 2023 Copyright Edward Tsang 36
Classifiers In Action
Cbid = 0.1
Ctax = 0
S3
22021818016220
Msg
10000001
B3
16
S4
22021819614620
Msg
0001
B4 R4
50
Classifiers01##:000000#0:110011##:1000##00:0001
Environment
S0
2002002002000
Msg
0111
B0
20S1
18020020020020
Msg0000
B1
20
20
S2
22018020018020
Msg
1100
0001
B2
2018
S5
22021819619620
![Page 23: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/23.jpg)
11 April 2023 Copyright Edward Tsang 37
More on Bucket Brigade
• Each classifier is “taxed”
• Si(t+1) = Si(t) - CbidSi(t) - CtaxSi(t) + Ri(t)– where Si(t) is strength of classifier i at time t– CbidSi(t) is the bid accepted– Ctax is tax– Ri(t) is payoff
• For stability, the bid value should be comparable to receipts from environment
![Page 24: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/24.jpg)
11 April 2023 Copyright Edward Tsang 38
Classifiers: Genetic Algorithms
• Tga = # of steps between GA calls
– GA called once every Tga cycles; or
– GA called with probability reflecting Tga
• A proportion of the population is replaced
• Selection: roulette wheel– weighted by strength Si
• Mutation: 0 {1,#}, 1 {0,#}, # {0,1}
![Page 25: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/25.jpg)
Genetic ProgrammingGenetic Programming
Building decision trees
GP for Machine Learning
![Page 26: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/26.jpg)
11 April 2023 Copyright Edward Tsang 40
Genetic Programming, Overview
• Young field– Koza: Genetic Programming, 1992– Langdon & Poli: Foundations of GP, 2001
• Diverse definitions– Must use trees? May use lists?
• Must one evolve programs?– Suitable for LISP
• Machine learning: evolving trees– dynamic data structure
![Page 27: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/27.jpg)
11 April 2023 Copyright Edward Tsang 41
Terminals and Functions
• Terminal set:– Inputs to the program– Constants
• Function set:– Statements, e.g. IF-THEN-ELSE, WHILE-DO– Functions, e.g. AND, OR, +, :=– Arity sensitive
![Page 28: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/28.jpg)
11 April 2023 Copyright Edward Tsang 42
Functions: Statements (e.g. IF-THEN-ELSE) or functions (e.g. AND, OR, +, )
Terminals: Input to the program or constants
GP: Example Tree (1)
Last race time
Won last time
If-then-else
Not-winWin
If-then-else
Win
5 min
<
![Page 29: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/29.jpg)
11 April 2023 Copyright Edward Tsang 43
GP Application
• A tree can be anything, e.g.:– a program
– a decision tree
• Choice of terminals and functions is crucial– Domain knowledge helps
– Larger grammar
larger search space
harder to search
![Page 30: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/30.jpg)
11 April 2023 Copyright Edward Tsang 44
GP: Example Tree (2)
Win Not-winWin
Last raced 3 months ago
Same Jockey
Not-winWin
Won last time
Same Stable
Boolean decisions only(limited functions)
Terminals: input to the program or constants
Functions: Statements (e.g. IF-THEN-ELSE) or functions (e.g. AND, OR, +, )
![Page 31: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/31.jpg)
11 April 2023 Copyright Edward Tsang 45
GP Operators
1
4
32
765
98
a
d cb
gfe
ih
1
4 3
2
76
5
98
a
d
cb
gfe
ih
Cro
ssov
er
Mutation: change a branch
![Page 32: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/32.jpg)
11 April 2023 Copyright Edward Tsang 46
Fitness in GP
• Generating Programs– How well does the program meet the
specification?
• Machine Learning: – How well can the tree predict the outcome?
• Function Fitting:– How great/small the error is
![Page 33: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/33.jpg)
11 April 2023 Copyright Edward Tsang 47
Generational GP Algorithm
• Initialize population
• Evaluate individuals
• Repeat– Repeat
Select parents, crossover, mutation
– Until enough offspring have been generated
• Until termination condition fulfilled
![Page 34: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/34.jpg)
11 April 2023 Copyright Edward Tsang 48
Steady-state GP Algorithm
• Initialize population P
• Repeat– Pick random subset of P for tournament– Select winners in tournament– Crossover on winners, mutation– Replace loser(s) with new offspring in P
• Until termination condition fulfilled
![Page 35: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/35.jpg)
Estimation of Distribution Algorithms Estimation of Distribution Algorithms (EDAs)(EDAs)
Population-based Incremental Learning (PBIL)
Building Bayesian networks
![Page 36: 25 August, 2014Copyright Edward Tsang1 Evolutionary Computation Edward Tsang, University of Essex 1. Overview of EA 2. Analysis of GA Fundamental Theorem,](https://reader035.fdocuments.net/reader035/viewer/2022070307/551abf6b55034656628b5864/html5/thumbnails/36.jpg)
11 April 2023 Copyright Edward Tsang 51
Population-based Incremental Learning (PBIL)
• Statistical approach • Related to ant-colonies, GA
Model M: x = v1 (0.5)x = v2 (0.5)y = v3 (0.5)y = v4 (0.5)
Sample from M solution X, eg <x,v1><y,v4>
Evaluation X
Modify the probabilities
0.60.4
0.60.4