Parallel Simulated Annealing with Adaptive Neighborhood determined by GA
description
Transcript of Parallel Simulated Annealing with Adaptive Neighborhood determined by GA
![Page 1: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/1.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
Doshisha University, Kyoto, Japan
Parallel Simulated Annealing with Adaptive Neighborhood determined by GA
Mitsunori MIKI Tomoyuki HIROYASU○ Toshihiko FUSHIMI
![Page 2: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/2.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Introduction Optimization problems become more complicated and larger.
•Simulated Annealing (SA)based on the simulation of the physical process “annealing”.
Important mattersImportant matters
1. Parallelization2. Adaptive parameter tuning
Heuristic search
•GA, CA, NN etc.
![Page 3: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/3.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Algorithm of Simulated Annealing
AlgorithmAlgorithm
Ene
rgy
low
1. Generation
2. Judge Transition
3. Cooling
Design spaceDesign space
high
( E = Enext - Enow)⊿
good acceptancegood acceptance
bad acceptancebad acceptance
1
Exp( )- E⊿Temperature
Metropolis probability
![Page 4: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/4.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Neighborhood range
•Can’t search optimum effectively.
•Often trapped in a local minimum.
Too large neighborhood range
Too small neighborhood range
The neighborhood range in the continuous Euclid space is the extent for generating next solution.
Global optimumGlobal optimum
•The range has to be small.
•The range has to be large.
![Page 5: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/5.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Background
For the control of the neighborhood range, some method are proposed.
These methods control the neighborhood range using an appropriate acceptance ratio.
• The adaptive neighborhood mechanism. [Corana 1987]• The advanced adaptive neighborhood mechanism. [Miki 2002]
This type of adaptive neighborhood method is very effective and useful, but the target acceptance ratio should be determined experimentally.
Propose a new adaptive neighborhood mechanismPropose a new adaptive neighborhood mechanism
![Page 6: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/6.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Purpose
Parallel Simulated Annealing with Adaptive NeighborhoodParallel Simulated Annealing with Adaptive Neighborhooddetermined by Genetic Algorithm (PSA/ANGA)determined by Genetic Algorithm (PSA/ANGA)
Controlling the neighborhood range adaptively during the search.
•This method is Parallel model.•This method parallels neighborhood ranges on each processes.•This neighborhood range is controlled by GA.
CharacteristicsCharacteristics
![Page 7: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/7.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Effect of Neighborhood RangesN
eig
hb
orh
oo
d ra
nge
large
small
Fixed neighborhood rangeFixed neighborhood range Search spaceSearch space
•The neighborhood range has a significant effect on the accuracy of the solution.•In order to verify this effect, some numerical experiments were carried out with various fixed neighborhood ranges.
Compare the qualities of the solutions.
Obtain the effect of the neighborhood ranges.
![Page 8: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/8.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Test problems
Rastrigin Griewangk Rosenbrock
MathematicalMathematicaltest functionstest functions
Griewangk function
Rastrigin function
Rosenbrock function
![Page 9: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/9.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Appropriate neighborhood range
AppropriateAppropriateneighborhood rangeneighborhood range
Rastrigin
The neighborhood range has a significant effect on the performance of SA.
Good solutionGood solution
![Page 10: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/10.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Appropriate neighborhood range
AppropriateAppropriateneighborhood rangeneighborhood range
Griewangk
The neighborhood range has a significant effect on the performance of SA.
Good solutionGood solution
![Page 11: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/11.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Appropriate neighborhood range
AppropriateAppropriateneighborhood rangeneighborhood range
Rosenbrock
The neighborhood range has a significant effect on the performance of SA.
Good solutionGood solution
![Page 12: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/12.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Concept of PSA/ANGA
The neighborhood range determined adaptively by GA.
PSA searches the solution with various neighborhood range.
•The appropriate neighborhood ranges depend on problems.•It is difficult to find the appropriate neighborhood ranges in advance.
There are the appropriate neighborhood ranges in SA when solving the continuous optimization problems.
Parallel Simulated Annealing with Adaptive NeighborhoodParallel Simulated Annealing with Adaptive Neighborhooddetermined by Genetic Algorithm (PSA/ANGA)determined by Genetic Algorithm (PSA/ANGA)
![Page 13: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/13.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Algorithms of PSA/ANGAN
eig
hb
orh
oo
d ra
nge
large
small
Multiple SA processes searches the solution with various neighborhood range.
GA operators are applied on neighborhood ranges.Fitness =
1
Energy
![Page 14: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/14.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Abstract of numerical experiments
Parallel SA with Fixed Neighborhood (PSA/FN)Parallel SA with Fixed Neighborhood (PSA/FN)
Optimum fixed neighborhood range
Comparative methodComparative method
Use the optimum fixed neighborhood range determined by preliminary numerical experiments.
PSA/ANGA is compared with a parallel SA with optimum fixed neighborhood range, PSA/FN.
![Page 15: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/15.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Parameters usedFunctions Rastrigin Griewangk Rosenbrock
Max temperature 10 20 1
Min temperature 0.01 0.001 0.001
Markov length 102400 307200 3072
No. of variable 3 3 3
Cooling rate 0.8 0.7 0.8
Optimum fixed neighborhood range
1.0 5.5 0.3
No. of processes 32
![Page 16: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/16.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
Performance of the proposed method
Proposed method
The proposed method, PSA/ANGA, provides better The proposed method, PSA/ANGA, provides better performance than PSA/FN in all problems.performance than PSA/FN in all problems.
![Page 17: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/17.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
History of Neighborhood range
Rastrigin
•History of the neighborhood ranges in 32 SA processes.
•The appropriate neighborhood range varies dynamically during the search.
The appropriate neighborhood range is automatically The appropriate neighborhood range is automatically determined using GA. determined using GA.
![Page 18: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/18.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
History of energy (Rastrigin)
•The proposed method, PSA/ANGA, shows fast convergence of the energy and obtains lower energy than PSA/FN.
•Accuracy of the solution improves because the neighborhood ranges were changed adaptively.
![Page 19: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/19.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
ConclusionsA new Parallel Simulated Annealing method with adaptive neighborhood range mechanism is proposed.
Parallel SA with Adaptive NeighborhoodParallel SA with Adaptive Neighborhood determined by Genetic Algorithm (PSA/ANGA)determined by Genetic Algorithm (PSA/ANGA)
PSA/ANGA shows good performance on the some test functions.
The appropriate neighborhood range varies according to the condition of the search.
The proposed method adapts to these appropriate The proposed method adapts to these appropriate neighborhood ranges.neighborhood ranges.
The method is effective in SA for continuous optimization The method is effective in SA for continuous optimization problems.problems.
![Page 20: Parallel Simulated Annealing with Adaptive Neighborhood determined by GA](https://reader036.fdocuments.net/reader036/viewer/2022062518/56814a12550346895db739a5/html5/thumbnails/20.jpg)
2003 IEEE International Conference on Systems, Man & Cybernetics
2003.10.06
questions and answers
Thank you for your kind attention.