Modern optimization methods 1

Comparison of optimization algorithms

Modern optimization methods 2

Overview of single-objective algorithms

• With one solution in given time:– Gradient methods– Hill-climbing– Simulated annealing– TABU search– (1+1)-ES

• Advantages: small number of evaluations, fast convergence

Modern optimization methods 3

• With a set of solutions:– Binary Genetic Algorithms– Evolution Strategies– Differential Evolution– SADE/GRADE + CERAF

• Advantage: Robustness

Overview of single-objective algorithms

Modern optimization methods 4

Example of comparison

Modern optimization methods 5

Benchmarks• 20 mathematical functions from

1 to 20 variables• In Matlab and C++

F1

http://klobouk.fsv.cvut.cz/~anicka/testfunc/testfunc.html

Quartic

PShubert1Goldprice

Branin

F3

Modern optimization methods 6

One run comparison:“Progress plots”

Modern optimization methods 7

One hundred runs comparison

Modern optimization methods 8

One hundred runs comparison

Modern optimization methods 9

Traditional measures

Mean best fitness

Success rate

Modern optimization methods 10

Comparison on reliabilityFunction Dim Fmincon GRADE GRADE+CERAFF1 1 100 100 100F3 1 100 100 100Branin 2 100 100 100Camelback 2 100 100 100Goldprice 2 100 100 100PShubert1 2 100 100 100PShubert2 2 100 100 100Quartic 2 100 100 100Shubert 2 100 100 100Hartman1 3 100 100 100Shekel1 4 100 100 100Shekel2 4 100 100 100Shekel3 4 100 100 100Hartman2 6 100 59 100Hosc45 10 100 100 100Brown1 20 100 100 100Brown3 20 100 100 100F5n 20 100 100 100F10n 20 0 78 100F15n 20 1 100 100

Modern optimization methods 11

Comparison on convergence speedFunction Dim Fmincon GRADE GRADE+CERAFF1 1 27 55 55F3 1 57 95 95Branin 2 24 348 348Camelback 2 40 198 198Goldprice 2 63 337 337PShubert1 2 2097 3879 1402PShubert2 2 1615 2333 896Quartic 2 56 320 331Shubert 2 375 606 603Hartman1 3 63 284 292Shekel1 4 335 47577 4078Shekel2 4 255 15356 2686Shekel3 4 284 7310 2496Hartman2 6 200 123727 9881Hosc45 10 264 2147 2096Brown1 20 286979 176628 182390Brown3 20 5660 36568 36090F5n 20 15838 6734 7284F10n 20 ------ 89715 226374F15n 20 374110 22378 25528

Modern optimization methods 12

Summary

• Disadvantages of traditional measures:– Unpractical setting of functions calls limit for

MBF type of measures– Need of optimum value knowledge for SR based

measures

• Result:– Whole progress plot is of importance

• Disadvantage:– Too much data need to be stored

Modern optimization methods 13

Proposed solution for two methods

• Store only 10 Dim results (like generations)• Use statistical test to judge the result of

comparison (Mann-Whitney-Wilcoxon test)

Modern optimization methods 14

Proposed solution cont.

Modern optimization methods 15

Proposed solution for more methods

• How to graphically compare more than two methods?– From multi-objective domain: Pair-vise comparison table

Modern optimization methods 16

Proposed solution for more methods

• How to graphically compare more than two methods?– From single-objective domain: Partial ordering?

[Carrano et. al.: GECCO’08]

Modern optimization methods 17

Proposed solution for more methods

• Relative Winning Score

scoresofwinnerisicasesNo

RWSi

.

11,0 ii RWSRWS

Modern optimization methods 18

Modern optimization methods 19

Modern optimization methods 20

Modern optimization methods 21

Traditional sizing problems

Modern optimization methods 22

Proposed solution for more methods

Modern optimization methods 23

References on Traditional measures

• A.E. Eiben, J.E. Smith: Introduction to Evolutionary Computing, Springer (2008).

• A.E. Eiben, M. Jelasity: A critical note on experimental research methodology in EC, Proceedings of the 2002 Congress on Evolutionary Computation, 2002. CEC '02.

• Thomas Bartz-Beielstein: Experimental Research in Evolutionary Computation - The New Experimentalism. Springer, Berlin, 2006.

• Thomas Bartz-Beielstein web-page: http://ls11-www.informatik.uni-

dortmund.de/people/tom/

Modern optimization methods 24

A humble plea. Please feel free to e-mail any suggestions, errors andtypos to matej.leps@fsv.cvut.cz.

Date of the last version: 23.11.2011Version: 001