Research Article Generalised Adaptive Harmony Search: A ...
Transcript of Research Article Generalised Adaptive Harmony Search: A ...
Hindawi Publishing CorporationJournal of Applied MathematicsVolume 2013 Article ID 380985 13 pageshttpdxdoiorg1011552013380985
Research ArticleGeneralised Adaptive Harmony Search A Comparative Analysisof Modern Harmony Search
Jaco Fourie1 Richard Green1 and Zong Woo Geem2
1 Department of Computer Science and Software Engineering University of Canterbury Christchurch 8140 New Zealand2Department of Energy and Information Technology Gachon University Seongnam 461-701 Republic of Korea
Correspondence should be addressed to Zong Woo Geem geemgachonackr
Received 30 January 2013 Accepted 22 March 2013
Academic Editor Xin-She Yang
Copyright copy 2013 Jaco Fourie et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
Harmony search (HS) was introduced in 2001 as a heuristic population-based optimisation algorithm Since then HS has become apopular alternative to other heuristic algorithms like simulated annealing and particle swarm optimisation However some flawslike the need for parameter tuning were identified and have been a topic of study for much research over the last 10 years Manyvariants of HS were developed to address some of these flaws andmost of them have made substantial improvements In this paperwe compare the performance of three recent HS variants exploratory harmony search self-adaptive harmony search and dynamiclocal-best harmony search We compare the accuracy of these algorithms using a set of well-known optimisation benchmarkfunctions that include both unimodal andmultimodal problems Observations from this comparison led us to design a novel hybridthat combines the best attributes of these modern variants into a single optimiser called generalised adaptive harmony search
1 Introduction
Harmony search (HS) is a relatively new metaheuristic opti-misation algorithm first introduced in 2001 [1] In thiscontext metaheuristic means that unlike general heuris-tic algorithms that use trial-and-error methods to solve aspecific problem it uses higher level techniques to solve ageneral class of problems efficiently HS fits into the categoryof population-based evolutionary algorithms together withgenetic algorithms (GAs) and the particle swarm optimisa-tion (PSO) algorithm
As is often the case with evolutionary metaheuristicsHS was inspired by a natural phenomenon In this case themethods used by professional musicians to collaborativelyimprovise new harmonies were taken as the inspiration forHS An analogy between improvisation and optimisation wasconstructed and an algorithmwas designed tomimic thewaya musician uses short-term memory and past experiences tolead her to the note that results in the most pleasing harmonywhen played together with the other musicians Harmonysearch is easy to implement and can easily be applied andadapted to solve almost any problem that can be modelled asthe minimisation or maximisation of an objective function
The objective function itself can be continuous or discreteand a smooth gradient is not required No initial solutionsor carefully chosen starting points are required In fact theobjective function is considered a black box by harmonysearch Any procedure that takes a solution vector as inputand gives a fitness score as output can be used as an objectivefunction
These properties make harmony search attractive It hasbeen successfully used in a wide range of disciplines includ-ing computer vision vehicle routing music compositionSudoku puzzle and various engineering disciplines [2ndash8]However it was soon realised that many aspects of HScan be improved and in particular that many of the HSparameters that are often difficult to set can be set andadjusted automatically
An early approach to this automatic adjustment of theHS parameters was the improved harmony search (IHS)algorithm developed by Mahdavi et al [9] They noticed thatthe pitch adjustment rate (PAR) and fret width (FW) (Thefret width (FW) was formally known as the bandwidth (BW)and it is still sometimes referred to as such The terminologyhas since been updated to better reflect the musical analogy)parameters are important to the fine tuning of optimised
2 Journal of Applied Mathematics
solution vectors and they suggested a method of adaptingPAR and FW to the relative progress of the optimiser insteadof keeping these parameters constant through all iterationsTheir algorithm alleviates the problem of choosing an exactvalue for the PAR and FW but it still requires that a range ofvalues specified by aminimum andmaximum value be given
Another early approach to improving HSrsquos performanceand solving the parameter setting problem is the global-bestharmony search (GHS) algorithm [10] It was inspired bythe swarm intelligence concept used in the particle swarmoptimisation algorithm (PSO) [11] Instead of using the FWparameter to adjust possible solutions closer to an optimumGHSmoves component values toward the value of the currentbest solution in the population
Both IHS and GHS improved on the performance of HSand alleviated the parameter setting problem to some degreeHowever just in the last few years many more HS variantswere developed that have shown to be superior to theseearly variants and some require even less manual parametersetting Most of these new variants are inspired by otherevolutionary algorithms and they combine the best aspectsof the simplex algorithm simulated annealing differentialevolution sequential quadratic programming and manyother aspects with HS to form new HS variants [12ndash16] Fora thorough summery of the latest research in HS variants seethe review article by Alia and Mandava [17]
In this paper we concentrate on the comparison ofthree of the most recent and promising HS variantsexploratory harmony search (EHS) self-adaptive harmonysearch (SAHS) and dynamic local-best harmony search(DLHS) All three improve on both the performance of theoriginal HS and many of the earlier HS variants We alsochose these variants because they were not developed ashybrid algorithms by combining HS with other optimisationalgorithms Instead the focuswas on investigating the steps ofHS and adding novelmodifications to key areas of the originalalgorithm
In the section that follows we briefly overview theHS algorithm and explain how the optimiser parametersinfluence the performance of the optimisation process Thisprovides the necessary background to introduce the threeHS variants in the sections that follow In Section 3 wecompare the performance of three HS variants using aset of well-known optimisation benchmark functions Theresults from these tests are then interpreted and discussedin Section 4 These observations then lead us to develop anovel hybrid algorithm called generalised adaptive harmonysearch (GAHS) that we introduce in Section 5 We concludein Section 6 with final remarks and our thoughts on futureresearch in the development of harmony search-based opti-misers
2 Harmony Search and OptimisersBased on Harmony Search
Theharmony search algorithm is ametaheuristic population-based optimisation algorithm inspired by the way musiciansin a band discover new harmonies through cooperative
improvisation [1] The analogy between the optimisationof an objective function and the improvisation of pleasingharmonies is explained by the actions of the musicians in theband Each musician corresponds to a decision variable inthe solution vector of the problem and is also a dimensionin the search space Each musician (decision variable) hasa different instrument whose pitch range corresponds to adecision variablersquos value range A solution vector also calledan improvisation at a certain iteration corresponds to themusical harmony at a certain time and the objective functioncorresponds to the audiencersquos aesthetics New improvisationsare based on previously remembered good ones representedby a data structure called the harmony memory (HM)
This analogy is common to all the HS variants that wewill investigate HS variants differ in the way solution vectors(improvisations) are generated and how the HM is updatedat the end of each iteration We start our explanation of howthis is done differently in each variant with an overview of theoriginal HS algorithm
21 An Overview of Harmony Search The core data structureof HS is a matrix of the best solution vectors called theharmony memory (HM) The number of vectors that aresimultaneously processed is known as the harmony memorysize (HMS) It is one of the algorithmrsquos parameters thathas to be set manually Memory is organised as a matrixwith each row representing a solution vector and the finalcolumn representing the vectorrsquos fitness In an119873-dimensionalproblem the HM would be represented as
[[[[[[[[
[
1199091
11199092
1sdot sdot sdot 119909
119873
1| 1199081
1199091
21199092
2sdot sdot sdot 119909
119873
2| 1199082
1199091
HMS 1199092
HMS sdot sdot sdot 119909119873
HMS | 119908HMS
]]]]]]]]
]
(1)
where each row consists of 119873 decision variables and the fit-ness score 119908 ([1199091 1199092 119909119873 119908]) Before optimisation startsthe HM is initialised with HMS randomly generated solutionvectors Depending on the problem these vectors can also berandomly chosen around a seed point that may represent anarea in the search space where the optimum is most likely tobe found [3]
The step after initialisation is called improvisation Anew solution is improvised by using three rules memoryconsideration pitch adjustment and random selection Eachdecision variable is improvised separately and any one ofthe three rules can be used for any variable The harmonymemory consideration rate (HMCR) is also one of the HSparameters that must be manually chosen It controls howoften the memory (HM) is taken into consideration dur-ing improvisation For standard HS memory considerationmeans that the decisions variablersquos value is chosen directlyfrom one of the solution vectors in the HM A randomnumber is generated for each decision variable If it is smallerthan the HMCR the memory is taken into considerationotherwise a value is randomly chosen from the range ofpossible values for that dimension
Journal of Applied Mathematics 3
If the HM is taken into consideration the improvisedvalue is chosen randomly from one of the values in the HMThe pitch adjustment rate (PAR) is set during initialisationand it controls the amount of pitch adjustment done whenmemory consideration is used Another random number isgenerated If it is smaller than the PAR the improvised valueis pitch adjusted using
1199091015840
new = 119909new + rand () sdot FW (2)
where 1199091015840
new is the new pitch-adjusted value 119909new is theold value chosen using memory consideration rand() is arandom value between minus1 and 1 and FW is the fret widthparameter The terminology has since been updated to betterreflect the musical analogy [5] that controls the maximumvariation in pitch adjustment and is one of the parameters thatmust be manually set
Once a new value has been improvised the memoryis updated by comparing the new improvisation with thevector in the memory with the lowest fitness If the newimprovisation has a higher fitness it replaces the vector withthe lowest fitness This process of improvisation and updatecontinues iteratively until some stopping criterion is fulfilledor the maximum number of iterations are reached
To summarise the main steps of the HS are as follows
(1) Initialise the HM with possible solutions from theentire search space of the function to be optimised
(2) Improvise a new solution vector using memory con-sideration pitch adjustment and random selection
(3) If the new improvisation is better than the worstsolution in the HM replace the worst solution withthe new improvisation
(4) Check the stopping criteria If it has not been metrepeat Steps (2) and (3) or else proceed to Step (5)
(5) Return the best solution (highest fitness) in the HMas the optimal solution vector of the function
This is a brief overview of HS The interested reader isreferred to [1 5] for a more detailed explanation
22 Exploratory Harmony Search The exploratory harmonysearch (EHS) algorithm was the result of improving theperformance of HS by focussing on the evolution of thepopulation variance in theHMduring optimisationDas et alrealised that the exploratory power of HS can be maximisedby dynamically adjusting the FW to be proportional to thestandard deviation of the HM population [18]
This realisation comes from the following lemma that isproven in [18]
Lemma 1 If the HMCR is chosen to be high (ie very nearto 1) and the FW is chosen to be proportional to the standarddeviation of the HM (ie FW prop 120590(HM) = radicVar(HM)) thenthe expected population variance (without updating the HM)can grow exponentially over iterations
This lemma led them to redefine the FW as a dynamicallyadjusting parameter defined by FW = 119896radicVar(HM) where 119896
is a proportionality constantTheFWis therefore recalculatedat each iteration based on the current HM variance insteadof using a constant value that must be chosen manually
To ensure exponential variance growth the PAR HMCRand 119896 must also be chosen such that ((HMS minus 1)HMS) sdot
HMCR sdot [1 + (13)1198962sdot PAR] is greater than 1 The proof of
this statement can also be found in [18]This simple change to HS results in an algorithm that
is less likely to get stuck in local optima requires lessiterations to reach a desired accuracy and frees the userfrom determining the best value for the FW parameterHowever one could argue that the FW parameter was simplyreplaced by the proportionality constant 119896 In their articleDas et al investigate the effect that variations in 119896 have onthe performance of EHS They determined that EHS is notsensitive to the choice of 119896 and used a constant value (119896 =
117) in all their experimentsThis contrastswith the choice ofthe FW inHS that is usually chosen to fit a particular objectivefunction and can have a significant impact on performance ifchosen poorly
23 Self-AdaptiveHarmony Search In 2010Wang andHuangdeveloped a variant of HS that we refer to as the self-adaptiveharmony search (SAHS) algorithm [19]Theirmain focuswasto alleviate the problem of choosing the best HS parametervalues for a specific problem SAHS does not require theFW and PAR parameters that need to be specified whenusing HS However SAHS like IHS requires that a minimumand maximum value for PAR be specified Specifying aminimum and maximum for PAR is typically much easierthan specifying an exact value and it is more reliable sincethe performance is not as sensitive to changes in the range ofPAR values as it is for a specific constant value
SAHS is somewhat similar to IHS in that it also replacesthe PAR with a range of PAR values Like IHS it alsorecalculates the current value of the PAR by linearly adjustingit between the minimum and maximum values based on theiteration count However it differs in that it decreases the PARby starting at the maximum value and linearly decreasing ituntil the minimum is reached at the final iteration instead ofincreasing it like IHS does
Unlike IHS the FW parameter is not replaced by a rangeof FW values Instead it is replaced by a novel method ofpitch adjustment that does not require an FW parameterWhen pitch adjustment of an improvised decision variableis required it is randomly adjusted for that decision variablebetween the minimum and maximum values found in thecurrent HM using the following equations
1199091015840
119894larr997888 1199091015840
119894+ [max (HM119894) minus 119909
1015840
119894] sdot 119903
1199091015840
119894larr997888 1199091015840
119894minus [1199091015840
119894minus min (HM119894)] sdot 119903
(3)
where 1199091015840119894is the new improvisationmax(HM119894) andmin(HM119894)
are the maximum and minimum values in the HM forthe 119894th decision variables and 119903 is a uniformly generatedrandomnumber between 0 and 1 Each time pitch adjustmentis performed the improvisation is updated by randomlyapplying with equal probability one of these two equations
4 Journal of Applied Mathematics
This approach causes progressively smaller changes to bemade to the new improvisation as max(HM119894) and min(HM119894)converge closer togetherwith increasing iterationsThereforepitch adjustment begins as a rough operator making largerchanges to favour exploration and then becomes a fineoperator favouring exploitation as the optimiser convergescloser to the optimum
Another important difference between SAHS and HS isin the initialisation of the HM In HS the HM is initialisedone decision variable at a time by randomly picking avalue from a uniform distribution of values in the allowablerange However this uniform distribution is only an approx-imation created by scaling the output from a pseudoran-dom number generator to the correct range of values Theresulting distribution is often unevenly distributed causingslow convergence or convergence to local optima Insteadof using a pseudorandom number generator SAHS uses alow discrepancy sequence [20] to initialise the HM Lowdiscrepancy sequences are more evenly spread out in thesearch space than pseudorandom ones causing the initialvectors in the HM to better represent the search space of theproblem
24 Dynamic Local-Best Harmony Search dynamic local-best harmony search (DLHS) was developed by Pan et al asan improvement to IHS and GHS that does not require theHMCR and PAR parameters [21] The focus of DLHS can beroughly divided into three categories First the improvisationstep was improved by basing new improvisations on thecurrent best solution vector in the HM instead of a randomlychosen one Second it divides theHM intomultiple indepen-dently optimised sub-HMs in an attempt to maintain greaterdiversity throughout the search process Third DLHS usesa self-learning parameter set list (PSL) to evolve the HMCRand PAR parameters into the optimal values without any userspecificationThe FW parameter is dynamically adjusted in asimilar way to the method that IHS uses
The modification of the improvisation process is inspiredby GHS Like GHS the random solution vector used inmemory consideration is replaced by the best solution vectorin the HM This tends to concentrate future improvisationsaround the current best solution which is likely a local opti-mum and it may be the global optimum To prevent possibleconvergence around local optima the random solution vectorthat was discarded in memory consideration is instead usedduring pitch adjustment So unlike the pitch adjustment of(2) DLHS uses the following equation
1199091015840
new = 119909rand + rand () sdot FW (4)
where 119909rand is randomly chosen from the current values inthe HM for a particular decision variable
Before improvisation starts the HM is randomly dividedinto 119898 equally sized sub-HMs Each sub-HM then usesits own small subset of solution vectors to independentlyconverge onto a local (or possibly global) optimum Dueto the loss of diversity that comes with restricting the sizeof the individual HMs by splitting them up into sub-HMsit is likely that convergence will not be towards the global
optimum but rather toward a local one To prevent this lossof diversity information is allowed to exchange between sub-HMs with a frequency controlled by the regrouping schedule119877 The regrouping schedule determines how often informa-tion exchange is allowed between sub-HMs by randomlyregrouping the individual solution vectors of all sub-HMsinto 119898 new sub-HM configurations every 119877 iteration Thisregrouping simultaneously increases the diversity of all sub-HMs and allows the best solutions from the entire HM to beshared among sub-HMs
Once the global optimum is identified the injection ofdiversity through the regrouping operation does not furtheroptimise but it can result in inaccurate convergence Forthis reason DLHS enters the final phase of the optimisationprocess after 90 of the iterations have been made In thisfinal phase the regrouping operation is halted and a newHM is formed by combining the best three solution vectorsfrom all sub-HMs into a single new HM The new HM isthen exclusively processed until the maximum number ofiterations has been reached
The HMCR and PAR parameter values are dynamicallydetermined by selecting from a self-learning parameter setlist (PSL)This process starts with the initialisation of the PSLby filling it with randomly generated HMCR and PAR valuesHMCR values are generated from a uniform distributionwhere HMCR isin [09 10] and PAR values are generatedfrom a uniform distribution where PAR isin [00 10] At thestart of each iteration one pair of HMCR and PAR valuesis removed from the PSL and used for that iteration If thecurrent iterationrsquos improvisation resulted in the HM beingupdated the current parameter pair is saved in the winningparameter set list (WPSL) This process continues until thePSL becomes empty The PSL is then refilled by randomlyselecting a pair from theWPSL 75of the time and randomlygenerating a new pair 25 of the time To prevent memoryeffects in the WPSL the WPSL is emptied each time the PSLis refilled The result of this is that the best parameter set isgradually learned and it is specific to the objective functionunder consideration The size of the PSL is set to 200 pairsIt was found that the effect that the size of the PSL has onperformance is insignificant [21]
Unlike the HMCR and the PAR the FW parameter isdynamically adjusted in a way that is similar to the wayIHS adjusts the PAR Like IHS DLHS favours a large FWduring the early iterations to encourage exploration of thesearch space while a small FW is favoured during the finaliterations to better exploit good solutions in the HMThe FWis therefore linearly decreasedwith increasing iterations usingthe following equation
FW (119894) =
FWmax minusFWmax minus FWmin
MI2119894 if 119894 lt
MI2
FWmin otherwise(5)
where FWmax and FWmin are the maximum and minimumvalues of the FW 119894 is the iteration number and MI is themaximum number of iterations Like IHS this means thata minimum and maximum value for the FW has to be setbefore optimisation starts but this is again assumed to bemuch easier than deciding on a single value for the FW
Journal of Applied Mathematics 5
221
100minus1
minus1minus2
minus2
minus3 minus3
119884
119883
2000400060008000
100001200014000
1
10
100
1000
10000
Figure 1 Rosenbrockrsquos valley is illustrated here in two dimensionsNotice the parabolic valley that includes the global optimum at[1 1]
3 Performance Analysis Using FiveBenchmark Functions
31 Benchmark Functions We start our performance analysiswith a brief description of the five benchmark functionsthat we will be using to compare performance All five ofthese functions were designed to be challenging to optimisefor different reasons and all have previously been used forbenchmarking harmony search-based algorithms
We start with a classic parametric optimisation functioncalled Rosenbrockrsquos valley [22] This multidimensional prob-lem has a global optimum inside a long narrow parabolic-shaped flat valley The valley itself is easy to find since thereare no other local minima anywhere in the search space butconverging onto the global optimum is difficult Rosenbrockrsquosvalley is defined by the following equation
119891 (x) =
119899minus1
sum
119894=1
100 (119909119894+1
minus 1199092
119894) + (1 minus 119909
119894)2
minus 2048 le 119909119894le 2048
(6)
Rosenbrockrsquos valley has a global optimum at 119909119894= 1 forall119894 where
the function evaluates to 0 An illustration of Rosenbrockrsquosvalley implemented in two dimensions is seen in Figure 1
Rastriginrsquos function uses cosine modulation to create ahighly multimodal function that often causes optimisationalgorithms to get stuck in the local optima without everfinding the global optimum [23] However it does have asingle global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Rastriginrsquos function is defined by
119891 (x) = 10119899 +
119899
sum
119894=1
(1199092
119894minus 10 cos (2120587119909
119894)) minus 512 le 119909
119894le 512
(7)
minus4
minus44
4
minus2minus2
2
2
00 10
10
20
2030
30 4040
5050
6060
70
70
80
80
119884
119883
Figure 2 This illustration of Rastriginrsquos function shows its highlymultimodal nature and it is difficult to visually identify the globaloptima at [0 0]
minus150
minus150
minus100
minus100minus50 minus500
0
50
50
100
100024681012
15
3
45
6
75
9
105
12
119884
119883
550minus100
minus100minus50 minus500
0
50
50
100
1010
119884
119883
Figure 3 This is a two-dimensional surface plot of Griewangkrsquosfunction The global optima are at [0 0] where the functionevaluates to 0
An illustration of this function in two dimensions is seen inFigure 2 One can clearly see the multiple local minima in thecontours overlaid on the floor of the surface plot
Griewangkrsquos function is similar to Rastriginrsquos functionin that it also uses cosine modulation to create a highlymultimodal function [23] However Griewangkrsquos functionis optimised over a larger search space and has differentproperties as one gets closer to the global optima situated at119909119894= 0 forall119894When one considers the entire search space the function
looks like a simple multidimensional unimodal parabolawhich could easily be minimised using gradient descent Asone moves closer to the optimum and starts to considersmaller areas it becomes clear that the function is not assmooth as first thought and it is in fact highly multimodalThis is clearly illustrated in Figure 3 where a surface plot ofa two-dimensional implementation of Griewangkrsquos function
6 Journal of Applied Mathematics
is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows
119891 (x) =
119899
sum
119894=1
1199092
119894
4000minus
119899
prod
119894=1
cos(119909119894
radic119894
) + 1
minus 600 le 119909119894le 600
(8)
Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation
119891 (x) = 20 exp(minus02radic1
119899
119899
sum
119894=1
1199092
119894)
minus exp(1
119899
119899
sum
119894=1
cos (2120587119909119894)) + 20 + 119890
minus 32768 le 119909119894le 32768
(9)
The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909
1 1199092
le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial
119891 (x) = (1 + (1199091+ 1199092+ 1)2
times (19 minus 141199091+ 31199092
1minus 14119909
2+ 611990911199092+ 31199092
2))
times (30 + (21199091minus 31199092)2
times (18 minus 321199091+ 12119909
2
1+ 48119909
2minus 36119909
11199092+ 27119909
2
2))
minus 32768 le 119909119894le 32768
(10)
32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised
In our first experiment all HM variants were imple-mented using the parameters suggested by the original
minus15
minus15
minus10
minus10
minus5 minus500
5
5
10
10
5
10
15
20
25
5
75
10
125
15
175
20
119884
119883
Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0
minus2 minus2minus15
minus15
minus1minus1
minus05minus05
00
0505
11
1515
200000400000
600000
800000
10
100
1000
10000
100000
119884
119883
Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909
1 1199092le 2 cube
authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1
Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
2 Journal of Applied Mathematics
solution vectors and they suggested a method of adaptingPAR and FW to the relative progress of the optimiser insteadof keeping these parameters constant through all iterationsTheir algorithm alleviates the problem of choosing an exactvalue for the PAR and FW but it still requires that a range ofvalues specified by aminimum andmaximum value be given
Another early approach to improving HSrsquos performanceand solving the parameter setting problem is the global-bestharmony search (GHS) algorithm [10] It was inspired bythe swarm intelligence concept used in the particle swarmoptimisation algorithm (PSO) [11] Instead of using the FWparameter to adjust possible solutions closer to an optimumGHSmoves component values toward the value of the currentbest solution in the population
Both IHS and GHS improved on the performance of HSand alleviated the parameter setting problem to some degreeHowever just in the last few years many more HS variantswere developed that have shown to be superior to theseearly variants and some require even less manual parametersetting Most of these new variants are inspired by otherevolutionary algorithms and they combine the best aspectsof the simplex algorithm simulated annealing differentialevolution sequential quadratic programming and manyother aspects with HS to form new HS variants [12ndash16] Fora thorough summery of the latest research in HS variants seethe review article by Alia and Mandava [17]
In this paper we concentrate on the comparison ofthree of the most recent and promising HS variantsexploratory harmony search (EHS) self-adaptive harmonysearch (SAHS) and dynamic local-best harmony search(DLHS) All three improve on both the performance of theoriginal HS and many of the earlier HS variants We alsochose these variants because they were not developed ashybrid algorithms by combining HS with other optimisationalgorithms Instead the focuswas on investigating the steps ofHS and adding novelmodifications to key areas of the originalalgorithm
In the section that follows we briefly overview theHS algorithm and explain how the optimiser parametersinfluence the performance of the optimisation process Thisprovides the necessary background to introduce the threeHS variants in the sections that follow In Section 3 wecompare the performance of three HS variants using aset of well-known optimisation benchmark functions Theresults from these tests are then interpreted and discussedin Section 4 These observations then lead us to develop anovel hybrid algorithm called generalised adaptive harmonysearch (GAHS) that we introduce in Section 5 We concludein Section 6 with final remarks and our thoughts on futureresearch in the development of harmony search-based opti-misers
2 Harmony Search and OptimisersBased on Harmony Search
Theharmony search algorithm is ametaheuristic population-based optimisation algorithm inspired by the way musiciansin a band discover new harmonies through cooperative
improvisation [1] The analogy between the optimisationof an objective function and the improvisation of pleasingharmonies is explained by the actions of the musicians in theband Each musician corresponds to a decision variable inthe solution vector of the problem and is also a dimensionin the search space Each musician (decision variable) hasa different instrument whose pitch range corresponds to adecision variablersquos value range A solution vector also calledan improvisation at a certain iteration corresponds to themusical harmony at a certain time and the objective functioncorresponds to the audiencersquos aesthetics New improvisationsare based on previously remembered good ones representedby a data structure called the harmony memory (HM)
This analogy is common to all the HS variants that wewill investigate HS variants differ in the way solution vectors(improvisations) are generated and how the HM is updatedat the end of each iteration We start our explanation of howthis is done differently in each variant with an overview of theoriginal HS algorithm
21 An Overview of Harmony Search The core data structureof HS is a matrix of the best solution vectors called theharmony memory (HM) The number of vectors that aresimultaneously processed is known as the harmony memorysize (HMS) It is one of the algorithmrsquos parameters thathas to be set manually Memory is organised as a matrixwith each row representing a solution vector and the finalcolumn representing the vectorrsquos fitness In an119873-dimensionalproblem the HM would be represented as
[[[[[[[[
[
1199091
11199092
1sdot sdot sdot 119909
119873
1| 1199081
1199091
21199092
2sdot sdot sdot 119909
119873
2| 1199082
1199091
HMS 1199092
HMS sdot sdot sdot 119909119873
HMS | 119908HMS
]]]]]]]]
]
(1)
where each row consists of 119873 decision variables and the fit-ness score 119908 ([1199091 1199092 119909119873 119908]) Before optimisation startsthe HM is initialised with HMS randomly generated solutionvectors Depending on the problem these vectors can also berandomly chosen around a seed point that may represent anarea in the search space where the optimum is most likely tobe found [3]
The step after initialisation is called improvisation Anew solution is improvised by using three rules memoryconsideration pitch adjustment and random selection Eachdecision variable is improvised separately and any one ofthe three rules can be used for any variable The harmonymemory consideration rate (HMCR) is also one of the HSparameters that must be manually chosen It controls howoften the memory (HM) is taken into consideration dur-ing improvisation For standard HS memory considerationmeans that the decisions variablersquos value is chosen directlyfrom one of the solution vectors in the HM A randomnumber is generated for each decision variable If it is smallerthan the HMCR the memory is taken into considerationotherwise a value is randomly chosen from the range ofpossible values for that dimension
Journal of Applied Mathematics 3
If the HM is taken into consideration the improvisedvalue is chosen randomly from one of the values in the HMThe pitch adjustment rate (PAR) is set during initialisationand it controls the amount of pitch adjustment done whenmemory consideration is used Another random number isgenerated If it is smaller than the PAR the improvised valueis pitch adjusted using
1199091015840
new = 119909new + rand () sdot FW (2)
where 1199091015840
new is the new pitch-adjusted value 119909new is theold value chosen using memory consideration rand() is arandom value between minus1 and 1 and FW is the fret widthparameter The terminology has since been updated to betterreflect the musical analogy [5] that controls the maximumvariation in pitch adjustment and is one of the parameters thatmust be manually set
Once a new value has been improvised the memoryis updated by comparing the new improvisation with thevector in the memory with the lowest fitness If the newimprovisation has a higher fitness it replaces the vector withthe lowest fitness This process of improvisation and updatecontinues iteratively until some stopping criterion is fulfilledor the maximum number of iterations are reached
To summarise the main steps of the HS are as follows
(1) Initialise the HM with possible solutions from theentire search space of the function to be optimised
(2) Improvise a new solution vector using memory con-sideration pitch adjustment and random selection
(3) If the new improvisation is better than the worstsolution in the HM replace the worst solution withthe new improvisation
(4) Check the stopping criteria If it has not been metrepeat Steps (2) and (3) or else proceed to Step (5)
(5) Return the best solution (highest fitness) in the HMas the optimal solution vector of the function
This is a brief overview of HS The interested reader isreferred to [1 5] for a more detailed explanation
22 Exploratory Harmony Search The exploratory harmonysearch (EHS) algorithm was the result of improving theperformance of HS by focussing on the evolution of thepopulation variance in theHMduring optimisationDas et alrealised that the exploratory power of HS can be maximisedby dynamically adjusting the FW to be proportional to thestandard deviation of the HM population [18]
This realisation comes from the following lemma that isproven in [18]
Lemma 1 If the HMCR is chosen to be high (ie very nearto 1) and the FW is chosen to be proportional to the standarddeviation of the HM (ie FW prop 120590(HM) = radicVar(HM)) thenthe expected population variance (without updating the HM)can grow exponentially over iterations
This lemma led them to redefine the FW as a dynamicallyadjusting parameter defined by FW = 119896radicVar(HM) where 119896
is a proportionality constantTheFWis therefore recalculatedat each iteration based on the current HM variance insteadof using a constant value that must be chosen manually
To ensure exponential variance growth the PAR HMCRand 119896 must also be chosen such that ((HMS minus 1)HMS) sdot
HMCR sdot [1 + (13)1198962sdot PAR] is greater than 1 The proof of
this statement can also be found in [18]This simple change to HS results in an algorithm that
is less likely to get stuck in local optima requires lessiterations to reach a desired accuracy and frees the userfrom determining the best value for the FW parameterHowever one could argue that the FW parameter was simplyreplaced by the proportionality constant 119896 In their articleDas et al investigate the effect that variations in 119896 have onthe performance of EHS They determined that EHS is notsensitive to the choice of 119896 and used a constant value (119896 =
117) in all their experimentsThis contrastswith the choice ofthe FW inHS that is usually chosen to fit a particular objectivefunction and can have a significant impact on performance ifchosen poorly
23 Self-AdaptiveHarmony Search In 2010Wang andHuangdeveloped a variant of HS that we refer to as the self-adaptiveharmony search (SAHS) algorithm [19]Theirmain focuswasto alleviate the problem of choosing the best HS parametervalues for a specific problem SAHS does not require theFW and PAR parameters that need to be specified whenusing HS However SAHS like IHS requires that a minimumand maximum value for PAR be specified Specifying aminimum and maximum for PAR is typically much easierthan specifying an exact value and it is more reliable sincethe performance is not as sensitive to changes in the range ofPAR values as it is for a specific constant value
SAHS is somewhat similar to IHS in that it also replacesthe PAR with a range of PAR values Like IHS it alsorecalculates the current value of the PAR by linearly adjustingit between the minimum and maximum values based on theiteration count However it differs in that it decreases the PARby starting at the maximum value and linearly decreasing ituntil the minimum is reached at the final iteration instead ofincreasing it like IHS does
Unlike IHS the FW parameter is not replaced by a rangeof FW values Instead it is replaced by a novel method ofpitch adjustment that does not require an FW parameterWhen pitch adjustment of an improvised decision variableis required it is randomly adjusted for that decision variablebetween the minimum and maximum values found in thecurrent HM using the following equations
1199091015840
119894larr997888 1199091015840
119894+ [max (HM119894) minus 119909
1015840
119894] sdot 119903
1199091015840
119894larr997888 1199091015840
119894minus [1199091015840
119894minus min (HM119894)] sdot 119903
(3)
where 1199091015840119894is the new improvisationmax(HM119894) andmin(HM119894)
are the maximum and minimum values in the HM forthe 119894th decision variables and 119903 is a uniformly generatedrandomnumber between 0 and 1 Each time pitch adjustmentis performed the improvisation is updated by randomlyapplying with equal probability one of these two equations
4 Journal of Applied Mathematics
This approach causes progressively smaller changes to bemade to the new improvisation as max(HM119894) and min(HM119894)converge closer togetherwith increasing iterationsThereforepitch adjustment begins as a rough operator making largerchanges to favour exploration and then becomes a fineoperator favouring exploitation as the optimiser convergescloser to the optimum
Another important difference between SAHS and HS isin the initialisation of the HM In HS the HM is initialisedone decision variable at a time by randomly picking avalue from a uniform distribution of values in the allowablerange However this uniform distribution is only an approx-imation created by scaling the output from a pseudoran-dom number generator to the correct range of values Theresulting distribution is often unevenly distributed causingslow convergence or convergence to local optima Insteadof using a pseudorandom number generator SAHS uses alow discrepancy sequence [20] to initialise the HM Lowdiscrepancy sequences are more evenly spread out in thesearch space than pseudorandom ones causing the initialvectors in the HM to better represent the search space of theproblem
24 Dynamic Local-Best Harmony Search dynamic local-best harmony search (DLHS) was developed by Pan et al asan improvement to IHS and GHS that does not require theHMCR and PAR parameters [21] The focus of DLHS can beroughly divided into three categories First the improvisationstep was improved by basing new improvisations on thecurrent best solution vector in the HM instead of a randomlychosen one Second it divides theHM intomultiple indepen-dently optimised sub-HMs in an attempt to maintain greaterdiversity throughout the search process Third DLHS usesa self-learning parameter set list (PSL) to evolve the HMCRand PAR parameters into the optimal values without any userspecificationThe FW parameter is dynamically adjusted in asimilar way to the method that IHS uses
The modification of the improvisation process is inspiredby GHS Like GHS the random solution vector used inmemory consideration is replaced by the best solution vectorin the HM This tends to concentrate future improvisationsaround the current best solution which is likely a local opti-mum and it may be the global optimum To prevent possibleconvergence around local optima the random solution vectorthat was discarded in memory consideration is instead usedduring pitch adjustment So unlike the pitch adjustment of(2) DLHS uses the following equation
1199091015840
new = 119909rand + rand () sdot FW (4)
where 119909rand is randomly chosen from the current values inthe HM for a particular decision variable
Before improvisation starts the HM is randomly dividedinto 119898 equally sized sub-HMs Each sub-HM then usesits own small subset of solution vectors to independentlyconverge onto a local (or possibly global) optimum Dueto the loss of diversity that comes with restricting the sizeof the individual HMs by splitting them up into sub-HMsit is likely that convergence will not be towards the global
optimum but rather toward a local one To prevent this lossof diversity information is allowed to exchange between sub-HMs with a frequency controlled by the regrouping schedule119877 The regrouping schedule determines how often informa-tion exchange is allowed between sub-HMs by randomlyregrouping the individual solution vectors of all sub-HMsinto 119898 new sub-HM configurations every 119877 iteration Thisregrouping simultaneously increases the diversity of all sub-HMs and allows the best solutions from the entire HM to beshared among sub-HMs
Once the global optimum is identified the injection ofdiversity through the regrouping operation does not furtheroptimise but it can result in inaccurate convergence Forthis reason DLHS enters the final phase of the optimisationprocess after 90 of the iterations have been made In thisfinal phase the regrouping operation is halted and a newHM is formed by combining the best three solution vectorsfrom all sub-HMs into a single new HM The new HM isthen exclusively processed until the maximum number ofiterations has been reached
The HMCR and PAR parameter values are dynamicallydetermined by selecting from a self-learning parameter setlist (PSL)This process starts with the initialisation of the PSLby filling it with randomly generated HMCR and PAR valuesHMCR values are generated from a uniform distributionwhere HMCR isin [09 10] and PAR values are generatedfrom a uniform distribution where PAR isin [00 10] At thestart of each iteration one pair of HMCR and PAR valuesis removed from the PSL and used for that iteration If thecurrent iterationrsquos improvisation resulted in the HM beingupdated the current parameter pair is saved in the winningparameter set list (WPSL) This process continues until thePSL becomes empty The PSL is then refilled by randomlyselecting a pair from theWPSL 75of the time and randomlygenerating a new pair 25 of the time To prevent memoryeffects in the WPSL the WPSL is emptied each time the PSLis refilled The result of this is that the best parameter set isgradually learned and it is specific to the objective functionunder consideration The size of the PSL is set to 200 pairsIt was found that the effect that the size of the PSL has onperformance is insignificant [21]
Unlike the HMCR and the PAR the FW parameter isdynamically adjusted in a way that is similar to the wayIHS adjusts the PAR Like IHS DLHS favours a large FWduring the early iterations to encourage exploration of thesearch space while a small FW is favoured during the finaliterations to better exploit good solutions in the HMThe FWis therefore linearly decreasedwith increasing iterations usingthe following equation
FW (119894) =
FWmax minusFWmax minus FWmin
MI2119894 if 119894 lt
MI2
FWmin otherwise(5)
where FWmax and FWmin are the maximum and minimumvalues of the FW 119894 is the iteration number and MI is themaximum number of iterations Like IHS this means thata minimum and maximum value for the FW has to be setbefore optimisation starts but this is again assumed to bemuch easier than deciding on a single value for the FW
Journal of Applied Mathematics 5
221
100minus1
minus1minus2
minus2
minus3 minus3
119884
119883
2000400060008000
100001200014000
1
10
100
1000
10000
Figure 1 Rosenbrockrsquos valley is illustrated here in two dimensionsNotice the parabolic valley that includes the global optimum at[1 1]
3 Performance Analysis Using FiveBenchmark Functions
31 Benchmark Functions We start our performance analysiswith a brief description of the five benchmark functionsthat we will be using to compare performance All five ofthese functions were designed to be challenging to optimisefor different reasons and all have previously been used forbenchmarking harmony search-based algorithms
We start with a classic parametric optimisation functioncalled Rosenbrockrsquos valley [22] This multidimensional prob-lem has a global optimum inside a long narrow parabolic-shaped flat valley The valley itself is easy to find since thereare no other local minima anywhere in the search space butconverging onto the global optimum is difficult Rosenbrockrsquosvalley is defined by the following equation
119891 (x) =
119899minus1
sum
119894=1
100 (119909119894+1
minus 1199092
119894) + (1 minus 119909
119894)2
minus 2048 le 119909119894le 2048
(6)
Rosenbrockrsquos valley has a global optimum at 119909119894= 1 forall119894 where
the function evaluates to 0 An illustration of Rosenbrockrsquosvalley implemented in two dimensions is seen in Figure 1
Rastriginrsquos function uses cosine modulation to create ahighly multimodal function that often causes optimisationalgorithms to get stuck in the local optima without everfinding the global optimum [23] However it does have asingle global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Rastriginrsquos function is defined by
119891 (x) = 10119899 +
119899
sum
119894=1
(1199092
119894minus 10 cos (2120587119909
119894)) minus 512 le 119909
119894le 512
(7)
minus4
minus44
4
minus2minus2
2
2
00 10
10
20
2030
30 4040
5050
6060
70
70
80
80
119884
119883
Figure 2 This illustration of Rastriginrsquos function shows its highlymultimodal nature and it is difficult to visually identify the globaloptima at [0 0]
minus150
minus150
minus100
minus100minus50 minus500
0
50
50
100
100024681012
15
3
45
6
75
9
105
12
119884
119883
550minus100
minus100minus50 minus500
0
50
50
100
1010
119884
119883
Figure 3 This is a two-dimensional surface plot of Griewangkrsquosfunction The global optima are at [0 0] where the functionevaluates to 0
An illustration of this function in two dimensions is seen inFigure 2 One can clearly see the multiple local minima in thecontours overlaid on the floor of the surface plot
Griewangkrsquos function is similar to Rastriginrsquos functionin that it also uses cosine modulation to create a highlymultimodal function [23] However Griewangkrsquos functionis optimised over a larger search space and has differentproperties as one gets closer to the global optima situated at119909119894= 0 forall119894When one considers the entire search space the function
looks like a simple multidimensional unimodal parabolawhich could easily be minimised using gradient descent Asone moves closer to the optimum and starts to considersmaller areas it becomes clear that the function is not assmooth as first thought and it is in fact highly multimodalThis is clearly illustrated in Figure 3 where a surface plot ofa two-dimensional implementation of Griewangkrsquos function
6 Journal of Applied Mathematics
is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows
119891 (x) =
119899
sum
119894=1
1199092
119894
4000minus
119899
prod
119894=1
cos(119909119894
radic119894
) + 1
minus 600 le 119909119894le 600
(8)
Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation
119891 (x) = 20 exp(minus02radic1
119899
119899
sum
119894=1
1199092
119894)
minus exp(1
119899
119899
sum
119894=1
cos (2120587119909119894)) + 20 + 119890
minus 32768 le 119909119894le 32768
(9)
The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909
1 1199092
le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial
119891 (x) = (1 + (1199091+ 1199092+ 1)2
times (19 minus 141199091+ 31199092
1minus 14119909
2+ 611990911199092+ 31199092
2))
times (30 + (21199091minus 31199092)2
times (18 minus 321199091+ 12119909
2
1+ 48119909
2minus 36119909
11199092+ 27119909
2
2))
minus 32768 le 119909119894le 32768
(10)
32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised
In our first experiment all HM variants were imple-mented using the parameters suggested by the original
minus15
minus15
minus10
minus10
minus5 minus500
5
5
10
10
5
10
15
20
25
5
75
10
125
15
175
20
119884
119883
Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0
minus2 minus2minus15
minus15
minus1minus1
minus05minus05
00
0505
11
1515
200000400000
600000
800000
10
100
1000
10000
100000
119884
119883
Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909
1 1199092le 2 cube
authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1
Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Applied Mathematics 3
If the HM is taken into consideration the improvisedvalue is chosen randomly from one of the values in the HMThe pitch adjustment rate (PAR) is set during initialisationand it controls the amount of pitch adjustment done whenmemory consideration is used Another random number isgenerated If it is smaller than the PAR the improvised valueis pitch adjusted using
1199091015840
new = 119909new + rand () sdot FW (2)
where 1199091015840
new is the new pitch-adjusted value 119909new is theold value chosen using memory consideration rand() is arandom value between minus1 and 1 and FW is the fret widthparameter The terminology has since been updated to betterreflect the musical analogy [5] that controls the maximumvariation in pitch adjustment and is one of the parameters thatmust be manually set
Once a new value has been improvised the memoryis updated by comparing the new improvisation with thevector in the memory with the lowest fitness If the newimprovisation has a higher fitness it replaces the vector withthe lowest fitness This process of improvisation and updatecontinues iteratively until some stopping criterion is fulfilledor the maximum number of iterations are reached
To summarise the main steps of the HS are as follows
(1) Initialise the HM with possible solutions from theentire search space of the function to be optimised
(2) Improvise a new solution vector using memory con-sideration pitch adjustment and random selection
(3) If the new improvisation is better than the worstsolution in the HM replace the worst solution withthe new improvisation
(4) Check the stopping criteria If it has not been metrepeat Steps (2) and (3) or else proceed to Step (5)
(5) Return the best solution (highest fitness) in the HMas the optimal solution vector of the function
This is a brief overview of HS The interested reader isreferred to [1 5] for a more detailed explanation
22 Exploratory Harmony Search The exploratory harmonysearch (EHS) algorithm was the result of improving theperformance of HS by focussing on the evolution of thepopulation variance in theHMduring optimisationDas et alrealised that the exploratory power of HS can be maximisedby dynamically adjusting the FW to be proportional to thestandard deviation of the HM population [18]
This realisation comes from the following lemma that isproven in [18]
Lemma 1 If the HMCR is chosen to be high (ie very nearto 1) and the FW is chosen to be proportional to the standarddeviation of the HM (ie FW prop 120590(HM) = radicVar(HM)) thenthe expected population variance (without updating the HM)can grow exponentially over iterations
This lemma led them to redefine the FW as a dynamicallyadjusting parameter defined by FW = 119896radicVar(HM) where 119896
is a proportionality constantTheFWis therefore recalculatedat each iteration based on the current HM variance insteadof using a constant value that must be chosen manually
To ensure exponential variance growth the PAR HMCRand 119896 must also be chosen such that ((HMS minus 1)HMS) sdot
HMCR sdot [1 + (13)1198962sdot PAR] is greater than 1 The proof of
this statement can also be found in [18]This simple change to HS results in an algorithm that
is less likely to get stuck in local optima requires lessiterations to reach a desired accuracy and frees the userfrom determining the best value for the FW parameterHowever one could argue that the FW parameter was simplyreplaced by the proportionality constant 119896 In their articleDas et al investigate the effect that variations in 119896 have onthe performance of EHS They determined that EHS is notsensitive to the choice of 119896 and used a constant value (119896 =
117) in all their experimentsThis contrastswith the choice ofthe FW inHS that is usually chosen to fit a particular objectivefunction and can have a significant impact on performance ifchosen poorly
23 Self-AdaptiveHarmony Search In 2010Wang andHuangdeveloped a variant of HS that we refer to as the self-adaptiveharmony search (SAHS) algorithm [19]Theirmain focuswasto alleviate the problem of choosing the best HS parametervalues for a specific problem SAHS does not require theFW and PAR parameters that need to be specified whenusing HS However SAHS like IHS requires that a minimumand maximum value for PAR be specified Specifying aminimum and maximum for PAR is typically much easierthan specifying an exact value and it is more reliable sincethe performance is not as sensitive to changes in the range ofPAR values as it is for a specific constant value
SAHS is somewhat similar to IHS in that it also replacesthe PAR with a range of PAR values Like IHS it alsorecalculates the current value of the PAR by linearly adjustingit between the minimum and maximum values based on theiteration count However it differs in that it decreases the PARby starting at the maximum value and linearly decreasing ituntil the minimum is reached at the final iteration instead ofincreasing it like IHS does
Unlike IHS the FW parameter is not replaced by a rangeof FW values Instead it is replaced by a novel method ofpitch adjustment that does not require an FW parameterWhen pitch adjustment of an improvised decision variableis required it is randomly adjusted for that decision variablebetween the minimum and maximum values found in thecurrent HM using the following equations
1199091015840
119894larr997888 1199091015840
119894+ [max (HM119894) minus 119909
1015840
119894] sdot 119903
1199091015840
119894larr997888 1199091015840
119894minus [1199091015840
119894minus min (HM119894)] sdot 119903
(3)
where 1199091015840119894is the new improvisationmax(HM119894) andmin(HM119894)
are the maximum and minimum values in the HM forthe 119894th decision variables and 119903 is a uniformly generatedrandomnumber between 0 and 1 Each time pitch adjustmentis performed the improvisation is updated by randomlyapplying with equal probability one of these two equations
4 Journal of Applied Mathematics
This approach causes progressively smaller changes to bemade to the new improvisation as max(HM119894) and min(HM119894)converge closer togetherwith increasing iterationsThereforepitch adjustment begins as a rough operator making largerchanges to favour exploration and then becomes a fineoperator favouring exploitation as the optimiser convergescloser to the optimum
Another important difference between SAHS and HS isin the initialisation of the HM In HS the HM is initialisedone decision variable at a time by randomly picking avalue from a uniform distribution of values in the allowablerange However this uniform distribution is only an approx-imation created by scaling the output from a pseudoran-dom number generator to the correct range of values Theresulting distribution is often unevenly distributed causingslow convergence or convergence to local optima Insteadof using a pseudorandom number generator SAHS uses alow discrepancy sequence [20] to initialise the HM Lowdiscrepancy sequences are more evenly spread out in thesearch space than pseudorandom ones causing the initialvectors in the HM to better represent the search space of theproblem
24 Dynamic Local-Best Harmony Search dynamic local-best harmony search (DLHS) was developed by Pan et al asan improvement to IHS and GHS that does not require theHMCR and PAR parameters [21] The focus of DLHS can beroughly divided into three categories First the improvisationstep was improved by basing new improvisations on thecurrent best solution vector in the HM instead of a randomlychosen one Second it divides theHM intomultiple indepen-dently optimised sub-HMs in an attempt to maintain greaterdiversity throughout the search process Third DLHS usesa self-learning parameter set list (PSL) to evolve the HMCRand PAR parameters into the optimal values without any userspecificationThe FW parameter is dynamically adjusted in asimilar way to the method that IHS uses
The modification of the improvisation process is inspiredby GHS Like GHS the random solution vector used inmemory consideration is replaced by the best solution vectorin the HM This tends to concentrate future improvisationsaround the current best solution which is likely a local opti-mum and it may be the global optimum To prevent possibleconvergence around local optima the random solution vectorthat was discarded in memory consideration is instead usedduring pitch adjustment So unlike the pitch adjustment of(2) DLHS uses the following equation
1199091015840
new = 119909rand + rand () sdot FW (4)
where 119909rand is randomly chosen from the current values inthe HM for a particular decision variable
Before improvisation starts the HM is randomly dividedinto 119898 equally sized sub-HMs Each sub-HM then usesits own small subset of solution vectors to independentlyconverge onto a local (or possibly global) optimum Dueto the loss of diversity that comes with restricting the sizeof the individual HMs by splitting them up into sub-HMsit is likely that convergence will not be towards the global
optimum but rather toward a local one To prevent this lossof diversity information is allowed to exchange between sub-HMs with a frequency controlled by the regrouping schedule119877 The regrouping schedule determines how often informa-tion exchange is allowed between sub-HMs by randomlyregrouping the individual solution vectors of all sub-HMsinto 119898 new sub-HM configurations every 119877 iteration Thisregrouping simultaneously increases the diversity of all sub-HMs and allows the best solutions from the entire HM to beshared among sub-HMs
Once the global optimum is identified the injection ofdiversity through the regrouping operation does not furtheroptimise but it can result in inaccurate convergence Forthis reason DLHS enters the final phase of the optimisationprocess after 90 of the iterations have been made In thisfinal phase the regrouping operation is halted and a newHM is formed by combining the best three solution vectorsfrom all sub-HMs into a single new HM The new HM isthen exclusively processed until the maximum number ofiterations has been reached
The HMCR and PAR parameter values are dynamicallydetermined by selecting from a self-learning parameter setlist (PSL)This process starts with the initialisation of the PSLby filling it with randomly generated HMCR and PAR valuesHMCR values are generated from a uniform distributionwhere HMCR isin [09 10] and PAR values are generatedfrom a uniform distribution where PAR isin [00 10] At thestart of each iteration one pair of HMCR and PAR valuesis removed from the PSL and used for that iteration If thecurrent iterationrsquos improvisation resulted in the HM beingupdated the current parameter pair is saved in the winningparameter set list (WPSL) This process continues until thePSL becomes empty The PSL is then refilled by randomlyselecting a pair from theWPSL 75of the time and randomlygenerating a new pair 25 of the time To prevent memoryeffects in the WPSL the WPSL is emptied each time the PSLis refilled The result of this is that the best parameter set isgradually learned and it is specific to the objective functionunder consideration The size of the PSL is set to 200 pairsIt was found that the effect that the size of the PSL has onperformance is insignificant [21]
Unlike the HMCR and the PAR the FW parameter isdynamically adjusted in a way that is similar to the wayIHS adjusts the PAR Like IHS DLHS favours a large FWduring the early iterations to encourage exploration of thesearch space while a small FW is favoured during the finaliterations to better exploit good solutions in the HMThe FWis therefore linearly decreasedwith increasing iterations usingthe following equation
FW (119894) =
FWmax minusFWmax minus FWmin
MI2119894 if 119894 lt
MI2
FWmin otherwise(5)
where FWmax and FWmin are the maximum and minimumvalues of the FW 119894 is the iteration number and MI is themaximum number of iterations Like IHS this means thata minimum and maximum value for the FW has to be setbefore optimisation starts but this is again assumed to bemuch easier than deciding on a single value for the FW
Journal of Applied Mathematics 5
221
100minus1
minus1minus2
minus2
minus3 minus3
119884
119883
2000400060008000
100001200014000
1
10
100
1000
10000
Figure 1 Rosenbrockrsquos valley is illustrated here in two dimensionsNotice the parabolic valley that includes the global optimum at[1 1]
3 Performance Analysis Using FiveBenchmark Functions
31 Benchmark Functions We start our performance analysiswith a brief description of the five benchmark functionsthat we will be using to compare performance All five ofthese functions were designed to be challenging to optimisefor different reasons and all have previously been used forbenchmarking harmony search-based algorithms
We start with a classic parametric optimisation functioncalled Rosenbrockrsquos valley [22] This multidimensional prob-lem has a global optimum inside a long narrow parabolic-shaped flat valley The valley itself is easy to find since thereare no other local minima anywhere in the search space butconverging onto the global optimum is difficult Rosenbrockrsquosvalley is defined by the following equation
119891 (x) =
119899minus1
sum
119894=1
100 (119909119894+1
minus 1199092
119894) + (1 minus 119909
119894)2
minus 2048 le 119909119894le 2048
(6)
Rosenbrockrsquos valley has a global optimum at 119909119894= 1 forall119894 where
the function evaluates to 0 An illustration of Rosenbrockrsquosvalley implemented in two dimensions is seen in Figure 1
Rastriginrsquos function uses cosine modulation to create ahighly multimodal function that often causes optimisationalgorithms to get stuck in the local optima without everfinding the global optimum [23] However it does have asingle global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Rastriginrsquos function is defined by
119891 (x) = 10119899 +
119899
sum
119894=1
(1199092
119894minus 10 cos (2120587119909
119894)) minus 512 le 119909
119894le 512
(7)
minus4
minus44
4
minus2minus2
2
2
00 10
10
20
2030
30 4040
5050
6060
70
70
80
80
119884
119883
Figure 2 This illustration of Rastriginrsquos function shows its highlymultimodal nature and it is difficult to visually identify the globaloptima at [0 0]
minus150
minus150
minus100
minus100minus50 minus500
0
50
50
100
100024681012
15
3
45
6
75
9
105
12
119884
119883
550minus100
minus100minus50 minus500
0
50
50
100
1010
119884
119883
Figure 3 This is a two-dimensional surface plot of Griewangkrsquosfunction The global optima are at [0 0] where the functionevaluates to 0
An illustration of this function in two dimensions is seen inFigure 2 One can clearly see the multiple local minima in thecontours overlaid on the floor of the surface plot
Griewangkrsquos function is similar to Rastriginrsquos functionin that it also uses cosine modulation to create a highlymultimodal function [23] However Griewangkrsquos functionis optimised over a larger search space and has differentproperties as one gets closer to the global optima situated at119909119894= 0 forall119894When one considers the entire search space the function
looks like a simple multidimensional unimodal parabolawhich could easily be minimised using gradient descent Asone moves closer to the optimum and starts to considersmaller areas it becomes clear that the function is not assmooth as first thought and it is in fact highly multimodalThis is clearly illustrated in Figure 3 where a surface plot ofa two-dimensional implementation of Griewangkrsquos function
6 Journal of Applied Mathematics
is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows
119891 (x) =
119899
sum
119894=1
1199092
119894
4000minus
119899
prod
119894=1
cos(119909119894
radic119894
) + 1
minus 600 le 119909119894le 600
(8)
Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation
119891 (x) = 20 exp(minus02radic1
119899
119899
sum
119894=1
1199092
119894)
minus exp(1
119899
119899
sum
119894=1
cos (2120587119909119894)) + 20 + 119890
minus 32768 le 119909119894le 32768
(9)
The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909
1 1199092
le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial
119891 (x) = (1 + (1199091+ 1199092+ 1)2
times (19 minus 141199091+ 31199092
1minus 14119909
2+ 611990911199092+ 31199092
2))
times (30 + (21199091minus 31199092)2
times (18 minus 321199091+ 12119909
2
1+ 48119909
2minus 36119909
11199092+ 27119909
2
2))
minus 32768 le 119909119894le 32768
(10)
32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised
In our first experiment all HM variants were imple-mented using the parameters suggested by the original
minus15
minus15
minus10
minus10
minus5 minus500
5
5
10
10
5
10
15
20
25
5
75
10
125
15
175
20
119884
119883
Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0
minus2 minus2minus15
minus15
minus1minus1
minus05minus05
00
0505
11
1515
200000400000
600000
800000
10
100
1000
10000
100000
119884
119883
Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909
1 1199092le 2 cube
authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1
Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
4 Journal of Applied Mathematics
This approach causes progressively smaller changes to bemade to the new improvisation as max(HM119894) and min(HM119894)converge closer togetherwith increasing iterationsThereforepitch adjustment begins as a rough operator making largerchanges to favour exploration and then becomes a fineoperator favouring exploitation as the optimiser convergescloser to the optimum
Another important difference between SAHS and HS isin the initialisation of the HM In HS the HM is initialisedone decision variable at a time by randomly picking avalue from a uniform distribution of values in the allowablerange However this uniform distribution is only an approx-imation created by scaling the output from a pseudoran-dom number generator to the correct range of values Theresulting distribution is often unevenly distributed causingslow convergence or convergence to local optima Insteadof using a pseudorandom number generator SAHS uses alow discrepancy sequence [20] to initialise the HM Lowdiscrepancy sequences are more evenly spread out in thesearch space than pseudorandom ones causing the initialvectors in the HM to better represent the search space of theproblem
24 Dynamic Local-Best Harmony Search dynamic local-best harmony search (DLHS) was developed by Pan et al asan improvement to IHS and GHS that does not require theHMCR and PAR parameters [21] The focus of DLHS can beroughly divided into three categories First the improvisationstep was improved by basing new improvisations on thecurrent best solution vector in the HM instead of a randomlychosen one Second it divides theHM intomultiple indepen-dently optimised sub-HMs in an attempt to maintain greaterdiversity throughout the search process Third DLHS usesa self-learning parameter set list (PSL) to evolve the HMCRand PAR parameters into the optimal values without any userspecificationThe FW parameter is dynamically adjusted in asimilar way to the method that IHS uses
The modification of the improvisation process is inspiredby GHS Like GHS the random solution vector used inmemory consideration is replaced by the best solution vectorin the HM This tends to concentrate future improvisationsaround the current best solution which is likely a local opti-mum and it may be the global optimum To prevent possibleconvergence around local optima the random solution vectorthat was discarded in memory consideration is instead usedduring pitch adjustment So unlike the pitch adjustment of(2) DLHS uses the following equation
1199091015840
new = 119909rand + rand () sdot FW (4)
where 119909rand is randomly chosen from the current values inthe HM for a particular decision variable
Before improvisation starts the HM is randomly dividedinto 119898 equally sized sub-HMs Each sub-HM then usesits own small subset of solution vectors to independentlyconverge onto a local (or possibly global) optimum Dueto the loss of diversity that comes with restricting the sizeof the individual HMs by splitting them up into sub-HMsit is likely that convergence will not be towards the global
optimum but rather toward a local one To prevent this lossof diversity information is allowed to exchange between sub-HMs with a frequency controlled by the regrouping schedule119877 The regrouping schedule determines how often informa-tion exchange is allowed between sub-HMs by randomlyregrouping the individual solution vectors of all sub-HMsinto 119898 new sub-HM configurations every 119877 iteration Thisregrouping simultaneously increases the diversity of all sub-HMs and allows the best solutions from the entire HM to beshared among sub-HMs
Once the global optimum is identified the injection ofdiversity through the regrouping operation does not furtheroptimise but it can result in inaccurate convergence Forthis reason DLHS enters the final phase of the optimisationprocess after 90 of the iterations have been made In thisfinal phase the regrouping operation is halted and a newHM is formed by combining the best three solution vectorsfrom all sub-HMs into a single new HM The new HM isthen exclusively processed until the maximum number ofiterations has been reached
The HMCR and PAR parameter values are dynamicallydetermined by selecting from a self-learning parameter setlist (PSL)This process starts with the initialisation of the PSLby filling it with randomly generated HMCR and PAR valuesHMCR values are generated from a uniform distributionwhere HMCR isin [09 10] and PAR values are generatedfrom a uniform distribution where PAR isin [00 10] At thestart of each iteration one pair of HMCR and PAR valuesis removed from the PSL and used for that iteration If thecurrent iterationrsquos improvisation resulted in the HM beingupdated the current parameter pair is saved in the winningparameter set list (WPSL) This process continues until thePSL becomes empty The PSL is then refilled by randomlyselecting a pair from theWPSL 75of the time and randomlygenerating a new pair 25 of the time To prevent memoryeffects in the WPSL the WPSL is emptied each time the PSLis refilled The result of this is that the best parameter set isgradually learned and it is specific to the objective functionunder consideration The size of the PSL is set to 200 pairsIt was found that the effect that the size of the PSL has onperformance is insignificant [21]
Unlike the HMCR and the PAR the FW parameter isdynamically adjusted in a way that is similar to the wayIHS adjusts the PAR Like IHS DLHS favours a large FWduring the early iterations to encourage exploration of thesearch space while a small FW is favoured during the finaliterations to better exploit good solutions in the HMThe FWis therefore linearly decreasedwith increasing iterations usingthe following equation
FW (119894) =
FWmax minusFWmax minus FWmin
MI2119894 if 119894 lt
MI2
FWmin otherwise(5)
where FWmax and FWmin are the maximum and minimumvalues of the FW 119894 is the iteration number and MI is themaximum number of iterations Like IHS this means thata minimum and maximum value for the FW has to be setbefore optimisation starts but this is again assumed to bemuch easier than deciding on a single value for the FW
Journal of Applied Mathematics 5
221
100minus1
minus1minus2
minus2
minus3 minus3
119884
119883
2000400060008000
100001200014000
1
10
100
1000
10000
Figure 1 Rosenbrockrsquos valley is illustrated here in two dimensionsNotice the parabolic valley that includes the global optimum at[1 1]
3 Performance Analysis Using FiveBenchmark Functions
31 Benchmark Functions We start our performance analysiswith a brief description of the five benchmark functionsthat we will be using to compare performance All five ofthese functions were designed to be challenging to optimisefor different reasons and all have previously been used forbenchmarking harmony search-based algorithms
We start with a classic parametric optimisation functioncalled Rosenbrockrsquos valley [22] This multidimensional prob-lem has a global optimum inside a long narrow parabolic-shaped flat valley The valley itself is easy to find since thereare no other local minima anywhere in the search space butconverging onto the global optimum is difficult Rosenbrockrsquosvalley is defined by the following equation
119891 (x) =
119899minus1
sum
119894=1
100 (119909119894+1
minus 1199092
119894) + (1 minus 119909
119894)2
minus 2048 le 119909119894le 2048
(6)
Rosenbrockrsquos valley has a global optimum at 119909119894= 1 forall119894 where
the function evaluates to 0 An illustration of Rosenbrockrsquosvalley implemented in two dimensions is seen in Figure 1
Rastriginrsquos function uses cosine modulation to create ahighly multimodal function that often causes optimisationalgorithms to get stuck in the local optima without everfinding the global optimum [23] However it does have asingle global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Rastriginrsquos function is defined by
119891 (x) = 10119899 +
119899
sum
119894=1
(1199092
119894minus 10 cos (2120587119909
119894)) minus 512 le 119909
119894le 512
(7)
minus4
minus44
4
minus2minus2
2
2
00 10
10
20
2030
30 4040
5050
6060
70
70
80
80
119884
119883
Figure 2 This illustration of Rastriginrsquos function shows its highlymultimodal nature and it is difficult to visually identify the globaloptima at [0 0]
minus150
minus150
minus100
minus100minus50 minus500
0
50
50
100
100024681012
15
3
45
6
75
9
105
12
119884
119883
550minus100
minus100minus50 minus500
0
50
50
100
1010
119884
119883
Figure 3 This is a two-dimensional surface plot of Griewangkrsquosfunction The global optima are at [0 0] where the functionevaluates to 0
An illustration of this function in two dimensions is seen inFigure 2 One can clearly see the multiple local minima in thecontours overlaid on the floor of the surface plot
Griewangkrsquos function is similar to Rastriginrsquos functionin that it also uses cosine modulation to create a highlymultimodal function [23] However Griewangkrsquos functionis optimised over a larger search space and has differentproperties as one gets closer to the global optima situated at119909119894= 0 forall119894When one considers the entire search space the function
looks like a simple multidimensional unimodal parabolawhich could easily be minimised using gradient descent Asone moves closer to the optimum and starts to considersmaller areas it becomes clear that the function is not assmooth as first thought and it is in fact highly multimodalThis is clearly illustrated in Figure 3 where a surface plot ofa two-dimensional implementation of Griewangkrsquos function
6 Journal of Applied Mathematics
is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows
119891 (x) =
119899
sum
119894=1
1199092
119894
4000minus
119899
prod
119894=1
cos(119909119894
radic119894
) + 1
minus 600 le 119909119894le 600
(8)
Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation
119891 (x) = 20 exp(minus02radic1
119899
119899
sum
119894=1
1199092
119894)
minus exp(1
119899
119899
sum
119894=1
cos (2120587119909119894)) + 20 + 119890
minus 32768 le 119909119894le 32768
(9)
The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909
1 1199092
le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial
119891 (x) = (1 + (1199091+ 1199092+ 1)2
times (19 minus 141199091+ 31199092
1minus 14119909
2+ 611990911199092+ 31199092
2))
times (30 + (21199091minus 31199092)2
times (18 minus 321199091+ 12119909
2
1+ 48119909
2minus 36119909
11199092+ 27119909
2
2))
minus 32768 le 119909119894le 32768
(10)
32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised
In our first experiment all HM variants were imple-mented using the parameters suggested by the original
minus15
minus15
minus10
minus10
minus5 minus500
5
5
10
10
5
10
15
20
25
5
75
10
125
15
175
20
119884
119883
Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0
minus2 minus2minus15
minus15
minus1minus1
minus05minus05
00
0505
11
1515
200000400000
600000
800000
10
100
1000
10000
100000
119884
119883
Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909
1 1199092le 2 cube
authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1
Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Applied Mathematics 5
221
100minus1
minus1minus2
minus2
minus3 minus3
119884
119883
2000400060008000
100001200014000
1
10
100
1000
10000
Figure 1 Rosenbrockrsquos valley is illustrated here in two dimensionsNotice the parabolic valley that includes the global optimum at[1 1]
3 Performance Analysis Using FiveBenchmark Functions
31 Benchmark Functions We start our performance analysiswith a brief description of the five benchmark functionsthat we will be using to compare performance All five ofthese functions were designed to be challenging to optimisefor different reasons and all have previously been used forbenchmarking harmony search-based algorithms
We start with a classic parametric optimisation functioncalled Rosenbrockrsquos valley [22] This multidimensional prob-lem has a global optimum inside a long narrow parabolic-shaped flat valley The valley itself is easy to find since thereare no other local minima anywhere in the search space butconverging onto the global optimum is difficult Rosenbrockrsquosvalley is defined by the following equation
119891 (x) =
119899minus1
sum
119894=1
100 (119909119894+1
minus 1199092
119894) + (1 minus 119909
119894)2
minus 2048 le 119909119894le 2048
(6)
Rosenbrockrsquos valley has a global optimum at 119909119894= 1 forall119894 where
the function evaluates to 0 An illustration of Rosenbrockrsquosvalley implemented in two dimensions is seen in Figure 1
Rastriginrsquos function uses cosine modulation to create ahighly multimodal function that often causes optimisationalgorithms to get stuck in the local optima without everfinding the global optimum [23] However it does have asingle global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Rastriginrsquos function is defined by
119891 (x) = 10119899 +
119899
sum
119894=1
(1199092
119894minus 10 cos (2120587119909
119894)) minus 512 le 119909
119894le 512
(7)
minus4
minus44
4
minus2minus2
2
2
00 10
10
20
2030
30 4040
5050
6060
70
70
80
80
119884
119883
Figure 2 This illustration of Rastriginrsquos function shows its highlymultimodal nature and it is difficult to visually identify the globaloptima at [0 0]
minus150
minus150
minus100
minus100minus50 minus500
0
50
50
100
100024681012
15
3
45
6
75
9
105
12
119884
119883
550minus100
minus100minus50 minus500
0
50
50
100
1010
119884
119883
Figure 3 This is a two-dimensional surface plot of Griewangkrsquosfunction The global optima are at [0 0] where the functionevaluates to 0
An illustration of this function in two dimensions is seen inFigure 2 One can clearly see the multiple local minima in thecontours overlaid on the floor of the surface plot
Griewangkrsquos function is similar to Rastriginrsquos functionin that it also uses cosine modulation to create a highlymultimodal function [23] However Griewangkrsquos functionis optimised over a larger search space and has differentproperties as one gets closer to the global optima situated at119909119894= 0 forall119894When one considers the entire search space the function
looks like a simple multidimensional unimodal parabolawhich could easily be minimised using gradient descent Asone moves closer to the optimum and starts to considersmaller areas it becomes clear that the function is not assmooth as first thought and it is in fact highly multimodalThis is clearly illustrated in Figure 3 where a surface plot ofa two-dimensional implementation of Griewangkrsquos function
6 Journal of Applied Mathematics
is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows
119891 (x) =
119899
sum
119894=1
1199092
119894
4000minus
119899
prod
119894=1
cos(119909119894
radic119894
) + 1
minus 600 le 119909119894le 600
(8)
Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation
119891 (x) = 20 exp(minus02radic1
119899
119899
sum
119894=1
1199092
119894)
minus exp(1
119899
119899
sum
119894=1
cos (2120587119909119894)) + 20 + 119890
minus 32768 le 119909119894le 32768
(9)
The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909
1 1199092
le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial
119891 (x) = (1 + (1199091+ 1199092+ 1)2
times (19 minus 141199091+ 31199092
1minus 14119909
2+ 611990911199092+ 31199092
2))
times (30 + (21199091minus 31199092)2
times (18 minus 321199091+ 12119909
2
1+ 48119909
2minus 36119909
11199092+ 27119909
2
2))
minus 32768 le 119909119894le 32768
(10)
32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised
In our first experiment all HM variants were imple-mented using the parameters suggested by the original
minus15
minus15
minus10
minus10
minus5 minus500
5
5
10
10
5
10
15
20
25
5
75
10
125
15
175
20
119884
119883
Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0
minus2 minus2minus15
minus15
minus1minus1
minus05minus05
00
0505
11
1515
200000400000
600000
800000
10
100
1000
10000
100000
119884
119883
Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909
1 1199092le 2 cube
authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1
Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
6 Journal of Applied Mathematics
is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows
119891 (x) =
119899
sum
119894=1
1199092
119894
4000minus
119899
prod
119894=1
cos(119909119894
radic119894
) + 1
minus 600 le 119909119894le 600
(8)
Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909
119894= 0 forall119894 where the function
evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation
119891 (x) = 20 exp(minus02radic1
119899
119899
sum
119894=1
1199092
119894)
minus exp(1
119899
119899
sum
119894=1
cos (2120587119909119894)) + 20 + 119890
minus 32768 le 119909119894le 32768
(9)
The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909
1 1199092
le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial
119891 (x) = (1 + (1199091+ 1199092+ 1)2
times (19 minus 141199091+ 31199092
1minus 14119909
2+ 611990911199092+ 31199092
2))
times (30 + (21199091minus 31199092)2
times (18 minus 321199091+ 12119909
2
1+ 48119909
2minus 36119909
11199092+ 27119909
2
2))
minus 32768 le 119909119894le 32768
(10)
32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised
In our first experiment all HM variants were imple-mented using the parameters suggested by the original
minus15
minus15
minus10
minus10
minus5 minus500
5
5
10
10
5
10
15
20
25
5
75
10
125
15
175
20
119884
119883
Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0
minus2 minus2minus15
minus15
minus1minus1
minus05minus05
00
0505
11
1515
200000400000
600000
800000
10
100
1000
10000
100000
119884
119883
Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909
1 1199092le 2 cube
authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1
Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Applied Mathematics 7
Table 1 Parameter sets as suggested by the original authors
HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values
then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance
Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3
Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum
A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the
0 10 20 30 40 50 600
5
10
15
20
25
30
35
Runs
Opt
imal
val
ue
Optimiser progress
Run optima
Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum
original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used
Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant
The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS
4 Interpretation of Results
In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
8 Journal of Applied Mathematics
Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold
Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger
HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42
DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44
EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48
AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50
daggerGoldstein-Price was implemented in two dimensions as it is only defined in two
Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations
Ackley Griewank Rosenbrock Rastrigindagger
HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0
DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32
EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0
AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function
than EHS and AHS on Rastrigin when it is worse in all otherexamples
We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch
adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Applied Mathematics 9
aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence
The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction
AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function
DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]
The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence
The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence
In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide
how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function
Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend
However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]
In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10
Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW
It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
10 Journal of Applied Mathematics
problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered
5 Generalised Adaptive Harmony Search
Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions
We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence
Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found
A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised
We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result
is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently
This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation
Wasted iterations = PLPSCSsum
119894=2
(119894 minus 1) (11)
To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured
Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum
The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison
It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers
It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Applied Mathematics 11
Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold
Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price
HSBest 00142 00050 1625829 02280 68464Standard deviation 00004 00080 495829 00130 93353Hits 50 34 0 0 42
DLHSBest 02281 00004 1424515 13140 62444Standard deviation 00130 00070 395126 31364 87724Hits 50 50 0 32 44
EHSBest 00009 00001 972555 3882894 40809Standard deviation 00006 00007 74711 205949 52910Hits 50 50 0 0 48
AHSBest 00003 0 964333 67471 3Standard deviation 00006 0 02943 23279 0Hits 50 50 0 0 50
GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50
daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two
one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise
Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations
An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge
6 Conclusions
After investigating the results from comparing some of thebest performing HS variants available today we noticed that
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
12 Journal of Applied Mathematics
0 100000 200000 300000 400000 5000000
5
10
15
20
25
Fitn
ess v
alue
Optimiser progress
This is the HMS = 50 instance that gets eliminated
HM maximumHM minimum
Iterations
Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line
the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected
We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum
However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony
search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser
Acknowledgment
This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)
References
[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001
[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009
[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010
[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010
[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010
[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010
[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005
[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany
[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007
[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008
[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995
[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Applied Mathematics 13
[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009
[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010
[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008
[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009
[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011
[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011
[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010
[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992
[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010
[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960
[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989
[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987
[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008
[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999
[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of