Multiple-Try Simulated Annealing Algorithm for Global ...

Research ArticleMultiple-Try Simulated Annealing Algorithm forGlobal Optimization

Wei Shao 1 and Guangbao Guo 2

1School of Management Qufu Normal University Rizhao Shandong 276826 China2Department of Statistics Shandong University of Technology Zibo 255000 China

Correspondence should be addressed to Wei Shao wshao1031gmailcom

Received 18 March 2018 Revised 23 May 2018 Accepted 29 May 2018 Published 17 July 2018

Academic Editor Guillermo Cabrera-Guerrero

Copyright copy 2018 Wei Shao andGuangbaoGuoThis is an open access article distributed under the Creative CommonsAttributionLicense which permits unrestricted use distribution and reproduction in anymedium provided the originalwork is properly cited

Simulated annealing is a widely used algorithm for the computation of global optimization problems in computational chemistryand industrial engineering However global optimum values cannot always be reached by simulated annealing without alogarithmic cooling schedule In this study we propose a new stochastic optimization algorithm ie simulated annealing basedon the multiple-try Metropolis method which combines simulated annealing and the multiple-try Metropolis algorithm Theproposed algorithm functions with a rapidly decreasing schedule while guaranteeing global optimum values Simulated and realdata experiments including a mixture normal model and nonlinear Bayesian model indicate that the proposed algorithm cansignificantly outperform other approximated algorithms including simulated annealing and the quasi-Newton method

1 Introduction

Since the 21st century the modern computers have greatlyexpanded the scientific horizon by facilitating the studies oncomplicated systems such as computer engineering stochas-tic process and modern bioinformatics A large volumeof high dimensional data can easily be obtained but theirefficient computation and analysis present a significant chal-lenge

With the development of modern computers Markovchain Monte Carlo (MCMC) methods have enjoyed a enor-mous upsurge in interest over the last few years [1 2] Duringthe past two decades various advanced MCMC methodshave been developed to successfully compute different typesof problems (eg Bayesian analysis high dimensional inte-gral and combinational optimization) As an extension ofMCMCmethods the simulated annealing (SA) algorithm [1ndash3] has become increasingly popular since it was first intro-duced by Kirkpatrick et al (1983) As Monte Carlo methodsare not sensitive to the dimension of data sets the SA algo-rithm plays an important role in molecular physics compu-tational chemistry and computer science It has also beensuccessfully applied tomany complex optimization problems

Several improved optimization methods have been pro-posed recently [4 5] and successfully applied to polynomialand vector optimization problems minndashmax models and soon [6ndash10] Although Sun andWang (2013) discussed the errorbound for generalized linear complementarity problems allof these improved methods were designed for special opti-mization problems and not for global optimization problemsThe SA algorithm is a global optimization algorithm thatcan obtain global optimization results with slowly decreasingtemperature schedule However Holley et al (1989) pointedout that only with the use of a ldquologarithmicrdquo cooling schedulecould the SA algorithm converge to the globalminimumwithprobability one [11 12] Liang et al (2014) improved the SAalgorithm by introducing the simulated stochastic approxi-mation annealing (SAA) algorithm [13 14] Such algorithmcan work with a square-root cooling schedule in whichthe temperature can decrease much faster than that in aldquologarithmicrdquo cooling schedule Karagiannis et al (2017) ex-tended the SAA algorithm by using population Monte Carloideas and introduced the parallel and interacting stochasticapproximation annealing (PISAA) algorithm [15]

In the present study we propose a variation of the SAalgorithm ie the multiple-try Metropolis based simulated

HindawiMathematical Problems in EngineeringVolume 2018 Article ID 9248318 11 pageshttpsdoiorg10115520189248318

2 Mathematical Problems in Engineering

annealing (MTMSA) algorithm for global optimizationTheMTMSA algorithm is a combination of the SA algorithm andmultiple-try Metropolis (MTM) algorithm [16] The MTMalgorithm which allows several proposals from differentproposal distributions in the multiple-try step simultane-ously achieves a higher rate of convergence than the stan-dard Metropolis algorithm (eg random walk Metropolisalgorithm) [1 2 17] Thus the MTMSA algorithm canguarantee that global minima are reached with a rapidlydecreasing cooling schedule Comparing with PISAA whichshould run on multicore computer to their advantage theMTMSA often owns high convergent rate by use of theefficient vector operation with MATLAB or R on personaland super computer Simulation and real data examples showthat under the framework of the multiple-try algorithmthe MTMSA algorithm can reach global minima under arapidly decreasing cooling schedule relative to that of the SAalgorithm

The remainder of this paper is organized as followsSection 2 describes the framework of theMTMSA algorithmSection 3 illustrates the comparison of the MTMSA algo-rithm with other optimization methods in a mixture normalmodel Section 4 presents the application of the MTMSAalgorithm to Bayesian analysis and half-space depth compu-tation through real data sets Finally Section 5 summarizesthe conclusions derived from the study

2 MTMSA Algorithm

21 Overview of SA Algorithm The SA algorithm originatesfrom the annealing process which is a thermodynamicprocess used to attain a low energy state in condensed matterphysics [1] The process comprises two steps The first stateis the high temperature state in which solids transform intoliquid and particles move freely to ward ideal Then thetemperature drops to zero slowly and the movement of theparticles become restricted such that the desired structureis achieved Realizing that the Metropolis algorithm [18] canbe used to simulate the movements of particles Kirkpatricket al (1983) proposed a computer simulation based physicalannealing process ie the SA algorithm

Suppose our goal is to find the minimum value of ℎ(x)x isin 119863 This goal is equivalent to the search for the maximumvalue of expminusℎ(x)119879 x isin 119863 with any positive temperature119879 Let 1198791 gt 1198792 gt sdot sdot sdot gt 119879119896 gt sdot sdot sdot be a decreasingtemperature sequence with large 1198791 and lim119896997888rarr+infin119879119896 = 0 Inevery temperature 119879119896 we use the Metropolis-Hastings (MH)algorithm (or Gibbs sampling algorithm [19]) to update theMarkov chain 119873119896 times with 120587119896(x) prop expminusℎ(x)119879119896 as itsstationary distribution When 119896 is increasing an increasingnumber of samples concentrate in the maximum valuenearby The SA algorithm can be summarized as follows

(1) Initialize x(0) with starting temperature 1198791(2) At current temperature 119879119896 update the Markov chain119873119896 times with120587119896(x) as its stationary distribution and

transmit the last state x to the next temperature

(3) Update 119896 to 119896 + 1

The SA algorithm can reach the global optimum whenthe temperature sequence decreases slowly (eg the inverselogarithmic rate ie the order of 119874(log(119871119896)minus1) where 119871119896 =1198731 + sdot sdot sdot + 119873119896 [1 2 11 12]) However no one can affordto use such a slow cooling schedule Various improved SAalgorithms have thus been designed and to overcome theexcessively slow cooling schedule and to successfully resolvevarious optimization problems in industries and commerce[14 20ndash23] Realizing that theMTM algorithm can overcomethe ldquolocal-traprdquo problem and enjoy a high convergence ratewe propose the MTMSA algorithm which is a combinationof the MTM algorithm and the SA algorithm

22 MTMSA Algorithm The Metropolis algorithm is thefirst iterative sampling algorithm of the MCMC algorithmHastings extended this algorithm by allowing the proposaldistribution to be an asymmetric distribution ie the MHalgorithm [24]

The MH algorithm is an iterative MCMC samplingalgorithm whose iterative points 1199090 1199091 119909119899 show alimiting distribution 120587(119909) The challenge of using the MHalgorithm is that it tends to suffer from the ldquolocal-traprdquoproblem when the target distribution function 120587(119909) is amultimodal distribution [2 19] It eventually impedes theconvergence of the SA algorithm The MTM algorithm canoverlap the ldquolocal-traprdquo problemTheMTMalgorithm allowsseveral proposal points simultaneously and selects the bestone as the next sampling point while keeping the stationarydistribution unchanged Thus combining the MTM algo-rithm and SA algorithm yields the MTMSA algorithm

Suppose the global optimization problem is

minxisin119863

ℎ (x) (1)

The whole procedure of the MTMSA algorithm forproblem (1) is summarized below

(1) Set the temperature parameters 119879119898119886119909 119879119898119894119899 119886 isin (0 1)the length of theMarkov chain119873119896 and the number ofmultiple-try 119898 in the MTM algorithm Initialize thestate of the Markov chain x and set 119896 = 1

(2) At current temperature 119879119896 = 119879119898119886119909 times 119886119896 let120587119896 (x) prop expminusℎ (x)119879119896 (2)

be the stationary distribution For 119897 = 1 2 119873119896 usethe MTM algorithm to update the Markov chain 119873119896times

(21) Propose a ldquoproposal setrdquo of size 119898 from 119879(x sdot)denoted as x1 x2 xm where xi isin 119863and 119879(x sdot) is any symmetric proposal transformdistribution

(22) Randomly choose a proposal state xlowast fromx1 x2 xm with probability 120587119896(x1) 120587119896(x2) 120587119896(xm)(23) Propose a ldquoreference setrdquo of size 119898 denoted

as x1 x2 xm where x1 xmminus1 is pro-posed from 119879(xlowast sdot) and set xm = x

Mathematical Problems in Engineering 3

Input 119879119898119886119909 119879119898119894119899 119886119873119896 119898 ℎ(x) 119879(x y)Output x

(1) Initialize x = 1 119896 = 1 1198791 = 119879119898119886119909(2) while 119879119896 gt 119879119898119894119899 do(3) set 120587119896(x) prop expminusℎ(x)119879119896(4) for 119897 = 1 to119873119896 do(5) 119904119901 = 0(6) 119904119903 = 0(7) for 119894 = 1 to119898 do(8) sample xi from 119879(x sdot)(9) 119901119894 = 120587119896(xi)(10) 119904119901 = 119904119901 + 119901119894(11) choose xlowast from x1 x2 xm with probability 1199011 1199012 119901119898(12) for 119895 = 1 to119898 minus 1 do(13) sample xj from 119879(xlowast sdot)(14) 119904119903 = 119904119903 + 120587119896(xj)(15) 119904119903 = 119904119903 + 120587119896(x)(16) set 119903 = min119904119901119904119903 1(17) sample u from uniform distribution in [0 1](18) if 119906 lt 119903 then(19) set x = xlowast(20) 119896 = 119896 + 1(21) 119879119896 = 119879119898119886119909 times 119886119896(22) return x

Algorithm 1 MTMSA algorithm used to detect the minimum of ℎ(x) x isin 119863

(24) Calculate the generalized Metropolis ratio

119903 = min120587119896 (x1) + 120587119896 (x2) + sdot sdot sdot + 120587119896 (xm)120587119896 (x1) + 120587119896 (x2) + sdot sdot sdot + 120587119896 (xm) 1 (3)

Then update the current state of the Markovchain with probability 119903 Set x = xlowast otherwisereject it and keep x unchanged

(3) If 119879119896 lt 119879119898119894119899 output the last solution x and the mini-mum value of (1) of the whole procedure otherwiseupdate 119896 to 119896 + 1 and proceed to step (2)

Furthermore Algorithm 1 gives the pseudocode ofMTMSA algorithm for the computation of problem (1)

The convergence of the MTMSA algorithm can be ob-tained from the stationary distribution of the MTM algo-rithm (ie the detailed balance condition of the MTMalgorithm [1]) Theoretically when 119879119896 approaches zero andthe step number of theMTMalgorithm is sufficiently large allsamples drawn from 120587119896 would be in the vicinity of the globalminimum of ℎ(x) in119863

The next proposition gives the computation complex ofthe MTMSA algorithm

Proposition 1 The computation complex of the MTMSAalgorithm is

119874 (119873119899119898) (4)

where119873 is the length of decreasing cooling temperature 119899 is thefrequency of the Markov chain update and119898 is the number ofmultiple-try points

Proof The proof of the proposition directly follows theprocedure of the MTMSA algorithm described above Thedecreasing cooling temperature and the length of theMarkovchain are the external loops of the MTMSA algorithmBy combining the external loops with the internal loop ofthe multiple-try model we then complete the proof of theproposition

The proposition indicates that the computation complexof the MTMSA algorithm is a polynomial in119873 and119898 Giventhe computation complex of a stationary distribution 120587119896(x)the computation complex of the MTMSA algorithm is notgreater than the polynomial in 119889 (where 119889 is the dimensionof x)

The MTMSA algorithm has many advantages over otherapproximation algorithms Compared with traditional opti-mization algorithms (such as theNelder-Mead (NM)method[25] and the quasi-Newton (QN) method [26]) which arelocal optimization methods the MTMSA algorithm getsmore accurate results often as shown in our simulated mul-timodal experiment In practice the speed of the MTMSAalgorithm is generally high particularly for an efficient vectoroperation (or parallel computing) with MATLAB or R inthe evaluation of multiple-try pointsTheMTMSA algorithmclearly outperforms the SA algorithm in our experimentFurthermore by setting the number of multiple-try points119898 = 1 we can obtain a special case of theMTMSA algorithmthat is the SA algorithm Simulated and real data examples insubsequent sections show the advantage of the MTMSA overother approximated algorithms


(a) Mesh plot of objective function

0

1

2

3

4

5

6

7

8

9

10

Y

1 2 3 4 5 6 7 8 9 100

X(b) Contour plot of objective function

Figure 1 Mesh and contour plot of the objective function

3 Simulated Experiment Results

This section presents a simulation example (ie mixturenormal model) The main purpose of this example is todemonstrate that the MTMSA algorithm could compute theoptimization problem in the case of multiple local maximaand outperform its counterparts in terms of accuracy andefficiency All results are obtained using R language (versionX64 234) and MATLAB (version R2017a) on a Dell Opti-Plex7020MT desktop computer with Intel(R) Core(TM) i7-4790 CPU 36GHz RAM 800GB and Windows 7 Ulti-mate with Service Pack 1 (x64) The R and MATLAB codesin this work are available upon request to the correspondingauthor

In this simulation example we consider a two-dimen-sional multimodalmixture normalmodel modified from [2728] In this model the objective function is the probabilitydensity function which is the combination of 20 normalmodels

119891 (x) prop 20sum119894=1

12059611989421205871205902119894 expminus(121205902

119894)(xminus120583i)

1015840(xminus120583i) (5)

where 1205901 = 1205902 = sdot sdot sdot = 12059020 = 01 and (1205961 1205962 12059620) whichare the weights of the 20 normal models are chosen to bethe arithmetic progression from 1 to 5 except the last one12059620 = 10 The 20 mean vectors are independently sampledfrom the uniform distribution from [0 10]

Figure 1 illustrates the mesh and contour plots of (5)which contains 20 modes in this objective function Thisexample poses a serious challenge for optimization becausemany classical optimization methods may converge on thelocal optimum in this multimodal example Clearly theglobal maximum point is the last mode (0183 7501) withthe maximum value of 2449

Four methods are used to find the global maximum pointof this optimization problem the NMmethod modified QNmethod SA algorithm andMTMSA algorithmTheNM andQN methods are commonly applied numerical optimization

algorithms and the QN method allows box constraints TheSA and its improved version ie the MTMSA algorithmare stochastic optimization algorithms Apart from the min-imum temperature 119879119898119894119899 another commonly used parameterthat controls the degree of decreasing temperature is119873119905119890119898119901119890119903

119879119896 = 119879119898119886119909 sdot 120572119896minus1 119896 = 1 2 119873119905119890119898119901119890119903 (6)

For the SA algorithm we set the degree of decreasingtemperature 119873119905119890119898119901119890119903 = 75 the starting temperature 119879119898119886119909 =10 the decreasing parameter 120572 = 09 the length of theMarkov chain 119873119898119888 = 100 and the proposal variance of theMetropolis algorithm V119901119903119900 = 2 For the MTMSA algorithmwe set the number of multiple-tries 119873119898119905119898 = 100 The otherparameters are 119873119905119890119898119901119890119903 = 25 120572 = 08 119879119898119886119909 = 1 119873119898119888 = 100and V119901119903119900 = 2 With different 119873119905119890119898119901119890119903 (75 and 25) and 120572 (09and 08 respectively) values the SA andMTMSA algorithmshave similar 119879119898119894119899

We tested these four algorithms (NM QN SA andMTMSA algorithms) to compute the optimization problemand repeated this computation 50 times The computationresults are summarized in Figure 2 and Table 1

The mean value standard deviation (sd) mean squareerror (MSE) total CPU time (in seconds) and average CPUtime for one accurate result (in seconds) of the resultsfrom different algorithms are summarized in Table 1 where119898119890119886119899 = (1119877)sum119877119894=1 119881119894 119904119889 = radic(1(119877 minus 1))sum119877119894=1(119881119894 minus 119898119890119886119899)2119872119878119864 = (1119877)sum119877119894=1(119881119894 minus 119881119890)2 and 119881119890 = 2449 is the exactmaximum value in the model (5) Figure 2 illustrates theboxplot of 50 computation results from these four algorithms

Suffering from the ldquolocal-traprdquo problem NM and QNalgorithms cannot find the global maximum successfully in50 computations (they often find other local modes in (5))Compared with the MTMSA algorithm the SA algorithmuses a slowly decreasing temperature schedule (119873119905119890119898119901119890119903 =75) and consumes more CPU time However only 10 resultsof 50 repetitions from the SA algorithm converge to the


Table 1 Computation results (mean sd MSE consumed total CPU time and average CPU time (in seconds)) of the 50 computations fromdifferent algorithms

DM QN SA MTMSAmean 03447 00311 17523 24392sd 04563 01355 06427 00086MSE 46319 58635 08901 00001total CPU time 41732 09121 13056 5281average CPU time +infin +infin 13056 10562

NM QN SA MTMSA

0

05

1

15

2

25

Figure 2 Boxplot of 50 results computed from theNMmethod QNmethod SA algorithm and MTMSA algorithm

global maximum point (0183 7501) and the mean of theSA algorithm is 17523 By contrast the MTMSA algorithmfunctions with a rapidly decreasing temperature scheduleThe MTMSA algorithm consumes minimal CPU time (onlyabout 8min) but it yields highly accurate results (all 50results converge to the global maximum) Furthermore theMTMSA algorithm only needs approximately 10 secondsto compute one accurate result whereas the SA algorithmrequires about 130 seconds All results from NM and QNalgorithms suffer from the ldquolocal-traprdquo problem

We compared the differences in the decreasing temper-ature schedules used in SA and MTMSA algorithms ldquoTheslower the betterrdquowas found to be applicable to the decreasingtemperature schedules A rapidly decreasing temperatureschedule may result in the ldquolocal-traprdquo problem In the nextsimulation we set the temperature schedules to decreasefrom 10 to 5times10minus3 and the length of decreasing temperaturewas set to 500 75 and 25 for SA and MTMSA algorithmsEach computation was repeated 50 times

The length of decreasing temperature of the SA algorithmis set to 75 500 and denoted as the SA1 and SA2 algorithmrespectivelyThe SA2 algorithm shows the slowest decreasingschedule It uses 500 steps to drop from the highest temper-ature to the lowest one Thus almost all 50 results convergeto the global maximum (about 94 percent of computationresults escape from local optima and reaches the globalmaximum) The SA1 algorithm uses a rapidly decreasingschedule and only about half of the 50 results converge to theglobal maximum (about 54 percent of computation resultsescape from local optima and reaches the global maximum)By contrast the MTMSA algorithm only uses 25 steps indecreasing temperature but all of the 50 results converge tothe global maximum

Table 2 Biochemical oxygen demand versus time

Time (days) BOD (mgI)1 832 1033 1904 1605 1567 198

Figure 3 shows the decreasing schedules and convergencepaths of the three algorithms We find that when the tem-perature decreases to about 002 (corresponding to the 50th400th and 20th steps in SA1 SA2 and MTMSA) all samplepaths from the three algorithms converge to their local andglobal optima All the sample paths of MTMSA converge tothe global optima and lots of sample paths of SA1 and SA2converge to the local optima because the average sample pathof MTMSA in Figure 3 is the highest and at the level about243 The MTMSA algorithm uses the rapidly decreasingschedule and achieves the fastest convergence rateThereforethe MTMSA algorithm is the most efficient and accurate inthis simulation example

4 Real Data Examples

41 Bayesian Analysis Using MTMSA In this section weillustrate the application of the MTMSA algorithm inBayesian analysis with real data from [29] In this examplewe fit a nonlinear model derived from exponential decay

119910119894 = 1205791 (1 minus exp minus1205792119909119894) + 120576119894 120576119894 sim 119873(0 1205902) (7)

with a fixed rate that is constant to a real data set [30](Table 2)

The variables BOD (mgI)) and time (days) in Table 2 arethe response and control variables in model (7) (denoted asthe BOD problem) with a constant variance 1205902 for indepen-dent normal errors The likelihood for the BOD problem is

119871 (1205791 1205792 1205902 | 119883 119884) prop expminus6 log120590minus 12

sum6119894=1 (119910119894 minus 1205791 (1 minus exp minus1205792119909119894))21205902 (8)

where119883 = (1199091 1199096) and 119884 = (1199101 1199106)


Step

the SA1 algorithmthe SA2 algorithmthe MTMSA algorithm

5004504003503002502001501005000

1

2

3

4

5

6

7

8

9

10Te

mpe

ratu

re

(a) The decreasing temperature schedules


Temperature step

0

05

1

15

2

25

Mar

kov

chai

n pa

th

500450400350300250200150100500

(b) The convergence paths of different algorithms

Figure 3 Decreasing temperature schedules (a) and convergence paths (b) of the SA1 algorithm SA2 algorithm and MTMSA algorithmThe convergence paths are the average of 50 paths

Table 3 Computation results (mean (10minus5) sd (10minus5)) in special temperature steps (1 5 10 15 20 25) from 20 repetitionsstep 1 step 5 step 10 step 15 step 20 step 25

mean 105 139 091 392 145 148sd 357 516 220 122 304 024

While choosing the flat prior for the parameters 1205902and (1205791 1205792) (ie the uniform distribution in (0 +infin) and[minus20 50] times [minus2 6] respectively) and integrating out 1205902 weobtain the following (improper) posterior distribution of(1205791 1205792)

119901 (1205791 1205792 | 119883 119884)prop [ 6sum119894=1

(119910119894 minus 1205791 (1 minus exp minus1205792119909119894))2]minus2

sdot 119868[minus2050]times[minus26] (1205791 1205792) (9)

where

119868[minus2050]times[minus26] (1205791 1205792)=

1 (1205791 1205792) isin [minus20 50] times [minus2 6]0 (1205791 1205792) notin [minus20 50] times [minus2 6]

(10)

For a Bayesian analysis one often treats the parameters(1205791 1205792) as randomvariables In this work we use the posteriordistribution of (1205791 1205792) for their statistical inference and usethe posterior mode of (9) as the estimation of (1205791 1205792)which coincides with the maximum likelihood estimationThe Bayesian statistical inference of the parameters (1205791 1205792) is

translated to the global optimization problem in [minus20 50] times[minus2 6]sup

(1205791 1205792)isin[minus2050]times[minus26]

119901 (1205791 1205792 | 119883 119884) (11)

In addition we use the MTMSA algorithm to computethe global optimization problem (11) The parameters of theMTMSA algorithm are set to be 119873119898119905119898 = 20 119873119905119890119898119901119890119903 =25 120572 = 06 119879119898119886119909 = 1 and 119873119898119888 = 1000 The computationis then repeated 20 times Figure 4 and Table 3 illustrate thedecreasing temperature schedule and the convergence pathsof 20 repetitions from the MTMSA algorithm After 20 stepsall 20 computation paths become convergent to 148 times 10minus3which has the largest mean and smallest sd

Figure 5 shows the mesh (a) and contour (b) plots ofthe posterior distribution (9) Figure 6 and Table 4 showthe locations of the scatters (1205791 1205792) from 20 repetitions atdifferent temperature steps With the temperature decreasingfrom 06 to 28times10minus6 all scatters converge to the optimizationpoint (1915 053)42 Half-Space Depth Computation Using MTMSA As apowerful tool for nonparametric multivariate analysis half-space depth (HD also known as Tukey depth) has beeneliciting increased interest since it was introduced by Tukey[31 32] HD which extends univariate order-related statisticsto multivariate settings provides a center-outward ordering


Table 4 Location results of (1205791 1205792) at different temperature levels

Level 06 28 times 10minus4 16 times 10minus4 28 times 10minus6

mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)

Figure 4 The decreasing temperature schedule (a) and the convergence paths of 20 repetitions (b) from the MTMSA algorithm

minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2

(a) Mesh plot of posterior distribution

minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201

(b) Contour plot of posterior distribution

Figure 5 Exact mesh (a) and contour (b) plots of the posterior distribution (9)

of multivariate samples and visualizes data in high dimen-sional cases [33 34] However the computation of HD ischallenging and the exact algorithm is often inefficientespecially when the dimension is high [35] In this subsectionwe useMTMSA to computeHDand comparedMTMSAwithother approximated and exact algorithms

Given a sample data set of size 119899 X119899 = X1X2 X119899 inR119889 x is a point inR119889 and the HD of x with respect to (wrt)X119899 is defined by

119867119863(xX119899) = minuisinS119889minus1

1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)

Figure 6 Locations of (1205791 1205792) from 20 repetitions at different temperature steps (06 28 times 10minus4 16 times 10minus4 28 times 10minus6 )

where S119889minus1 = 119906 isin R119889 | 119906 = 1N = 1 2 119899 and sdotdenotes the counting measureThen the computation of HD(12) is a global optimization problem in S119889minus1

Next we considered a concrete data set (Table 6) obtainedfrom [35] and can be found in the Records Office of theLaboratory School of the University of Chicago The originaldata consisted of 64 subjectsrsquo scores obtained from eighth-grade levels to eleventh-grade levels Then we comparedMTMSA with three approximated algorithms (NM QNand SA) and the exact algorithm from [35] for the HDcomputation of the first data point wrt the data set

We tested two sets of parameters for the SA algorithmThe first is119873119905119890119898119901119890119903 = 20119873119898119888 = 50 119879119898119886119909 = 1 and 119886 = 07 anddenoted as the SA1 algorithmThe second one is119873119905119890119898119901119890r = 20119873119898119888 = 200 119879119898119886119909 = 1 and 119886 = 07 and denoted as the SA2algorithm For the MTMSA algorithm we set the parameterto be 119873119905119890119898119901119890119903 = 20 119898 = 100 119873119898119888 = 30 119879119898119886119909 = 1 and119886 = 07 The three algorithms (SA1 SA2 and MTMSA) usethe same decreasing temperature schedule Then we usedthe six algorithms (exact NM QN SA1 SA2 and MTMSA)for this computation and repeated the computation 50 timesFigure 7 and Table 5 show the computation results



exact NM QN SA1 SA2 MTMSAmean 02344 03653 03841 02609 02425 02344sd 0 00519 00485 00243 00079 0MSE 0 00199 00247 00013 00001 0total CPU time 2450 006 005 57410 23103 1587average CPU time 49 +infin +infin 09570 09626 03174

Table 6 Concrete data set

subject Grade 8 Grade 9 Grade 10 Grade 111 175 260 376 3682 090 247 244 3433 080 093 040 2274 242 415 456 4215 minus131 minus131 minus066 minus2226 minus156 167 018 2337 109 150 052 2338 minus192 103 050 3049 minus161 029 073 32410 247 364 287 53811 minus095 041 021 18212 166 274 240 21713 207 492 446 47114 330 610 719 74615 275 253 428 59316 225 338 579 44017 208 174 412 36218 014 001 148 27819 013 319 060 31420 219 265 327 27321 minus064 minus131 minus037 40922 202 345 532 60123 205 180 391 24924 148 047 363 38825 197 254 326 56226 135 463 354 52427 minus056 minus036 114 13428 026 008 117 21529 122 141 466 26230 minus143 080 minus003 10431 minus117 166 211 14232 168 171 407 33033 minus047 093 130 07634 218 642 464 48235 421 708 600 56536 826 955 1024 105837 124 490 242 25438 594 656 936 77239 087 336 258 173

Table 6 Continued

subject Grade 8 Grade 9 Grade 10 Grade 1140 minus009 229 308 33541 324 478 352 48442 103 210 388 28143 358 467 383 51944 141 175 370 37745 minus065 minus011 240 35346 152 304 274 26347 057 271 190 24148 218 296 478 33449 110 265 172 29650 015 269 269 35051 minus127 126 071 26852 281 519 633 59353 262 354 486 58054 011 225 156 39255 061 114 135 05356 minus219 minus042 154 11657 155 242 111 21858 004 050 260 26159 310 200 392 39160 minus029 262 160 18661 228 339 491 38962 257 578 512 49863 minus219 071 156 23164 minus004 244 179 264

Figure 7 and Table 5 show that the exact algorithmconsumed the most CPU time (about 2450 seconds) andobtained exact computation results (02344) MTMSA alsoobtained the exact results but consumed only 1587 secondsThe SA algorithms (SA1 and SA2) consumed suitable CPUtime (5741 and 23103 seconds respectively) but obtainedonly 6 and 24 exact results respectively The results of NMand QN fell into the local optima because all of them werelarger than the exact result With regard to the average CPUtime MTMSA used only 03174 for the computation of oneexact result which is the least amount of time compared withthe time for the other exact and approximated algorithmsHence MTMSA outperformed the other algorithms in thisexperiment example


Exact NM QN SA1 SA2 MTMSA

025

03

035

04

045

05

Figure 7 Boxplot of the results computed from the exact NM QNSA1 SA2 and MTMSA algorithms

5 Conclusions

Wedeveloped theMTMSA algorithm for global optimizationproblems in the fields of mathematicalbiological sciencesengineering Bayesian data analysis operational research lifesciences and so onTheMTMSA algorithm is a combinationof the SA algorithm and the MTM algorithm Using simu-lated and real data examples it demonstrated that relativeto the QN and SA algorithm the MTMSA algorithm canfunction with a rapidly decreasing cooling schedule whileguaranteeing that the global energy minima are reached

Several directions can be taken for future work Firstcombined with the quasi-Monte Carlo method the low-discrepancy sequences and experimental design [36ndash39] canbe used to accelerate the convergence of the SA algorithmSecond aside from the MTM algorithm the MTMSA algo-rithm can also be implemented with several parallel interact-ing Markov chains to improve the SA algorithm by makingfull use of modern multicore computer [40 41] Third weanticipate that a parallel SA algorithm can be used efficientlyfor variable selection in high dimensional cases [42ndash45]because the variable selection problem is a special case ofthe optimization problem Finally data depth [32 33 35 46]is an important tool for multidimensional data analysis butthe computation of data depth in high dimensional cases ischallengingThe example of half-space depth computation inSection 4 shows the advantage of the MTMSA algorithm inlow dimensional case Hence we believe that the MTMSAalgorithm can be successfully applied to compute highlycomplex data depths (eg projection and regression depths)in high dimensional cases Further analysis along thesedirections would be interesting

Data Availability

The data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

The authors declare that they have no conflicts of interestregarding the publication of this paper

Acknowledgments

The research was partially supported by the National Nat-ural Science Foundation of China (11501320 71471101 and11426143) the Natural Science Foundation of ShandongProvince (ZR2014AP008) and the Natural Science Founda-tion of Qufu Normal University (bsqd20130114)

References

[1] J S LiuMonteCarlo Strategies in ScientificComputing Springer2001

[2] F Liang C Liu and R J Carroll Advanced Markov ChainMonte Carlo Methods Learning From Past Samples John Wileyamp Sons 2011

[3] S Kirkpatrick J Gelatt andM P Vecchi ldquoOptimization by sim-ulated annealingrdquo American Association for the Advancement ofScience Science vol 220 no 4598 pp 671ndash680 1983

[4] Y Wang L Qi S Luo and Y Xu ldquoAn alternative steepestdirection method for the optimization in evaluating geometricdiscordrdquo Pacific Journal of Optimization vol 10 no 1 pp 137ndash149 2014

[5] C Wang and Y Wang ldquoA superlinearly convergent projectionmethod for constrained systems of nonlinear equationsrdquo Jour-nal of Global Optimization vol 44 no 2 pp 283ndash296 2009

[6] Y Wang L Caccetta and G Zhou ldquoConvergence analysis of ablock improvement method for polynomial optimization overunit spheresrdquo Numerical Linear Algebra with Applications vol22 no 6 pp 1059ndash1076 2015

[7] L Qi X Tong and Y Wang ldquoComputing power systemparameters to maximize the small signal stability margin basedonmin-maxmodelsrdquoOptimization and Engineering vol 10 no4 pp 465ndash476 2009

[8] H Chen Y Chen G Li and L Qi ldquoA semidefinite programapproach for computing the maximum eigenvalue of a classof structured tensors and its applications in hypergraphs andcopositivity testrdquo Numerical Linear Algebra with Applicationsvol 25 no 1 2018

[9] G Wang and X X Huang ldquoLevitin-Polyak well-posednessfor optimization problems with generalized equilibrium con-straintsrdquo Journal of Optimization Theory and Applications vol153 no 1 pp 27ndash41 2012

[10] G Wang ldquoLevitin-Polyak well-posedness for vector optimiza-tion problems with generalized equilibrium constraintsrdquo PacificJournal of Optimization vol 8 no 3 pp 565ndash576 2012

[11] S Geman and D Geman ldquoStochastic relaxation gibbs distri-butions and the Bayesian restoration of imagesrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 6 no 6pp 721ndash741 1984

[12] R A Holley S Kusuoka and D W Stroock ldquoAsymptotics ofthe spectral gap with applications to the theory of simulatedannealingrdquo Journal of Functional Analysis vol 83 no 2 pp 333ndash347 1989

[13] F Liang C Liu and R J Carroll ldquoStochastic approximation inMonte Carlo computationrdquo Journal of the American StatisticalAssociation vol 102 no 477 pp 305ndash320 2007

[14] F Liang Y Cheng and G Lin ldquoSimulated stochastic approx-imation annealing for global optimization with a square-rootcooling schedulerdquo Journal of the American Statistical Associa-tion vol 109 no 506 pp 847ndash863 2014


[15] G Karagiannis B A Konomi G Lin and F Liang ldquoParalleland interacting stochastic approximation annealing algorithmsfor global optimisationrdquo Statistics and Computing vol 27 no 4pp 927ndash945 2017

[16] J S Liu F Liang and W H Wong ldquoThe multiple-try methodand local optimization in metropolis samplingrdquo Journal of theAmerican Statistical Association vol 95 no 449 pp 121ndash1342000

[17] R Casarin R Craiu and F Leisen ldquoInteracting multiple tryalgorithms with different proposal distributionsrdquo Statistics andComputing vol 23 no 2 pp 185ndash200 2013

[18] N Metropolis A W Rosenbluth M N Rosenbluth A HTeller and E Teller ldquoEquation of state calculations by fast com-putingmachinesrdquoThe Journal of Chemical Physics vol 21 no 6pp 1087ndash1092 1953

[19] W Shao G Guo F Meng and S Jia ldquoAn efficient proposal dis-tribution for metropolis-hastings using a b-splines techniquerdquoComputational Statistics amp Data Analysis vol 57 pp 465ndash4782013

[20] W Shao and Y Zuo ldquoSimulated annealing for higher dimen-sional projection depthrdquo Computational Statistics amp Data Anal-ysis vol 56 no 12 pp 4026ndash4036 2012

[21] W Shao G Guo G Zhao and F Meng ldquoSimulated annealingfor the bounds of kendallrsquos tau and spearmanrsquos rhordquo Journal ofStatistical Computation and Simulation vol 84 no 12 pp 2688ndash2699 2014

[22] Y Luo B Zhu and Y Tang ldquoSimulated annealing algorithm foroptimal capital growthrdquo Physica A Statistical Mechanics and itsApplications vol 408 pp 10ndash18 2014

[23] O S Sarıyer and C Guven ldquoSimulated annealing algorithm foroptimal capital growthrdquo Physica A Statistical Mechanics and itsApplications vol 408 pp 10ndash18 2014

[24] W K Hastings ldquoMonte Carlo sampling methods using Markovchains and their applicationsrdquo Biometrika vol 57 no 1 pp 97ndash109 1970

[25] J A Nelder and RMead ldquoA simplexmethod for functionmini-mizationrdquoTheComputer Journal vol 7 no 4 pp 308ndash313 1965

[26] R H Byrd P Lu J Nocedal and C Y Zhu ldquoA limited memoryalgorithm for bound constrained optimizationrdquo SIAM Journalon Scientific Computing vol 16 no 5 pp 1190ndash1208 1995

[27] S C KouQ Zhou andWHWong ldquoEqui-energy samplerwithapplications in statistical inference and statistical mechanicsrdquoThe Annals of Statistics vol 34 no 4 pp 1581ndash1652 2006

[28] F Liang andW HWong ldquoReal-parameter evolutionary MonteCarlo with applications to Bayesian mixture modelsrdquo Journal ofthe American Statistical Association vol 96 no 454 pp 653ndash666 2001

[29] C Ritter andM A Tanner ldquoFacilitating the Gibbs samplerTheGibbs stopper and the GriddyndashGibbs samplerrdquo Journal of theAmerican Statistical Association vol 87 no 419 pp 861ndash8681992

[30] D M Bates and D G Watts Nonlinear Regression Analysis andIts Applications John Wiley amp Sons New York NY USA 1988

[31] J W Tukey ldquoMathematics and the picturing of datardquo Proceed-ings of the International Congress of Mathematicians vol 2 pp523ndash531 1975

[32] Y Zuo and R Serfling ldquoGeneral notions of statistical depthfunctionrdquo The Annals of Statistics vol 28 no 2 pp 461ndash4822000

[33] X Liu Y Zuo and QWang ldquoFinite sample breakdown point ofTukeyrsquos halfspace medianrdquo Science China Mathematics vol 60no 5 pp 861ndash874 2017

[34] X Liu ldquoFast implementation of the Tukey depthrdquo Computa-tional Statistics vol 32 no 4 pp 1395ndash1410 2017

[35] X Liu and Y Zuo ldquoComputing halfspace depth and regressiondepthrdquo Communications in StatisticsmdashSimulation and Compu-tation vol 43 no 5 pp 969ndash985 2014

[36] Z Li S Zhao and R Zhang ldquoOn general minimum lower orderconfounding criterion for s-level regular designsrdquo Statistics ampProbability Letters vol 99 pp 202ndash209 2015

[37] J Wang Y Yuan and S Zhao ldquoFractional factorial split-plot designs with two- and four-level factors containing cleareffectsrdquoCommunications in StatisticsmdashTheory andMethods vol44 no 4 pp 671ndash682 2015

[38] S Zhao D K Lin and P Li ldquoA note on the construction ofblocked two-level designs with general minimum lower orderconfoundingrdquo Journal of Statistical Planning and Inference vol172 pp 16ndash22 2016

[39] S-L Zhao and Q Sun ldquoOn constructing general minimumlower order confounding two-level block designsrdquoCommunica-tions in StatisticsmdashTheory and Methods vol 46 no 3 pp 1261ndash1274 2017

[40] G Guo W You G Qian and W Shao ldquoParallel maximumlikelihood estimator for multiple linear regression modelsrdquoJournal of Computational and AppliedMathematics vol 273 pp251ndash263 2015

[41] G Guo W Shao L Lin and X Zhu ldquoParallel temperingfor dynamic generalized linear modelsrdquo Communications inStatisticsmdashTheory and Methods vol 45 no 21 pp 6299ndash63102016

[42] M Wang and G-L Tian ldquoRobust group non-convex estima-tions for high-dimensional partially linear modelsrdquo Journal ofNonparametric Statistics vol 28 no 1 pp 49ndash67 2016

[43] MWang L Song andG-l Tian ldquoScad-penalized least absolutedeviation regression in high-dimensional modelsrdquo Communi-cations in StatisticsmdashTheory and Methods vol 44 no 12 pp2452ndash2472 2015

[44] G-L Tian M Wang and L Song ldquoVariable selection in thehigh-dimensional continuous generalized linear model withcurrent status datardquo Journal of Applied Statistics vol 41 no 3pp 467ndash483 2014

[45] MWang andXWang ldquoAdaptive Lasso estimators for ultrahighdimensional generalized linear modelsrdquo Statistics amp ProbabilityLetters vol 89 pp 41ndash50 2014

[46] P J Rousseeuw and M Hubert ldquoRegression depthrdquo Journal ofthe American Statistical Association vol 94 no 446 pp 388ndash433 1999

Hindawiwwwhindawicom Volume 2018

MathematicsJournal of


Mathematical Problems in Engineering

Applied MathematicsJournal of


Probability and StatisticsHindawiwwwhindawicom Volume 2018

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of



Engineering Mathematics

International Journal of


Operations ResearchAdvances in

Journal of


Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018

International Journal of Mathematics and Mathematical Sciences


Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018Volume 2018

Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in

Nature and SocietyHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Dierential EquationsInternational Journal of

Volume 2018


Decision SciencesAdvances in


AnalysisInternational Journal of


Stochastic AnalysisInternational Journal of

Submit your manuscripts atwwwhindawicom


annealing (MTMSA) algorithm for global optimizationTheMTMSA algorithm is a combination of the SA algorithm andmultiple-try Metropolis (MTM) algorithm [16] The MTMalgorithm which allows several proposals from differentproposal distributions in the multiple-try step simultane-ously achieves a higher rate of convergence than the stan-dard Metropolis algorithm (eg random walk Metropolisalgorithm) [1 2 17] Thus the MTMSA algorithm canguarantee that global minima are reached with a rapidlydecreasing cooling schedule Comparing with PISAA whichshould run on multicore computer to their advantage theMTMSA often owns high convergent rate by use of theefficient vector operation with MATLAB or R on personaland super computer Simulation and real data examples showthat under the framework of the multiple-try algorithmthe MTMSA algorithm can reach global minima under arapidly decreasing cooling schedule relative to that of the SAalgorithm

The remainder of this paper is organized as followsSection 2 describes the framework of theMTMSA algorithmSection 3 illustrates the comparison of the MTMSA algo-rithm with other optimization methods in a mixture normalmodel Section 4 presents the application of the MTMSAalgorithm to Bayesian analysis and half-space depth compu-tation through real data sets Finally Section 5 summarizesthe conclusions derived from the study

2 MTMSA Algorithm

21 Overview of SA Algorithm The SA algorithm originatesfrom the annealing process which is a thermodynamicprocess used to attain a low energy state in condensed matterphysics [1] The process comprises two steps The first stateis the high temperature state in which solids transform intoliquid and particles move freely to ward ideal Then thetemperature drops to zero slowly and the movement of theparticles become restricted such that the desired structureis achieved Realizing that the Metropolis algorithm [18] canbe used to simulate the movements of particles Kirkpatricket al (1983) proposed a computer simulation based physicalannealing process ie the SA algorithm

Suppose our goal is to find the minimum value of ℎ(x)x isin 119863 This goal is equivalent to the search for the maximumvalue of expminusℎ(x)119879 x isin 119863 with any positive temperature119879 Let 1198791 gt 1198792 gt sdot sdot sdot gt 119879119896 gt sdot sdot sdot be a decreasingtemperature sequence with large 1198791 and lim119896997888rarr+infin119879119896 = 0 Inevery temperature 119879119896 we use the Metropolis-Hastings (MH)algorithm (or Gibbs sampling algorithm [19]) to update theMarkov chain 119873119896 times with 120587119896(x) prop expminusℎ(x)119879119896 as itsstationary distribution When 119896 is increasing an increasingnumber of samples concentrate in the maximum valuenearby The SA algorithm can be summarized as follows

(1) Initialize x(0) with starting temperature 1198791(2) At current temperature 119879119896 update the Markov chain119873119896 times with120587119896(x) as its stationary distribution and

transmit the last state x to the next temperature

(3) Update 119896 to 119896 + 1

The SA algorithm can reach the global optimum whenthe temperature sequence decreases slowly (eg the inverselogarithmic rate ie the order of 119874(log(119871119896)minus1) where 119871119896 =1198731 + sdot sdot sdot + 119873119896 [1 2 11 12]) However no one can affordto use such a slow cooling schedule Various improved SAalgorithms have thus been designed and to overcome theexcessively slow cooling schedule and to successfully resolvevarious optimization problems in industries and commerce[14 20ndash23] Realizing that theMTM algorithm can overcomethe ldquolocal-traprdquo problem and enjoy a high convergence ratewe propose the MTMSA algorithm which is a combinationof the MTM algorithm and the SA algorithm

22 MTMSA Algorithm The Metropolis algorithm is thefirst iterative sampling algorithm of the MCMC algorithmHastings extended this algorithm by allowing the proposaldistribution to be an asymmetric distribution ie the MHalgorithm [24]

The MH algorithm is an iterative MCMC samplingalgorithm whose iterative points 1199090 1199091 119909119899 show alimiting distribution 120587(119909) The challenge of using the MHalgorithm is that it tends to suffer from the ldquolocal-traprdquoproblem when the target distribution function 120587(119909) is amultimodal distribution [2 19] It eventually impedes theconvergence of the SA algorithm The MTM algorithm canoverlap the ldquolocal-traprdquo problemTheMTMalgorithm allowsseveral proposal points simultaneously and selects the bestone as the next sampling point while keeping the stationarydistribution unchanged Thus combining the MTM algo-rithm and SA algorithm yields the MTMSA algorithm

Suppose the global optimization problem is

minxisin119863

ℎ (x) (1)

The whole procedure of the MTMSA algorithm forproblem (1) is summarized below

(1) Set the temperature parameters 119879119898119886119909 119879119898119894119899 119886 isin (0 1)the length of theMarkov chain119873119896 and the number ofmultiple-try 119898 in the MTM algorithm Initialize thestate of the Markov chain x and set 119896 = 1

(2) At current temperature 119879119896 = 119879119898119886119909 times 119886119896 let120587119896 (x) prop expminusℎ (x)119879119896 (2)

be the stationary distribution For 119897 = 1 2 119873119896 usethe MTM algorithm to update the Markov chain 119873119896times

(21) Propose a ldquoproposal setrdquo of size 119898 from 119879(x sdot)denoted as x1 x2 xm where xi isin 119863and 119879(x sdot) is any symmetric proposal transformdistribution

(22) Randomly choose a proposal state xlowast fromx1 x2 xm with probability 120587119896(x1) 120587119896(x2) 120587119896(xm)(23) Propose a ldquoreference setrdquo of size 119898 denoted

as x1 x2 xm where x1 xmminus1 is pro-posed from 119879(xlowast sdot) and set xm = x













119874 (119873119899119898) (4)







0

1

2

3

4

5

6

7

8

9

10

Y

1 2 3 4 5 6 7 8 9 100






119891 (x) prop 20sum119894=1

12059611989421205871205902119894 expminus(121205902

119894)(xminus120583i)

1015840(xminus120583i) (5)





119879119896 = 119879119898119886119909 sdot 120572119896minus1 119896 = 1 2 119873119905119890119898119901119890119903 (6)








NM QN SA MTMSA

0

05

1

15

2

25






Time (days) BOD (mgI)1 832 1033 1904 1605 1567 198









where119883 = (1199091 1199096) and 119884 = (1199101 1199106)


Step


5004504003503002502001501005000

1

2

3

4

5

6

7

8

9

10Te

mpe

ratu

re



Temperature step

0

05

1

15

2

25

Mar

kov

chai

n pa

th

500450400350300250200150100500




mean 105 139 091 392 145 148sd 357 516 220 122 304 024


119901 (1205791 1205792 | 119883 119884)prop [ 6sum119894=1



where



(10)




119901 (1205791 1205792 | 119883 119884) (11)






mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)


minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2


minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201






1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018




















119874 (119873119899119898) (4)







0

1

2

3

4

5

6

7

8

9

10

Y

1 2 3 4 5 6 7 8 9 100






119891 (x) prop 20sum119894=1

12059611989421205871205902119894 expminus(121205902

119894)(xminus120583i)

1015840(xminus120583i) (5)





119879119896 = 119879119898119886119909 sdot 120572119896minus1 119896 = 1 2 119873119905119890119898119901119890119903 (6)








NM QN SA MTMSA

0

05

1

15

2

25






Time (days) BOD (mgI)1 832 1033 1904 1605 1567 198









where119883 = (1199091 1199096) and 119884 = (1199101 1199106)


Step


5004504003503002502001501005000

1

2

3

4

5

6

7

8

9

10Te

mpe

ratu

re



Temperature step

0

05

1

15

2

25

Mar

kov

chai

n pa

th

500450400350300250200150100500




mean 105 139 091 392 145 148sd 357 516 220 122 304 024


119901 (1205791 1205792 | 119883 119884)prop [ 6sum119894=1



where



(10)




119901 (1205791 1205792 | 119883 119884) (11)






mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)


minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2


minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201






1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018










0

1

2

3

4

5

6

7

8

9

10

Y

1 2 3 4 5 6 7 8 9 100






119891 (x) prop 20sum119894=1

12059611989421205871205902119894 expminus(121205902

119894)(xminus120583i)

1015840(xminus120583i) (5)





119879119896 = 119879119898119886119909 sdot 120572119896minus1 119896 = 1 2 119873119905119890119898119901119890119903 (6)








NM QN SA MTMSA

0

05

1

15

2

25






Time (days) BOD (mgI)1 832 1033 1904 1605 1567 198









where119883 = (1199091 1199096) and 119884 = (1199101 1199106)


Step


5004504003503002502001501005000

1

2

3

4

5

6

7

8

9

10Te

mpe

ratu

re



Temperature step

0

05

1

15

2

25

Mar

kov

chai

n pa

th

500450400350300250200150100500




mean 105 139 091 392 145 148sd 357 516 220 122 304 024


119901 (1205791 1205792 | 119883 119884)prop [ 6sum119894=1



where



(10)




119901 (1205791 1205792 | 119883 119884) (11)






mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)


minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2


minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201






1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018











NM QN SA MTMSA

0

05

1

15

2

25






Time (days) BOD (mgI)1 832 1033 1904 1605 1567 198









where119883 = (1199091 1199096) and 119884 = (1199101 1199106)


Step


5004504003503002502001501005000

1

2

3

4

5

6

7

8

9

10Te

mpe

ratu

re



Temperature step

0

05

1

15

2

25

Mar

kov

chai

n pa

th

500450400350300250200150100500




mean 105 139 091 392 145 148sd 357 516 220 122 304 024


119901 (1205791 1205792 | 119883 119884)prop [ 6sum119894=1



where



(10)




119901 (1205791 1205792 | 119883 119884) (11)






mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)


minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2


minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201






1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018









Step


5004504003503002502001501005000

1

2

3

4

5

6

7

8

9

10Te

mpe

ratu

re



Temperature step

0

05

1

15

2

25

Mar

kov

chai

n pa

th

500450400350300250200150100500




mean 105 139 091 392 145 148sd 357 516 220 122 304 024


119901 (1205791 1205792 | 119883 119884)prop [ 6sum119894=1



where



(10)




119901 (1205791 1205792 | 119883 119884) (11)






mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)


minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2


minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201






1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018











mean (2022 180) (1663 106) (1825 105) (1915 053)sd (2171 239) (1764 192) (958 156) (011 001)

5 10 15 20 250

Step

0

01

02

03

04

05

06

Tem

pera

ture

(a)

times10-3

5 10 15 20 250

Step

0

05

1

15

Valu

e

(b)


minus56

0

4

5

p

40

10

2 200 0

minus2 minus20

times10-4

1

2


minus2

minus1

0

1

2

3

4

5

6

2

minus10 0 10 20 30 40 50minus201






1119899 119894 | u119879X119894 ge u119879x 119894 isinN (12)


2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018









2

minus10 0 10 20 30 40 50minus201

minus2

minus1

0

1

2

3

4

5

6

(a)2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(b)

minus10 0 10 20 30 40 50minus201

2

minus2

minus1

0

1

2

3

4

5

6

(c)

2

minus2

minus1

0

1

2

3

4

5

6

minus10 0 10 20 30 40 50minus201

(d)










Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018













Table 6 Continued





025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018










025

03

035

04

045

05


5 Conclusions



Data Availability




Acknowledgments


References























































Journal of












Journal of







Volume 2018






Volume 2018
















































Journal of












Journal of







Volume 2018






Volume 2018








Multiple-Try Simulated Annealing Algorithm for Global ...

Documents

Transcript of Multiple-Try Simulated Annealing Algorithm for Global ...