The Impact of Design Choices of Multiobjective AntColony Optimization Algorithms on Performance:

An Experimental Study on the Biobjective TSP

Manuel López-IbáñezIRIDIA, CoDE, Université Libre de Bruxelles

Brussels, Belgiummanuel.lopez-ibanez@ulb.ac.be

Thomas StützleIRIDIA, CoDE, Université Libre de Bruxelles

Brussels, Belgiumstuetzle@ulb.ac.be

ABSTRACTOver the last few years, there have been a number of propos-als of ant colony optimization (ACO) algorithms for tacklingmultiobjective combinatorial optimization problems. Theseproposals adapt ACO concepts in various ways, for example,some use multiple pheromone matrices and multiple heuris-tic matrices and others use multiple ant colonies.

In this article, we carefully examine several of the mostprominent of these proposals. In particular, we identify com-monalities among the approaches by recasting the originalformulation of the algorithms in different terms. For exam-ple, several proposals described in terms of multiple coloniescan be cast equivalently using a single ant colony, where antsuse different weights for aggregating the pheromone and/orthe heuristic information. We study algorithmic choices forthe various proposals and we identify previously undetectedtrade-offs in their performance.

Categories and Subject DescriptorsI.2.8 [Computing Methodologies]: Artificial Intelligence—Problem Solving, Control Methods, and Search

General TermsAlgorithms

KeywordsMultiobjective Optimization, Ant Colony Optimization, Trav-elling Salesman Problem

1. INTRODUCTIONAnt colony optimization (ACO) [8] is one of the most

prominent swarm intelligence techniques. Its initial applica-tion area was single objective combinatorial problems. How-ever, the available ACO algorithms were soon extended to

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.GECCO’10, July 7–11, 2010, Portland, Oregon, USA.Copyright 2010 ACM 978-1-4503-0072-8/10/07 ...$10.00.

tackle multiobjective combinatorial optimization problems(MCOPs), where the quality of candidate solutions is evalu-ated according to several, conflicting objectives. A few tensof papers have described ACO applications to MCOPs. Inaddition, two recent papers have reviewed many of the avail-able multiobjective ACO (MOACO) algorithms [2, 9]. Al-though some initial proposals of MOACO algorithms focusedon problems that define a relative order between the objec-tives, most MOACO algorithms tackle problems in terms ofPareto optimality.

Previous reviews considered MOACO algorithms as theywere proposed in their original form, trying to make as fewchanges as needed to adapt them to the particular casestudy. They either categorized existing algorithms withoutany empirical comparison [2], or were interested in identify-ing the “best” MOACO algorithm for a particular problemmore in a traditional “horse-race” analysis [9].

In contrast, we make an effort to identify the algorithmiccomponents of each MOACO algorithm that are specific fortackling multiobjective problems. We implement those com-ponents on top of a common underlying ACO algorithm. Wetest both MAX -MIN ant system (MMAS) [19] and antcolony system (ACS) [7] as the underlying ACO algorithms.Finally, we design experiments that systematically test thesealgorithm components. For this experimental analysis wechoose the example application to the biobjective travelingsalesperson problem (bTSP). The experimental comparisonsare done in two stages. First, we highlight for some algo-rithms the influence of some specific MOACO design. Ata second stage, we compare the performance of several ofthese methods and identify the most promising proposals.

Thanks to this approach, we identify in this paper com-monalities between various existing algorithms that havegone largely unnoticed until now. For example, we showhow several MOACO algorithms that were originally formu-lated as multi-colony can easily be cast in an equivalent formas single colony algorithms, using some form of weighted ag-gregation of multiple pheromone and/or heuristic factors. Infact, we identify several MOACO algorithms as particular in-stantiations of a common parametrized MOACO algorithm.

Finally, our systematic experimental analysis allows us todraw a direct connection between the multiobjective algo-rithmic components utilized by existing MOACO algorithmsand the quality and distribution of solutions in the Paretofronts obtained. This knowledge will help to design betterMOACO algorithms in the future.

2. PRELIMINARIESIn this paper, we apply MOACO algorithms to the bTSP,

a problem that is a direct extension of the single objectiveTSP to a biobjective one. In a bTSP instance is given acomplete graph G = (V,A) with n = |V | nodes {v1, . . . , vn},and a set of arcs A. Each arc has assigned a vector of costswith two components c1(vi, vj) and c2(vi, vj), i 6= j, whichcorresponds to the cost of the first and the second objective,respectively. We consider here the symmetric bTSP, where itadditionally holds that cq(vi, vj) = cq(vj , vi), i 6= j, q = 1, 2.The goal in the bTSP is to find the set of Hamiltonian toursp = (p1, . . . , pn) that “minimizes” the total tour cost, whichis given by

fq(p) = cq`vp(n), vp(1)

´+

n−1Xi=1

cq`vp(i), vp(i+1)

´q = 1, 2.

Here we assume that the bTSP is solved without knowingthe preferences of the decision maker. The goal then is toidentify a set of feasible solutions that “minimizes” the ob-

jective vector ~f in the sense of Pareto optimality. A vector~u dominates ~v (~u ≺ ~v) iff ~u 6= ~v and ui ≤ vi, i = 1, . . . , d,where d is the number of objectives. ~u and ~v are nondomi-nated iff ~u ⊀ ~v and ~v ⊀ ~u. We apply the same notation tosolutions, meaning that a solution s dominates another solu-

tion s′ iff ~f(s) ≺ ~f(s′). A solution is Pareto optimal iff there

is no feasible solution s′ for which we have ~f(s′) ≺ ~f(s).Since finding the set of all Pareto optimal solutions for anMCOP is typically intractable, the goal usually becomes todetermine a set of mutually nondominated solutions thatapproximates the Pareto-optimal set.

3. MULTI-OBJECTIVE ACOSince the end of 1990s, different proposals have appeared

in the literature extending single-objective ACO algorithmsfor tackling MCOPs. The majority of these MOACO al-gorithms tackle problems in the Pareto sense, where no apriori assumptions on the preferences of the decision makerare made. Recently, two reviews of MOACO algorithmshave been published [2, 9]. The comprehensive review byGarcıa-Martınez et al. [9] describes the available algorithms,classifies them according to the usage of one or several pher-omone matrices and the usage of one or several heuristic in-formation matrices, and also provides a comparison of someof the original algorithms using the bTSP as a case study.This methodology indicates which of the original algorithmsperforms better for the bTSP. However, one cannot concludewhether the observed performance differences are due to anyparticular, multiobjective-specific algorithmic component ofthe MOACO algorithm, e.g., the use of single versus mul-tiple pheromone matrices, or due to the use of ACS versusant system as the underlying ACO algorithm. The more re-cent review by Angus and Woodward [2] provides a detailedclassification of the available MOACO algorithms based onadditional features, such as the pheromone update and de-cay, the type of solution construction, or how candidate so-lutions are evaluated. In contrast to the earlier review [9],this later review does not provide any experimental analysis,and hence, the practical relevance of the design decisions inMOACO algorithms remains unclear.

Among the available papers on MOACO algorithms, fewdo some experimental analysis of their proposals. Iredi et al.

are among the first to propose ACO extensions for tacklingmultiobjective problems and they investigate some designoptions based on the usage of multiple colonies [11]. Alayaet al. [1] study MOACO algorithms for the multiobjectiveknapsack problem and compare the performance of four vari-ants that differ in the number of colonies and the aggregationof pheromone matrices. An earlier article compared differentdesign options for MOACO algorithms on the biobjectivequadratic assignment problem (bQAP)[13], and this workwas recently extended to the bTSP [12].

There are a number of design questions when extendingACO algorithms to MCOPs. First, since the quality of so-lutions cannot be represented by a single scalar value, themeaning of the pheromone information associated to a so-lution component is unclear in the multiobjective context.In fact, some MOACO algorithms use several pheromonematrices, each one of them associated to a different objec-tive [1, 6, 11]. Another important design decision is thechoice of ants that deposit pheromone. These may be someor all nondominated solutions [11], or a number of the bestsolutions with respect to the objective associated to thepheromone matrix that is updated [1, 6]. During the solu-tion construction, multiple pheromone or heuristic matricesare often combined by means of weights [6, 11], althoughexisting MOACO algorithms differ in the number of weightsand the manner in which the are used.

Interestingly, the idea of using multiple “colonies” of antscan be found in many MOACO algorithms. However, thedefinition of “colony” is far from consistent across these pro-posals. In fact, some authors give the name “colony” toany group of ants that share some characteristics but antsfrom different colonies may still use the same pheromoneand heuristic information to construct solutions. For exam-ple, COMPETants [5] (see also section 4.5) is described asmulti-colony when, in fact, it can be seen as weighted aggre-gation of the pheromone matrices. Iredi et al. [11] charac-terize “colonies” as multi-start versions of single-colony algo-rithms, each of them possibly with different settings, whichmay cooperate through the exchange of solutions. Follow-ing this definition, each colony has independent pheromoneinformation and an associated number of ants that use ex-clusively that information to construct solutions. We believethat this latter definition is more consistent with the idea ofcooperation between independent colonies.

4. ANALYSIS OF MOACO ALGORITHMSIn this section, we review existing MOACO algorithms

with the aim to differentiate between algorithmic designchoices for multiobjective optimization, such as the use ofseveral pheromone matrices, and other design choices, suchas those that concern the underlying ACO algorithm.

Table 1 summarizes the MOACO algorithms reviewed inthis paper. They are classified according to the follow-ing multiobjective algorithmic components: the number ofpheromone matrices, the number of heuristic matrices, themethod used to aggregate multiple pheromone (heuristic)matrices, the number of weights used for aggregation, andthe method used to select the solutions that update the pher-omone information. In the following sections, we will explainhow these algorithmic components suffice to characterize theMOACO algorithms reviewed in this paper.

For each of the MOACO algorithms in Table 1, we ex-amine experimentally specific design choices. To this aim,

Table 1: Taxonomy of MOACO algorithms (d is number of objectives, Na is number of ants).

Algorithm [τ ] [η] Aggregation Num. Weights Update

MOAQ [9, 16] 1 d – d (Λ = {0, 1} if d = 2) Nondominated solutionsP-ACO [6] d d weighted sum Na Best-of-objectiveMACS [3] 1 d weighted product Na Nondominated solutionsBicriterionAnt [11] d d weighted product Na Nondominated solutionsCOMPETants [5] d d weighted sum d+ 1 (Λ = {0, 0.5, 1}) Best-of-objectivemACO-1 [1] d d random (τ), weighted sum (η) d+ 1 (Λ = {0, 0.5, 1}) Best-of-objective (per weight)mACO-2 [1] d d weighted sum d+ 1 (Λ = {0, 0.5, 1}) Best-of-objective (per weight)mACO-3 [1] 1 1 – – Nondominated solutionsmACO-4 [1] d 1 random (τ) 1 (Λ = {0.5}) Best-of-objective

we have implemented all MOACO algorithms in C usingACOTSP [18] as the underlying ACO package, and com-piled with gcc, version 3.4. Experiments are carried out onAMD Opteron 2216 dual-core 2.4 GHz processors with 2 MBL2-Cache under Rocks Cluster GNU/Linux. Given that theimplementation is sequential, all codes run on a single core.

We performed experiments with two underlying ACO al-gorithms, namelyMMAS [19] and ACS [7]. In some cases,we noticed significant differences in performance dependingon the choice of the underlying ACO algorithm; we givesome examples below. Basic parameters follow the defaultvalues defined for MMAS and ACS with two exceptions. Wetypically use ∆τ = 1 for the amount of pheromone depositedby an ant, and the evaporation rate is set to ρ = 0.05. Allalgorithms use m = 24 ants, which is a value close to the 20ants used by Garcıa-Martınez et al. [9] and also divisible by2 and 3 colonies. We do not use local search, since this mayhide differences between other algorithmic components [12].All experiments have been run on the instances kroAB100,kroAB200, euclidAB300 taken from Luis Paquete’s webpageat http://eden.dei.uc.pt/~paquete/tsp. The experimen-tal comparison between pairs of algorithms is done by meansof a graphical technique [14, 15] based on the empirical at-tainment function (EAF) [10]. The EAF of an algorithm de-scribes the statistical distribution of its output across mul-tiple runs by giving an estimated probability of attaining(dominating or being equal to) a point in the objective space.In each figure, for example, Fig. 2, two side-by-side plotsdescribe differences between the EAFs of two algorithms.Each side identifies points where the probability is higherfor one algorithm (the one indicated below the plot) thanfor the algorithm in the other side. Strong differences (plot-ted darker) indicate that one algorithm reaches better per-formance than the other one in that region of the objec-tive space. For reasons of space, we only provide here rep-resentative examples of the results obtained. All remain-der plots can be found in an online supplementary page athttp://iridia.ulb.ac.be/supp/IridiaSupp2010-003.

4.1 MOAQIn the original application of Multiple Objective Ant-Q

(MOAQ) [16], the design of irrigation water distributionnetworks, the order of the objectives was fixed as in lexico-graphical optimization. Garcıa-Martınez et al. [9] mentionthe difficulty to generalize MOAQ beyond its initial appli-cation, and, in particular, to adapt it to the multiobjectiveTSP in terms of Pareto optimality. They decided to usean alternative variant of MOAQ that tries to capture themain ideas for multiobjective optimization proposed by theoriginal MOAQ. In particular, they use a single pheromone

matrix and several heuristic matrices, one for each objec-tive. For the pheromone update they use all nondominatedsolutions of the current iteration P ib, and each ant depositsan amount of pheromone inversely proportional to the sumof the values of each of the two objectives. In our implemen-tation, we do not use the objective function values, whichmay be incommensurable and require normalization, and,for simplicity we let each ant deposit a constant amount ofpheromone ∆τ = 1.0. Garcıa-Martınez et al. [9] further di-vide the ants into two groups, and each group only uses theheuristic information corresponding to one objective. Wereformulate this by giving half of the ants the weight λ = 0and the other half the weight λ = 1 for aggregating themultiple heuristic factors in the action choice rule:

pkij =[τij ]

α ·ˆ(1− λk)η1

ij + λkη2ij

˜βPl∈Nk

i[τil]

α · [(1− λk)η1il + λkη2

il]β

if j ∈ N ki ,

whereN ki is the feasible neighborhood of ant k, that is, those

cities not visited yet by ant k, τij is the pheromone strengthon arc (i, j), η1

ij and η2ij are the heuristic information as-

sociated with the first and second objective, and λk is theweight of ant k, which is either one or zero.

The fact that MOAQ does not aggregate the heuristicinformation for each objective but instead alternates be-tween them, leads to radical differences in the shape ofthe nondominated fronts that are generated depending onwhether heuristic information is used (β = 2) or not (β = 0).When using heuristic information, MOAQ is unable to ob-tain trade-off solutions (those located in the center of thePareto front). However, without heuristic information theresults are (as expected) very poor, but the nondominatedfront has a very different shape (Fig. 1).

1e+05 3e+05

1e

+0

53

e+

05

objective 1

ob

jective

2

β = 0

1e+05 3e+05

1e

+0

53

e+

05

objective 1

ob

jective

2

β = 2

Figure 1: Nondominated solutions obtained by 15runs of MOAQ with β = 0 and β = 2. ACS, 25 000iterations, kroAB200.

4.2 Pareto Ant Colony OptimizationDoerner et al. [6] proposed Pareto Ant Colony Optimiza-

tion (P-ACO), which is characterized by the use of multiplepheromone matrices, one for each objective, aggregated bymeans of a weighted sum. Each ant k uses a different weightλk for aggregating the pheromone matrices. Moreover, pher-omone matrices are updated with the best and second-bestsolution for each objective. In the original paper, the authorsused just a single heuristic matrix, since it was difficult todefine appropriate heuristic information for each objectivein the problem. However, in later publications [17], they usemultiple heuristic matrices, one for each objective, which areaggregated in the same way as the pheromone matrices, andthe decision rule of P-ACO is given by

pkij =

ˆ(1− λk)τ1

ij + λkτ2ij

˜α · ˆ(1− λk)η1ij + λkη

2ij

˜βPl∈Nk

i[(1− λk)τ1

il + λkτ2il]α · [(1− λk)η1

il + λkη2il]β

τ1ij , τ

2ij are the pheromone trails for arc (i, j) for each pher-

omone matrix, η1ij , η

2ij are the heuristic values for the corre-

sponding objectives, and λk is a weight, with 0 ≤ λk ≤ 1.The distinction between using only one or several heuristic

matrices is very important. Figure 2 shows the differencesbetween using a single or multiple heuristic matrices. Witha single heuristic matrix, a better performance is obtainedin the center of the approximation to the Pareto front, whilethe usage of multiple heuristic information matrices leads toclearly better results on the extremes of the front.

4.3 BicriterionAntIredi et al. [11], besides giving a general discussion of

how to tackle MCOP problems with ACO algorithms, pro-posed also one specific MOACO algorithm that uses multiplepheromone and heuristic matrices, which are aggregated byweighted product as

pkij =

h(τ1ij)

(1−λk) · (τ2ij)

λk

iα·h(η1ij)

(1−λk) · (η2ij)

λk

iβPl∈Nk

i[(τ1

il)(1−λk) · (τ2

il)λk ]

α · [(η1il)

(1−λk) · (η2il)λk ]

β

Here, each of τ1ij and τ2

ij is the pheromone trail strength as-sociated with each of the two pheromone matrices. A weightλk is associated to each ant. In the original proposal, Irediet al. [11] suggest to update the pheromone matrices by us-ing the nondominated front in the current iteration P ib byan amount equal to ∆τ = 1/|P ib|. However, this approachonly works for heterogeneous pheromone matrices, that is,when each matrix is mapped to different solution compo-nents. In the bTSP (bQAP and other problems), this is notthe case, and such an update will result in multiple identi-cal pheromone matrices. Garcıa-Martınez et al. [9] thereforepropose to use the objective function value of each objectiveto update each matrix ∆τ q = 1/fq(sa).

In addition, Iredi et al. [11] proposed the use of multiplecolonies, where a colony is a group of ants that constructsolutions according to some particular pheromone informa-tion. Different colonies may be forced to specialize in differ-ent regions of the Pareto frontier by various means. First,when using weights to aggregate pheromone information,the set of weights available are divided among the differentcolonies. Second, before updating the pheromone matrices,the solutions from all colonies are put in a single archive (ei-ther the iteration-best or best-so-far archive) and dominated

solutions are removed. This way all colonies contribute in-formation to identify the best solutions. Third, the solutionsare then reassigned to the various colonies either by updateby origin or by update by region. In the first case, eachsolution is used for updating the pheromone information ofthe colony that generated it. In the second case, each colonymay be (loosely) assigned a region of the Pareto frontier, andsolutions update the pheromone information of the colonyof its corresponding region. In the biobjective case, a trivialimplementation is to sort the nondominated front accordingto one objective, divide it into as many equal-sized parts ascolonies, and use each part to update each colony.

The effect of using one or several colonies is shown in Fig-ure 3. The usage of three colonies results in clearly superiorperformance when compared to the single colony case.

The choice between a weighted sum aggregation (analo-gously to P-ACO) or a weighted product aggregation (thedefault in the proposal of Iredi et al. [11]) of the pheromoneand heuristic information may also affect the results. In Bi-criterionAnt, the usage of the weighted product aggregationappears to be better (Fig. 4); and the differences are strongerwhen algorithms stop after the same number of iterations.

4.4 Multiple Ant Colony SystemThe defining characteristic of Multiple Ant Colony Sys-

tem (MACS) [3] is the use of one heuristic matrix for eachobjective and a single pheromone matrix. The heuristic ma-trices are aggregated by weighted product, and each ant usesa different weight. The decision rule in MACS is

pkij =[τij ]

α ·h(η1ij)

(1−λk) · (η2ij)

λk

iβPl∈Nk

i[τil]

α · [(η1il)

(1−λk) · (η2il)λk ]

βif j ∈ N k

i ,

In addition, pheromone information is updated with non-dominated solutions. Finally, note that MACS can be seenas a generalization of the MOAQ defined by Garcıa-Martınezet al. [9] that uses more than two weight vectors to aggregatethe heuristic information.

4.5 COMPETantsCOMPETants [5] is described as a multi-colony approach,

with one colony for each objective. Each “colony” has onepheromone and heuristic matrix and constructs solutions in-dependently, except for a number of ants (called “spies”),which aggregate the two pheromone matrices by weightedsum (with λ = 0.5) using either the first or the secondheuristic matrix (thus creating two solutions). Finally, anumber of ants Nup from each “colony” are used to updatethe pheromone matrix of each “colony”.

However, COMPETants can be implemented as a singlecolony algorithm that uses two pheromone and heuristic ma-trices, which are aggregated by weighted sum. The onlyweights used are Λ = {0, 0.5, 1}. The ants using λ = 0.5are special in the sense that they do not aggregate bothheuristic matrices but they create one solution using onlyone of the heuristic matrices. Finally, each pheromone ma-trix would be updated with the Nup best solutions for thecorresponding objective (best-of-objective update), which isbasically the same update method used by P-ACO.

For the sake of simplicity, we disregard the fact that in theoriginal algorithm the number of ants for each weight waschosen adaptively based on the algorithm progress; here eachweight is used by one third of the total number of ants.

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

Single Heuristic

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

Multiple Heuristic

Figure 2: EAF differences between using single (left)or multiple (right) heuristic matrices in P-ACO.MMAS, 200 seconds, kroAB200.

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

One colony

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

Three colonies

Figure 3: EAF differences between using either oneor three colonies in BicriterionAnt. MMAS, 300 sec-onds, euclidAB300

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

Weighted product

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

Weighted sum

Figure 4: EAF differences between using eitherweighted product or weighted sum aggregation inBicriterionAnt. MMAS, 300 seconds, euclidAB300.

4.6 mACO Variant 1 (mACO-1)Alaya et al. [1] proposed four alternatives for the design of

a MOACO algorithm. The first variant (mACO-1) has one“colony” per objective, and an “extra colony” that builds so-lutions by aggregating the pheromone matrices of the othertwo colonies in a stochastic manner. In particular, at eachconstruction step, ants from the extra colony select ran-

domly one objective and use the pheromone information as-sociated to that objective. Each colony considers heuristicinformation for its corresponding objective, whereas the ex-tra “colony” aggregates these multiple heuristic factors intoa single heuristic factor. The pheromone information of eachcolony is updated by solutions of the same colony. Only onesolution is used to update each colony, the one correspond-ing to the associated objective. The update of the extracolony is slightly different. It keeps a best solution for eachobjective, and updates each pheromone matrix with the cor-responding solution.

This proposal can be formulated in terms of a single colonyalgorithm with multiple pheromone matrices and multipleheuristic matrices. The algorithm makes use of three weightsλ = {0, 0.5, 1}, a random aggregation rule for the pheromoneinformation and a weighted sum aggregation for the heuristicinformation. Therefore, the decision rule in mACO-1 can bedefined as

pkij =

ˆτrij˜α · ˆ(1− λk)η1

ij + λkη2ij

˜βPl∈Nk

i[τril]

α · [(1− λk)η1il + λkη2

il]β

if j ∈ N ki , (1)

where τr is a pheromone matrix that is randomly chosenwith a probability λ

τr ∼ rand(λ){τ1, τ2} =

(τ1 if U(0, 1) < 1− λ,

τ2 otherwise.

where U(0, 1) is an uniform random value within [0, 1) andλ is a parameter that biases the choice.

4.7 mACO Variant 2 (mACO-2)The second variant (mACO-2) by Alaya et al. [1] only

differs from mACO-1 in the pheromone aggregation. InmACO-2, pheromone matrices are aggregated by summingthe pheromone matrices of each objective. In fact, this corre-sponds to a weighted sum aggregation using a weight vector(1, 1). For both MMMAS and ACS, there is little differencebetween this and using λ = 0.5 [4]. Hence, mACO-2 is verysimilar to P-ACO (Section 4.2) but using only 3 weights(Λ = {0, 0.5, 1}), and several ants per weight.

Differences in quality between mACO-1 and mACO-2 ap-pear to be minor. The shape of the nondominated frontsis essentially the same and there are only minor differencesamong the performance of the two algorithms (Fig. 5).

4.8 mACO Variant 3 (mACO-3)The third variant proposed by Alaya et al. [1] uses in-

stead a single pheromone matrix. This pheromone matrix isupdated by using nondominated solutions. In particular,

∆τij =

(1 if {i, j} appears in a solution in P0 otherwise.

where P is a set of nondominated solutions. In general, itcould be the set of nondominated solutions found in the cur-rent iteration (iteration-best set), or since the start of therun (best-so-far set). Alaya et al. [1] emphasize that everypheromone value is updated only once despite how many so-lutions contain it. This is in contrast with other algorithmsthat use a “nondominated update” such as MOAQ, MACS,and BicriterionAnt. The heuristic factor is also a single ma-trix. Since in some problems there are several heuristic fac-tors per objective, these are aggregated in a fix manner into

a single factor ηij = η1ij + η2

ij . Finally, since there is a singlepheromone matrix and a single heuristic matrix, the deci-sion rule of mACO-3 is the same as for the single-objectiveACO algorithms.

4.9 mACO Variant 4 (mACO-4)In the last variant proposed by Alaya et al. [1], there is one

pheromone matrix per objective, which are aggregated inthe same random manner as for mACO-1 (Eq. 1). However,there is only one heuristic factor, which may be obtainedby aggregating individual heuristic factors (in the original

publication, by ηij =Pdk=1 η

kij). Finally, each pheromone

matrix is updated with the best solution for each objective.The comparison of mACO-3 and mACO-4 is somehow in-

conclusive. When the underlying ACO algorithm is MMAS,mACO3 is often the best (but not always; among the testswe have run, for kroAB100 with computation time limits,mACO4 is better); when the underlying ACO algorithm isACS, mACO4 is always the best (Fig. 6).

5. COMPARISON OF ALGORITHMSAs a final step, we compare the MOACO algorithms. Given

the limited space, a pairwise comparison of the MOACO al-gorithms was deemed not to be feasible. Instead, we choosethe BicriterionAnt algorithm with three colonies (because ofits overall very good performance) as the baseline to whichwe compare all others. In Figures 7 to 9 we show the dif-ferences in the empirical attainment functions obtained onthe largest instance, euclidAB300, where also the differencesamong the algorithms have been most pronounced.

BicriterionAnt is clearly superior to MACS and COM-PETants across the largest part of the nondominated front.Only in the extremes towards the best solutions for each ofthe single objectives, MACS and to a somewhat larger ex-tent COMPETants obtain slightly better solutions. Whencompared to the mACO-x variants, BicriterionAnt obtainsclearly better results. In particular, mACO-4 is dominatedby the results of BicriterionAnt (similar results hold formACO-3, which generates fronts of a similar shape as mACO-4); mACO-2 performs in very small parts of the nondomi-nated front slightly better than BicriterionAnt.

Finally, the comparison to P-ACO with either one or mul-tiple heuristic matrices confirms the large differences amongthese two variants. Except for the extremes of the nondom-inated front towards each of the single objectives, P-ACOwith multiple pheromone matrices is clearly of inferior per-formance to BicriterionAnt. This result can be seen as anindication that the combination of solution components thatare very good for each of the single objectives are not neces-sarily appropriate to generate good solutions in the center ofthe nondominated front. A very different behavior is givenby P-ACO with a single pheromone matrix. With this vari-ant, very good results in the center of the front are obtained,improving quite strongly over those of BicriterionAnt in thisarea. However, the solutions generated are not very widelyspread and therefore BicriterionAnt performs better on awider range of the objective space.

6. CONCLUSIONSThere are a number of interesting observations that can

be taken from our review. The first is that many existingMOACO algorithms share more similarities than differences.

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−1

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−2

5e+04 2e+05 3e+05 4e+05

objective 11e+

05

3e+

05

obje

ctive 2

mACO−1

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−2

Figure 5: EAF differences between mACO-1 andmACO-2. MMAS, 25 0000 iterations (top) and 200seconds (bottom), kroAB200.

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−3

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−4

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−3

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

5e+04 2e+05 3e+05 4e+05

objective 1

1e+

05

3e+

05

obje

ctive 2

mACO−4

Figure 6: EAF differences between mACO-3 andmACO-4. MMAS (top) and ACS (bottom), 200 sec-onds, kroAB200.

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

BCritAnt (3 colonies)

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

MACS

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

BCritAnt (3 colonies)

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

COMPETants

Figure 7: EAF differences between BicriterionAnt(3 colonies), MACS and COMPETants. MMAS, 300secs, euclidAB300.

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

BCritAnt (3 colonies)

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

mACO−2

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

BCritAnt (3 colonies)

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

mACO−4

Figure 8: EAF differences between BicriterionAnt(3 colonies), mACO-2 and mACO-4. MMAS, 300secs, euclidAB300.

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

BCritAnt (3 colonies)

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

PACO

0 1.5e+05 3.5e+05 5.5e+05

objective 10

2e+

05

4e+

05

6e+

05

obje

ctive 2

BCritAnt (3 colonies)

[0.8, 1.0]

[0.6, 0.8)

[0.4, 0.6)

[0.2, 0.4)

[0.0, 0.2)

0 1.5e+05 3.5e+05 5.5e+05

objective 1

02e+

05

4e+

05

6e+

05

obje

ctive 2

PACO (single heuristic)

Figure 9: EAF differences between BicriterionAnt(3 colonies) and P-ACO with multiple (top) andone (bottom) heuristic matrix. MMAS, 300 secs,euclidAB300.

By using the definition of multiple colonies proposed by Irediet al. [11], some of the “multi-colony” MOACO algorithmscannot be said to use multiple colonies, and in fact, theybecome extreme variants (using 3 weights) of P-ACO [17].This is the case of COMPETants, mACO-1, and mACO-2.Another conclusion is that the variant of MOAQ proposedby Garcıa-Martınez et al. [9] is an extreme case (with asmany weights as objectives) of MACS [3].

Our computational analysis confirms some conclusions ob-tained by Garcıa-Martınez et al. [9]. For example, COM-PETants and MOAQ are rather poor performing algorithmsfor the bTSP, whereas MACS and BicriterionAnt are amongthe best performing ones. However, our analysis shows thatthe poor performance of COMPETants and MOAQ is be-cause they do not aggregate the multiple heuristic infor-mation they use, which makes them unable to obtain solu-tions in the center of the nondominated front. On the otherhand, algorithms that aggregate multiple heuristic matricesby means of several weights, such as MACS and Bicriterio-nAnt, obtain a much better distribution of solutions alongthe nondominated front. Finally, the algorithms that ag-gregate the heuristic information in a fixed way (mACO-3,mACO-4, and the single heuristic variant of P-ACO [9]) ob-tain very narrow Pareto fronts but of very high quality. Oursystematic analysis allows us to claim that these results arenot caused by some particularity of the underlying ACO al-gorithm but entirely due to the multiobjective algorithmicdesign choices of each proposed algorithm.

Another interesting observation is that the use of a mod-

erate number of colonies in BicriterionAnt, which impliesseveral pairs of pheromone matrices, turns out to improvethe results noticeably along the whole nondominated front.This result is in stark contrast with the fronts obtained byP-ACO, mACO-1 and mACO-2, which use just one pair ofpheromone matrices.

For the bTSP, we conclude that the use of multiple heuris-tic factors is essential to obtain a wide spread Pareto front.This is not enough, however, and several weights (definitelymore than 3) must be used to aggregate these heuristic fac-tors. As it has been observed before, the use of a smallnumber of colonies further improves the results. We stronglybelieve, that these conclusions apply generally to problemswhere the heuristic information is important.

The analysis presented here should be extended to prob-lems with different characteristics. Particularly interestingwould be the application to the bQAP, where heuristic in-formation is not useful. Moreover, further research shouldtest the interactions between these components and localsearch. The results here should provide guidelines for prob-lems similar to the bTSP, but more work is necessary tohave a complete understanding for other types of problems.

Acknowledgments. This work was supported by the META-

X project, an Action de Recherche Concertee funded by the Sci-

entific Research Directorate of the French Community of Bel-

gium. Thomas Stutzle acknowledges support from the Belgian

F.R.S.-FNRS, of which he is a Research Associate. The authors

also acknowledge support from the FRFC project “Methodes de

recherche hybrides pour la resolution de problemes complexes”.

References[1] I. Alaya, C. Solnon, and K. Ghedira. Ant colony opti-

mization for multi-objective optimization problems. InProc. of ICTAI 2007, pages 450–457, Los Alamitos, CA,2007. IEEE Computer Society Press.

[2] D. Angus and C. Woodward. Multiple objective antcolony optimization. Swarm Intelligence, 3(1):69–85,2009.

[3] B. Baran and M. Schaerer. A multiobjective ant colonysystem for vehicle routing problem with time windows.In Proc. of 21st IASTED Int. Conf. on Applied Infor-matics, pages 97–102, Insbruck, Austria, 2003.

[4] M. Birattari, P. Pellegrini, and M. Dorigo. On the in-variance of ant colony optimization. IEEE Transactionson Evolutionary Computation, 11(6):732–742, 2007.

[5] K. F. Doerner, R. F. Hartl, and M. Reimann. Are Com-petAnts more competent for problem solving? The caseof a multiple objective transportation problem. Cen-tral European Journal for Operations Research and Eco-nomics, 11(2):115–141, 2003.

[6] K. F. Doerner, W. J. Gutjahr, R. F. Hartl, C. Strauss,and C. Stummer. Pareto ant colony optimization: Ametaheuristic approach to multiobjective portfolio se-lection. Annals of Operations Research, 131:79–99,2004.

[7] M. Dorigo and L. M. Gambardella. Ant Colony Sys-tem: A cooperative learning approach to the traveling

salesman problem. IEEE Transactions on EvolutionaryComputation, 1(1):53–66, 1997.

[8] M. Dorigo and T. Stutzle. Ant Colony Optimization.MIT Press, Cambridge, MA, 2004.

[9] C. Garcıa-Martınez, O. Cordon, and F. Herrera. A tax-onomy and an empirical analysis of multiple objectiveant colony optimization algorithms for the bi-criteriaTSP. European Journal of Operational Research, 180(1):116–148, 2007.

[10] V. Grunert da Fonseca, C. M. Fonseca, and A. O. Hall.Inferential performance assessment of stochastic opti-misers and the attainment function. In E. Zitzler etal., editors, EMO 2001, volume 1993 of LNCS, pages213–225. Springer, Heidelberg, 2001.

[11] S. Iredi, D. Merkle, and M. Middendorf. Bi-criterion op-timization with multi colony ant algorithms. In E. Zit-zler et al., editors, EMO 2001, volume 1993 of LNCS,pages 359–372. Springer, Heidelberg, 2001.

[12] M. Lopez-Ibanez and T. Stutzle. An analysis of algo-rithmic components for multiobjective ant colony opti-mization: A case study on the biobjective TSP. In EA2009, volume 5975 of LNCS, pages 134–145. Springer,Heidelberg, 2009.

[13] M. Lopez-Ibanez, L. Paquete, and T. Stutzle. On thedesign of ACO for the biobjective quadratic assignmentproblem. In M. Dorigo et al., editors, Proc. of ANTS2004, volume 3172 of LNCS, pages 214–225. Springer,Heidelberg, 2004.

[14] M. Lopez-Ibanez, L. Paquete, and T. Stutzle. Hy-brid population-based algorithms for the bi-objectivequadratic assignment problem. Journal of Mathemati-cal Modelling and Algorithms, 5(1):111–137, 2006.

[15] M. Lopez-Ibanez, L. Paquete, and T. Stutzle. Ex-ploratory analysis of stochastic local search algorithmsin biobjective optimization. In T. Bartz-Beielstein etal., editors, Experimental Methods for the Analysis ofOptimization Algorithms. Springer, 2010. To appear.

[16] C. E. Mariano and E. Morales. MOAQ: An Ant-Q al-gorithm for multiple objective optimization problems.In W. Banzhaf et al., editors, Proc. of GECCO-1999,pages 894–901. Morgan Kaufmann, San Francisco, CA,1999.

[17] M. Schilde, K. F. Doerner, R. F. Hartl, and G. Kiechle.Metaheuristics for the bi-objective orienteering prob-lem. Swarm Intelligence, 3(3):179–201, 2009.

[18] T. Stutzle. ACOTSP: A software package of vari-ous ant colony optimization algorithms applied to thesymmetric traveling salesman problem, 2002. URLhttp://www.aco-metaheuristic.org/aco-code/.

[19] T. Stutzle and H. H. Hoos. MAX–MIN Ant System.Future Generation Computer Systems, 16(8):889–914,2000.