1

Ant Colony OptimisationAlexander Mathews, Angeline Honggowarsito & Perry Brown

Image Source: http://baynature.org/articles/the-ants-go-marching-one-by-one/

2

Contents

● Introduction to Ant Colony Optimisation● Natural Inspiration● Ant System

● Advanced Ant Colony Optimisation● Extensions to Ant System● Max-Min Ant System● Best-Worst Ant System

● Applications of Ant Colony Optimisation● The Generalised Assignment Problem (Static)● Packet-switched network Routing (Dynamic)

3

ACO:Introduction

●Originally proposed by Marco Dorigo in 1992●To solve the Travelling Salesman Problem (TSP)●By simulation the movements of ants

●The original ACO was not competitive with specialised TSP algorithms●Many improvements have been proposed

●ACO can be extended to solve generic optimisation problems●Most applications are to routing and scheduling problems

4

Natural Inspiration

Model the behaviour of ants searching for food● Model based on experimental observations● Ants mark a path to food using pheromone● Other ants can detect the pheromone trail● Ants tend to follow paths with more

pheromone

This is location based communication

An ant trail. Source: Wikipedia, Ants, Retrieved 12/9/12

5

Natural Inspiration Example● Ants leave the nest at

regular intervals

● 50% take the top path

● 50% take the lower path

● Ants which find food return to the nest by a random path

● Ants taking the top path return first

● The top path starts to accumulate pheromone at A

● Ants leaving the nest tend to take the top path

● Has more pheromone

Graphics adapted from: Dorigo et al. 2006

6

Towards an Algorithm

Want to solve different problems those solved by real ants● Intermediate nodes: are components of a solution● Edges: are possible connections between components● Food Sources: are a complete solutions (with a fitness)

Virtual ants are allowed:● Perfect memory: ants can exactly retrace their steps● Variable pheromone strength: strength related to quality of

solution found

7

Towards an Algorithm

Typical example Travelling Salesman Problem (TSP)

(Find the shortest cycle that visits every city once and only once)

● Intermediate nodes: are cities● Edges: are paths between cities● Food: A completed cycle of the graph

Note: that nodes are cities not partially constructed tours.

8

Ant System (AS)

We have:

● A population of ants (starting at random cities)● Ants which construct one solution per iteration● A simple heuristic giving the 'fitness' of an edge ● Only pheromone information kept to the next

iteration

9

Ant System: Overview

While not done:

randomise ant positions

For each ant: Construct a random tour

For each edge: Evaporate pheromone

For each ant: Lay pheromone on each edge in tour

10

Ant System: Overview

● Evaporate pheromone from every edge

● Increase the pheromone on edges in the random tour

● Construct a random tour

● Initialise the pheromone for every edge

11

Ant System: Overview

After a number of iterations we get the optimal tour.

Hopefully.

12

Ant System: Ant Movement

Until a solution is completed:

For each edge to a node not in the tour:

Calculate heuristic for edge

Check pheromone strength for edge

Calculate attractiveness of edge

Randomly select edge with probability based on attractiveness

Attractiveness is calculated as:

(need normalise across outgoing edges to get a probability)

(Pheromone)α∗(Heuristic)β

13

Ant System: Ant Movement

● Calculate Attractiveness

● A(A) = 28.3

● A(B) = 8.9

● Convert to Probability

● P(A) = 0.76

● P(B) = 0.24

● An edge to (A) is likely

● Discard edges to nodes already in the tour

● Get the Heuristic and Pheromone values

● H(A) = 300-100 = 200

● H(B) = 300-220 = 80

● P(A) = 2, P(B) = 1

● An ant is at the green point trying to complete the tour

14

Parameters

N : Number of ants

α : Influence of the pheromone trail

β : Influence of the heuristic

ρ : Fraction of pheromone that evaporates

● Low evaporation leads to slower convergence & possibly better solutions

Q : scales the strength of the pheromone laid

(Pheromone)α∗(Heuristic)β

Pheromone Strength=Q

Length of Tour

15

Termination Conditions

● Iteration/Time limit● Solution is good enough● Stagnation: Only one tour will be tried as the

pheromone is too strong.

Or: run forever to maintain a good solution to a dynamic problem.

16

Contents

● Introduction to Ant Colony Optimisation● Natural Inspiration● Ant System

● Advanced Ant Colony Optimisation● Extensions to Ant System● Max-Min Ant System● Best-Worst Ant System

● Applications of Ant Colony Optimisation● The Generalised Assignment Problem (Static)● Packet-switched network Routing (Dynamic)

17

Elitist Ant System

• First improvement on Ant System

• Also called Elitist Strategy for Ant System

• Provide strong additional reinforcement to the arcs belonging to the best tour found

• Pheromone update rule

18

Elitist Ant System

• Appropriate value for parameter e allows EAS to:

• Find better tours

• Find them in a lower number of iterations

• Evaporation rule is implemented as in Ant System

19

Rank-Based Ant System

• Introduced by Bullnheimer, Hartl and Strauss in 1997

• Ranking of pheromone update

• The m ants are ascending ranked based on solutions’ quality . (S’1, S’2, ....... , S’m)

• Each ants deposit an amount of pheromone that decrease with its rank

• Experimental evaluation suggests that this system performs slightly better than EAS and significantly better than AS.

20

Ant Colony System

• Another extensions of Ant System

• Three major improvements

• Different transition rule, pseudo-random proportional rule

• Probability Distribution

21

ACS - Improvement

• Daemon triggers pheromone update

• Ant with global best solution, Sglobal-best

• Pheromone evaporation before update

• Online step-by-step pheromone trail update

• Rule:

22

Max-Min Ant System (MMAS)

• Developed by Stuzzle and Hoos in 1996

• AS extension

• Similar to ACS, offline pheromone trail update

• Best Ant chosen to add pheromone

• Iteration-best ant or best-so-far ant

• Improve ant’s solutions using local optimisers

23

MMAS - Extensions

• Pheromone trails range [ Tmin, Tmax ]

• Tmax :

• Tmin : constant factor lower than Tmax

• Pheromone trails are initialised to maximum allowed value

• Re-initialization of pheromone trails

24

Best-Worst Ant System

• Proposed by Cordon et al. in 1999

• Incorporates Evolutionary Computation

• Transition rule and pheromone evaporation

• Similar with MMAS, use local optimisers to improve ant’s solutions

25

Best-Worst Ant System

• Improvements:

• The best-worst pheromone trail update rule.

• Penalise all connection of the worst solution. Scurrent-worse

• Update rule:

26

BWAS Improvement• Pheromone trail mutation for diversity in search

process.

• Pheromone is mutated with probability Pm.

• Adding or substracting the same amount in each iteration

• mut(it,Tthreshold) depend on the average of pheromone trails

• Re-initialization of the pheromone trail, setting every pheromone to T0

27

Contents

● Introduction to Ant Colony Optimisation● Natural Inspiration● Ant System

● Advanced Ant Colony Optimisation● Extensions to Ant System● Max-Min Ant System● Best-Worst Ant System

● Applications of Ant Colony Optimisation● The Generalised Assignment Problem (Static)● Packet-switched network Routing (Dynamic)

28

Further applications of ACO• Two major types of problem that ACO is particularly suited to

• (Static) NP-hard problems, including:

– Routing problems (e.g. TSP)

– Assignment problems (e.g. university timetabling)

– Scheduling problems (e.g. scheduling project tasks)

– Subset problems (e.g. multiple knapsack problem)

• Dynamic shortest path problems

– e.g. routing in packet-switched networks

• N.B.: The terms static and dynamic here refer to whether or not the problem instance changes at run-time

29

Defining the pheromone trails• Key consideration for applications: decide what the pheromone

trails should signify

• Relative positioning

– Trails represent the desirability of choosing a particular solution component j directly after some other component i

– Intuitively suited to problems for which the position of components in the overall sequence is inconsequential (e.g. in TSP, no preference as to which cities get visited sooner or later)

• Absolute positioning

– Trails represent the desirability of choosing a particular solution component j at the ith position in the sequence

– Intuitively suited to problems for which the position of components in the overall sequence is significant (e.g. in scheduling project tasks, different tasks are due by different dates)

• Either approach is possible for any problem, but the intuitive choices described above generally give much better performance

30

Example application: Generalised Assignment Problem (GAP)

• Example of a static NP-hard problem

• A number of tasks to be distributed among a number of agents, each with a limited capacity

• Assigning a particular task to a particular agent involves using a certain amount of that agent’s capacity

• Each task-to-agent assignment also incurs a specific cost

• Goal is to minimise the total cost of assigning all tasks within the agents’ capacity constraints

31

Representing the problem graphically• Both tasks and agents represented by nodes

• Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned

Tasks

Agents

32

Representing the problem graphically• Both tasks and agents represented by nodes

• Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned

Tasks

Agents

Assignment

33

Representing the problem graphically• Both tasks and agents represented by nodes

• Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned

Tasks

Agents

Assignment

Move on to next task

34

Representing the problem graphically• Both tasks and agents represented by nodes

• Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned

Tasks

Agents

Assignment

Move on to next task

35

Aspects of the problem representation• Two types of pheromone deposits

– One to determine which agent to assign a particular task to (relative)

– Other to determine best order in which to assign tasks (absolute)

• Heuristic information (static)

– Our pre-existing knowledge of this particular problem suggests that assignments with low cost and/or capacity usage should be preferred

• Range of feasible solutions limited by capacity constraints

– Strategy: assign a task to an agent with sufficient remaining capacity whenever possible

– Ants that produce an infeasible solution (i.e. one that violates capacity constraints) may have their pheromone contribution reduced

36

Example application: packet-switched network routing (dynamic)

• Problem involves a network with links connecting pairs of nodes, reaching one node from another may involve traversing a number of links in sequence

• Each link has a cost associated with using it

• Goal to find minimum cost path between every pair of nodes

• Normally an easy problem, except that in this instance the link costs are subject to change over time

• ACO compared with several state-of-the-art algorithms for this application, performed superior to all of them

37

Aspects of the problem representation• Two types of pheromone deposits

– One to determine which agent to assign a particular task to (relative)

– Other to determine best order in which to assign tasks (absolute)

• Heuristic information (static)

– Our pre-existing knowledge of this particular problem suggests that assignments with low cost and/or capacity usage should be preferred

• Range of feasible solutions limited by capacity constraints

– Strategy: assign a task to an agent with sufficient remaining capacity whenever possible

– Ants that produce an infeasible solution (i.e. one that violates capacity constraints) may have their pheromone contribution reduced

38

References

• M. Dorigo and T. Stutzle, "The ant colony optimization metaheuristic: algorithms, applications, and advances," in Handbook of Metaheuristics, Eds. F. Glover and G. Kochenberger, New York: Springer, 2003, pp. 250-285.

• M. Dorigo et al., "Ant colony optimization," IEEE Comput. Intell. Mag, vol. 1, (4), pp. 28-39, Nov. 2006.

• G. Di Caro and M. Dorigo, "AntNet: distributed stigmergetic control for communications networks," J. Artif. Int. Res., vol. 9, (1), pp. 317-365, Aug. 1998.

39

References

• B. Bullnheimer, R. F. Hartl and C. Strauss, A New Rank Based Version of the Ant System: A Computational Study. Central European Journal for Operations Research and Economics, 7(1):25-38, 1999.

• O. Cordon, F. Herrera, and T. Stützle, A Review on the Ant Colony Optimization Metaheuristic: Basis, Models and New Trends. Mathware and Soft Computing, 9(2-3), pp. 141--175, 2002h

40

References

• M.Dorigo and L. M. Gambardella. Ant Colony System: A cooperative learning approach to the travelling salesman problem. IEEE Transaction on Evolutionary Computation, 1(1):53-66, 1997.

• T. Stützle and H. H. Hoos, MAX-MIN Ant System. Future Generation Computer Systems

41

References

• O.Cordon, I. Fernandez de Viana and F. Herrera and L. Moreno. A new ACO model integration evolutionary computation concepts: The best-worst Ant System. In M. Dorigo, M. Middendorf, and T.Stutzle, editors, Abstract proceedings of ANTS2000 - From Ant Colonies to Artificial Ants: A Series of International Workshops on Ant Algorithms, pages 22 - 29, IRIDIA, Universite Libre de Bruxelles, Belgium, 2000.