Algorithms and their Applications

CS2004 (2012-2013)

Dr Stephen Swift

14.1 Ant Colony Optimisation and Particle Swarm Optimisation

Previously On CS2004…So far we have looked at:

Concepts of computation and algorithmsComparing algorithmsSome mathematical foundationThe Big-Oh notationComputational complexityData structuresSorting algorithmsHill Climbing and Simulated AnnealingTabu Search and Iterated Local SearchApplications and OptimisationGenetic AlgorithmsEvolutionary Computation and Related Topics

Ant Colony Optimisation and Particle Swarm Optimisation Slide 2

Ant Colony Optimisation and Particle Swarm Optimisation Slide 3

This Lecture

In this lecture we are going to cover:

Swarm IntelligenceAnt Colony OptimisationParticle Swarm Optimisation

Ant Colony Optimisation and Particle Swarm Optimisation Slide 4

Swarm IntelligenceA new field of study

<25 years oldBefore the mid 80s Artificial Intelligence (AI) was:

Symbol basedFocused on reasoning, knowledge, planning, ...

New problem domains emerged that involved the environment changing dynamically

E.g. telecommunications and schedulingThese problems cannot be framed by calculations and simple algorithms

Holland’s Genetic Algorithm created a new way of thinkingBehaviour based AI

Intelligence is controlled by a set of independent semi autonomous modules

Ant Colony Optimisation and Particle Swarm Optimisation Slide 5

Recap: ComplexityThe interaction of many simple parts creating complex behaviour

Not to be confused with Computational Complexity) – but easily done!

The net effect is greater than the sum of the individuals, e.g. ant coloniesEmergent behaviour as a side effect of the systemOften incorporates elements of Chaos Theory, Artificial Life and Artificial Intelligence

Ant Colony Optimisation and Particle Swarm Optimisation Slide 6

Social Insects

Several million years of successEfficientFlexibleRobust

Can solve many problems:Find food, feed the brood, defend the nestBuild a nest …

Ant Colony Optimisation and Particle Swarm Optimisation Slide 7

Flocking“Boids” model created by Craig Reynolds in 1987Boids =“bird-oid” objects (also schooling fish)Video links:

http://www.red3d.com/cwr/boids/http://www.youtube.com/watch?v=Psq0FSOF_xU&feature=related

Only three simple rules …

Ant Colony Optimisation and Particle Swarm Optimisation Slide 8

Boid RulesRule 1: Separation

Steer to avoid crowding local flock-mates

Rule 2: Alignment Steer towards the average heading of local flock-mates

Rule 3: Cohesion Steer to move toward the average position of local flock-mates

Ant Colony Optimisation and Particle Swarm Optimisation Slide 9

Swarm Intelligence – Part 1

First used by Beni, Hackwood and Wang in 1989 for work on cellular robotic systems

Later: for anything swarm inspired

Ant Colony Optimisation and Particle Swarm Optimisation Slide 10

Swarm Intelligence – Part 2Study of collective behaviour of decentralised, self-organised systems

No central control

Only simple rules for each individualSimple but extremely powerfulThe problems are usually difficult to defineSolutions result from the behaviour and interactions between individual agentsSolutions are emergent in the systems

Ant Colony Optimisation and Particle Swarm Optimisation Slide 11

Swarm Intelligence Algorithms

Most Popular AlgorithmsAnt Colony Optimisation (ACO)Particle Swarm Optimisation (PSO)

More esotericBacteria Colony OptimisationPlant Propagation Algorithms

Strawberry Plant

Ant Colony Optimisation and Particle Swarm Optimisation Slide 12

Ant Colony OptimisationFirst proposed by Marco Dorigo in 1992A Heuristic optimisation method inspired by biological systemsA multiple agent based approach for solving difficult combinatorial optimisation problems

Mainly Graph based problemsTraveling Salesman, vehicle routing, sequential ordering, graph colouring, routing in communications networks, etc…

Ant Colony Optimisation and Particle Swarm Optimisation Slide 13

Ant BehaviorAnts (blind) navigate from nest to food sourcesThe shortest path is discovered via pheromone trails

Each ant moves at randomPheromone is deposited onto the pathAnts detect the lead ants path and are inclined to followMore pheromone on the path means an increased probability of the path being followedPheromone upgrade: evaporation

Ant Colony Optimisation and Particle Swarm Optimisation Slide 14

Dealing With ObstaclesThe more ants follow a trail, the more attractive to follow

that trail becomes

Ant Colony Optimisation and Particle Swarm Optimisation Slide 15

StigmergyIndirect coordination/communication between agents or actions

Individual behaviour modifies the environment, which in turn modifies the behaviour of other individualsStimulates the performance of subsequent actions leading to the spontaneous emergence of coherent, apparently systematic activityReduces (or eliminates) communications between agentsSupports efficient collaboration between simple agentsProduces complex, seemingly intelligent structures, without need for any planning, control, or even direct communication between the agents

Ant Colony Optimisation and Particle Swarm Optimisation Slide 16

Route SelectionAt the beginning of the search process, a constant amount of pheromone is assigned to all arcsWhen located at a node i an ant k uses the pheromone trail to compute the probability of choosing j as the next node:

The probability is zero for nodes that are unreachable from node iSimilar to Roulette Wheel Selection in a Genetic Algorithm

paths validallfor Pheromone of Sum

to node from Pheromone jipk

ij

Ant Colony Optimisation and Particle Swarm Optimisation Slide 17

Pheromone UpdateThe pheromone value of an arc (i,j) is updated when traversed by ant k as follows:

The probability of an arc being taken by subsequent ants is proportional to how “good” it was deemed by ants that have already traversed it

far so Ant ofLength Tour The

1

kkij

kijijij

Ant Colony Optimisation and Particle Swarm Optimisation Slide 18

Pheromone EvaporationThe pheromones “evaporate” by applying the following equation to all the arcs:

Here p (0,1) is a parameterAn iteration is a complete cycle involving the ant’s movement, pheromone depositing and pheromone evaporation

ijij p )1(

ACO Flowchart

Is thetrail

complete?

Have the stopping

criteria been met?

START

Locate ants randomlyat different nodes

Move to next node

Determine probabilistically as to which node to visit next

Record the length of tour

Determine the shortest tour until now and update pheromone

NO

YES

STOP

YESNO

Slide 19Ant Colony Optimisation and Particle Swarm Optimisation

Ant Colony Optimisation and Particle Swarm Optimisation Slide 20

ACO AdvantagesCan be used in dynamic applications

Adapts to changes such as new distances, etc…Similar to GAs, good choice for constrained discrete problems

Not a gradient based algorithmPerforms better against other global optimisation techniques Neural Networks, Genetic Algorithms, Simulated Annealing, under certain conditionsConvergence is “guaranteed”Compared to GAs (Genetic Algorithms):

Retains memory of the colony instead of previous generation onlyLess affected by poor initial solutions (due to combination of random path selection and colony memory)

Ant Colony Optimisation and Particle Swarm Optimisation Slide 21

ACO DisadvantagesTheoretical analysis is difficult

Due to sequences of random decisions (not independent)Probability distribution changes by iterationResearch is experimental rather than theoretical

Time to convergence uncertainSlower convergence sometimes than other Heuristics

Performed poorly for large scale problems

E.g. more than 75 cities when solving the TSPNo centralised processor to guide the ants towards good solutions

Ant Colony Optimisation and Particle Swarm Optimisation Slide 22

ACO ApplicationsGraph based and combinatorial optimisation type problems, e.g.

TSP (Travelling Salesman Problem)Vehicle RoutingGraph ColouringTimetable SchedulingManufacturing SchedulingNetwork RoutingEtc...

Ant Colony Optimisation and Particle Swarm Optimisation Slide 23

Particle Swarm Optimisation (PSO)It was developed in 1995 by James Kennedy (social psychologist) and Russell Eberhart (electrical engineer) Population based stochastic optimisation techniqueIt uses a number of agents (particles) that constitute a swarm moving around in the search space looking for the best solutionEach particle is treated as a point in an n-dimensional space which adjusts its “flying” according to its own flying experience as well as the flying experience of other particles

Ant Colony Optimisation and Particle Swarm Optimisation Slide 24

PSO Concepts – Part 1Each particle keeps track of its coordinates in the solution space and associated fitness, along with the best solution and fitness it has achieved so far

This value is called the personal best, pbestThe PSO algorithm also tracks the best value obtained so far by any particle in swarm

This value is called the global best, gbestThe basic concept of PSO lies in accelerating each particle toward its pbest and the gbest locations, with a random weighted acceleration at each time step

Ant Colony Optimisation and Particle Swarm Optimisation Slide 25

PSO Concepts – Part 2

Each particle tries to modify its position using the following information:

The current position The current velocityThe distance between the current position and pbest

The distance between the current position and gbest

sk : current position in the search spacesk+1: modified positionvk: current velocityvk+1: modified velocityvpbest: velocity based on pbest

vgbest: velocity based on gbest

sk

vk

vpbest

vgbest

sk+1

vk+1

sk

vk

vpbest

vgbest

sk+1

vk+1

PSO Concepts – Part 3

x

y

Slide 26Ant Colony Optimisation and Particle Swarm Optimisation

Ant Colony Optimisation and Particle Swarm Optimisation Slide 27

PSO Update Formulae

11

,1

)1,0(~,

ki

ki

ki

kibestg

kiibestp

ki

ki

gp

vss

sgrsprvv

URrr

(inertia), and are parameters...

Particle i best position

Particle i position at iteration k

Position of global best

Particle i velocity at iteration k+1

START

Initialise particles with random position and velocity vectors

For each particle’s position (p) evaluate fitness

If fitness(p) better than fitness(pbest) then pbest= p

Loo

p u

nti

l all

p

arti

cles

exh

aust

ed

Update gbest

Update particles velocity and position

Loo

p u

nti

l max

iter

atio

ns

STOP: giving gbest, optimal solution

PSO Flow Chart

Slide 28Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 1

x

y

fitnessmin

max

search space

Slide 29Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 2

x

y

search space

fitnessmin

max

Slide 30Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 3

x

y

fitnessmin

max

search space

Slide 31Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 4

x

y

fitnessmin

max

search space

Slide 32Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 5

x

y

fitnessmin

max

search space

Slide 33Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 6

x

y

fitnessmin

max

search space

Slide 34Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 7

x

y

fitnessmin

max

search space

Slide 35Ant Colony Optimisation and Particle Swarm Optimisation

PSO Simulation 8

x

y

fitnessmin

max

search space

Slide 36Ant Colony Optimisation and Particle Swarm Optimisation

Ant Colony Optimisation and Particle Swarm Optimisation Slide 37

PSO Comparison With ECNo selection operation in PSO

All particles in PSO are kept as members of the population through the course of the runPSO is the only algorithm (population based Heuristic search) that does not implement the survival of the fittest operator

No crossover operator in PSOThe PSO update formulae resembles mutation in EP (Evolutionary Programming)In EP, the balance between the global and local search can be adjusted through a number of parameters, whilst in PSO the balance is achieved through the inertial weight factor ()

Next LectureWe will be talking about:

Bin packingData clustering

The laboratory will involve looking at an number of simulations of ACO and PSO

Not assessed but useful revision!

Slide 38Ant Colony Optimisation and Particle Swarm Optimisation