Simulated Annealing

●Presented By:

Ranak Ghosh Septembar,2010

For more contact-08013835148

CONTENT

●Motivation●The algorithm●Its applications●Examples●Conclusion

Introduction

●Various algorithms proposed for placement in circuits.

●Constructive placement vs Iterative improvement.

●Simulated Annealing – an iterative improvement algorithm.

Motivation

●Annealing in metals●Heat the solid state metal to a high

temperature●Cool it down very slowly according to a

specific schedule.●If the heating temperature is sufficiently high

to ensure random state and the cooling process is slow enough to ensure thermal equilibrium, then the atoms will place themselves in a pattern that corresponds to the global energy minimum of a perfect crystal.

Contd….

Motivation●The connection between this algorithm and

mathematical minimization was first noted by Pincus.

●He proposed that it forms the basis of an optimization technique for combinatorial (and other) problems.

●SA's major advantage over other methods is an ability to avoid becoming trapped at local minima.

●The algorithm employs a random search which not only accepts changes that decrease objective function f, but also some changes that increase it.

●The latter is accepted with a probability P=exp(-df/T), where df is the  increase in f and T is a control parameter,

Simulated Annealing-The Algorithm

●Analogy

Cntnd.

Analogy●Slowly cool down a heated solid, so that all particles

arrange in the ground energy state●At each temperature wait until the solid reaches its

thermal equilibrium●Probability of being in a state with energy E :

Pr { E = E } = 1/Z(T) . exp (-E / kB.T)

E EnergyT Temperature

kB Boltzmann constant

Z(T) Normalization factor (temperature dependant)

PROCEDURE OF SIMULATED ANNEALING

Cooling schedule

Convergence of simulated annealing

Ball on terrain example – Simulated Annealing vs Greedy Algorithms

●The ball is initially placed at a random position on the terrain. From the current position, the ball should be fired such that it can only move one step left or right.What algorithm should we follow for the ball to finally settle at the lowest point on the terrain?

Ball on terrain example – SA vs Greedy Algorithms

Example: Travel Salesperson Problem (TSP)

How to transverse n cities once and onlyonce with a minimal total distance?

Traveling Salesperson Problem (TSP)

●TSP is famous discrete optimization problem●Many successful uses of SAN with TSP●Basic problem is to find best way for salesperson to hit

every city in territory once and only onceoSetting arises in many problems of optimization on

networks (communications, transportation, etc.)●If tour involves n cities, there are (n–1) ! /2 possible

solutionsoExtremely rapid growth in solution space as n

increasesoProblem is “NP hard”

●Perturbations in SAN steps based on three operations on network: inversion, translation, and switching

9/8/2010

TSP●Move sets for TSP

1-2-3-4-5-6-7-8-9-10-11-12

1

2

3

4 5

67

8 9

10

11

12

1

2

3

4 5

67

8 9

10

11

12

1-2-3-4-5-9-8-7-6-10-11-12

1

2

3

4 5

67

89

10

11

12

1-2-3-4-8-7-5-9-6-10-11-12

1

2

3

4 5

67

89

10

11

12

1-2-11-4-8-7-5-9-6-10-3-12

Inversion

Switching

Translation

Inversion reverses order 6-7-8-9; translation removes section 6-7-8-9 and places 8-7 in place of 5-9; switching interchanges order of 3 and 11.

Small-Scale TSP

●10-city tour (very small by industrial standards)oKnow by enumeration that minimum cost of tour =

440●Randomly chose inversion, translation, or switching

at each iterationoTuning required to choose “good” probabilities of

selecting these operators●8 of 10 SA runs find minimum cost tour

oSample mean cost of initial tour is 700; sample mean of final tour is 444

●Essential to success is adequate use of inversion operator; 0 of 10 SA runs find optimal tour if probability of inversion is 0.50

●SA successfully used in much larger TSPs oE.g., seminal 1983 (!) Kirkpatrick et al. paper in Science considers TSP with 400 cities

Simulated Annealing for TSP-ANALOGY

●Initial configuration: permutation =>1, 2, 3, 4, …, Ntemperature T = 2cooling rate α = 0.99energy = d(1,2)+d(2,3)+…+d(N,1)●Generate a new configuration from the current one

at random●Evaluate ΔE = current energy – previous energy●If ΔE < 0 accept the current configuration (downhill)

Else accept configuration with probability :P=exp (-∆E / kB.T)●T= αT●Stopping criteria if energy < threshold or number of

iterations is reached.

Case Study: 20-city problem

Applications

●Circuit partitioning and placement.●Strategy scheduling for capital

products with complex product structure.

●Umpire scheduling in US Open Tennis tournament!

●Event-based learning situations.

Jigsaw puzzles – Intuitive usage of Simulated Annealing

●Given a jigsaw puzzle such that one has to obtain the final shape using all pieces together.

●Starting with a random configuration, the human brain unconditionally chooses certain moves that tend to the solution.

●However, certain moves that may or may not lead to the solution are accepted or rejected with a certain small probability.

●The final shape is obtained as a result of a large number of iterations.

Performance

●SA is a general solution method that is easily applicable to a large number of problems

●"Tuning" of the parameters (initial T, decrement of T, stop criterion) is relatively easy

●Generally the quality of the results of SA is good, although it can take a lot of time

●Results are generally not reproducible: another run can give a different result

●SA can leave an optimal solution and not find it again(so try to remember the best solution found so far)

●Proven to find the optimum under certain conditions; one of these conditions is that you must run forever

Thank You!