Traveling Salesperson Problem

Steven JankeColorado College

-

3 4 6

2 -

5 2

7 4 - 3

5 6 7 -

A B C D

A

B

C

D

Optimization Problem: Find the least cost tour starting at A, traveling

through the other three cities exactly once and returning to A.

A

B

C

D

Decision Problem: Is there a TSP tour with cost less than k?

Traveling Salesperson Problem (TSP)

T. P. Kirkman

In 1856, he published sufficientconditions for a polyhedralgraph to have a Hamiltoniancircuit.

1806 - 1895Sir William Hamilton1805 - 1865

• Discovered quaternions in 1843• Invented Icosian Game in 1857• Hamiltonian Circuits

Icosian Game 1857

Proctor & Gamble Contest 1962

Twentieth Century History

1931-32 Merrill Flood, A.W.Tucker, Hassler Whitney (Princeton) pose the Traveling Salesperson Problem.

1947 George B. Dantzig designs simplex method for linear programming. 1954 Connection with the Assignment Problem established.

1962 Dynamic programming formalized.

1972 Karp proved TSP is NP-complete.

1985 – 2008 Techniques leading to solving a 85,000-city problem

TSP Applications:

School Bus routing.

Robotic welding in the car industry.

Printed circuit board drilling and laser cutting of integrated circuits.

Genome sequencing. Finding most likely ordering of markers. Job processing: Chemical plants – cost of setup to produce chemicals.

Admissions Office Campus Tour Colorado College

Mathais

Slocum

Packard

El Pomar

Herb ‘n Farm

Cutler Hall

Versions of the TSP:

Complete or Incomplete graph.

Symmetric vs. Asymmetric cost matrix.

Euclidean (Triangle inequality holds.)

Upper Triangular cost matrix.

Circulant cost matrix.

Bipartite graph.

-

3 4 6

2 -

5 2

7 4 - 3

5 6 7 -

A B C D

A

B

C

D

Traveling Salesperson Problem: Find the least cost tour starting at A, traveling through the other three cities exactly once and returning to A.

A

B

C

D

Generating Permutations:

A < BCD B < ACD C < ABD D < ABC BDC ADC ADB ACB CBD CAD BAD BAC CDB CDA BDA BCA DBC DAC DAB CAB DCB DCA DBA CBA

ABCD CABD DCBA DBACBACD ACBD DCAB DBCABCAD ACDB DACB BDCABCDA CADB ADCB BDAC CBDA CDAB ADBC BADCCBAD CDBA DABC ABDC

Single exchange:

Recursion:

Naïve Algorithm

A BCD A = 18A BDC A = 19A CBD A = 15A CDB A = 15A DBC A = 24A DCB A = 19

2. Find a tour with minimum cost:

A CBD A = 15 (one optimal tour)

1. List all tours and their costs:

Timing results for Naïve Algorithm:

Cities Seconds Tours Sec/Tour

11 0.50 3,628,800 1.38 x 10-7

12 5.61 39,916,800 1.41 x 10-7

13 69.12 479,001,600 1.44 x 10-7

Estimate for 17 cities: 34.9 days (2.09 x 1013 tours)

Complexity:

If there are n-1 cities (other than the home city A), then there are (n-1)! possible tours.

The naïve algorithm takes at least (n-1)! steps.

A random algorithm could possibly find an optimal tour in (n-1) steps.

Is there a deterministic algorithm that can find an optimal tour in a polynomial number of steps?

Theorem: (Karp, 1972) TSP is NP-Complete. (That is, every otherhard problem is reducible to TSP.)

Open problem: Is there a polynomial time algorithm for TSP?If not, can you prove it? (In technical terms, does P = NP?)

Solving the open problem earns you $1,000,000 from the Clay Mathematics Institute.

Approaches to the TSP:

Constant TSP: Analyze the cost matrix.

Linear Programming

Euclidean TSP: Probabilistic analysis.

Approximation algorithms

Search techniques Branch and Bound Dynamic Programming Genetic Algorithm Ant Colony

-

1 3 5

10 -

6 8

8 2 - 6

9 3 5 -

A B C D

A

B

C

D

A BCD A = 22 A CBD A = 22 A DBC A = 22A BDC A = 22 A CDB A = 22 A DCB A = 22

A

B

C

D

Constant TSP

Theorem: (Constant TSP)The only cost matrices that give the same cost for all TSP tours are those with entries that satisfy:

cij = ai + bj

Sketch of Proof:

The set C of matrices with constant cost form a linear subspace.

The dimension of C is 2n-1.

The matrices with either a single row of ones or a single column of ones belong to C and any set of 2n-1 of them are linearly independent.

-

1 3 5

10 -

6 8

8 2 - 6

9 3 5 -

Constant TSP:

6

0

2

4

1

4

2

3

a b c

Minimize: x1+ x2

Subject to: 2x1 + x2 >= 5 x1 + 2x2 >= 6

x1 >= 0 x2 >= 0

x1+ x2 = 1 x1+ x2 = 3.67

x1+ x2 = 5.8

Feasible region

x2

x1

Linear Program

The TSP Polytope:

Let x = (xij ) be a vector with an entry 1 indicating that edge ij is included. Entry 0 indicates the edge is excluded. Hence the vector has length equal to the number of edges.

Let xt be the vector corresponding to the tour t.

Polytope P = convex hull of {xt | t is a tour}

Interestingly, dim P = |E| - |V| = n(n-3)/2

(Where E is the set of edges and V is the set of vertices.)

Linear Programming formulation of the TSP:

Relaxation of the Linear Program:

Additional contraints:

Euclidean TSP:

The triangle inequality holds for the distance matrix.

For any three cities A, B, C,

AB + BC >= AC

(Note that the distance measure could be something other than the Euclidean distance.)

For the Euclidean TSP, the tour does not cross itself.

The Euclidean TSP is still NP-Complete, but there areapproximation algorithms.

Edge Crossings in Geometric TSP

A B

C D

e

Ae + eB >= AB and Ce + eD >= CD

AD + CB >= AB + CD

Euclidean TSP on the unit cube

2 4

5

7

6

7

11

9

10 9

AB

CD

E

Minimum Spanning Tree: Pick next smallest edge connected to partial tree but not forming a cycle.

2 4

5

7

6

7

11

9

10 9

AB

CD

E

Minimum Spanning Tree Algorithm:

To form a TSP tour, trace the minimum spanning tree twice using short cuts if possible. This gives A-E-B-C-D-A.

The cost of this tour is 31. The spanning tree has cost 19.

Let OPT = cost of optimal TSP tour. MST = cost of minimum spanning tree. RMT = cost of route derived from minimum spanning tree.

The optimal TSP tour minus the last edge is a minimum spanning tree, so MST < OPT.

Then RMT <= 2*MST <= 2*OPT

With a little more care the shortcuts can be optimized to give: RMT <= 1.5*OPT

Recently an algorithm scheme was discovered that givesRMT <= (1+e)*OPT for arbitrary e > 0

Euclidean Approximation Guarantee:

Approximations for the General TSP:

Can an approximation algorithm find a tour that is, say, within 110% of the optimal route?

For an approximation algorithm A and TSP instance I, let A(I) be the cost of the tour it generates and let OPT(I) be the optimal cost.

Bottom line: No polynomial time algorithm A can guarantee that for all instances I of TSP, A(I) < r OPT(I) for any constant r, unless P=NP.

Nevertheless, there are “rules” (called heuristics) that help find good, if not optimal, TSP tours:

Nearest NeighborDissectionNearest InsertionTour Improvement

B

A

C D

C D B D B C

C D B C B

Total nodes: 1+ (n-1) + (n-1)(n-2) + … + (n-1)!

(For 4 cities, there are 10 nodes.)

Little workrequired.

D

Search Tree

B

A

C D

C D B B C

C C

Branch and Bound techniques possibly reduce the number ofnodes visited.

D

Branch and Bound Technique

Estimate must be a lower bound on the cost and is called the bounding function. Technique depends on how accurate estimate is.

Add A to the possible list and assign arbitrary cost.

While the possible list is not empty do: - Select node with smallest estimated cost and remove from list. - Generate children of selected node and add to list. - If children complete a tour, update optimal tour, otherwise estimate cost of completed tour using this child. - If estimated cost is greater than current optimal, remove node from list.

A Possible Bounding Function B(i) :

C1 = cost of partial tour.

C2 = sum of the minimum edges into each remaining city.

Each node i in the search tree represents a partial tour. Thosecities not in the partial tour form a set of “remaining” cities.

B(i) = C1 + C2

B(i) <= Cost of any tour containing the partial route.

Shortest Campus Tour: 1281 seconds

Elapsed time: 1.05 secondsTotal nodes visited: 710,050

Cutler

Armstrong

Mathais

Tutt Library

Herb ‘n Farm

El PomarSlocum

15 0.05 26,599 8.7x1010

20 0.33 188,360 1.2x1017

25 11.86 5,294,282 6.2x1023

Visited TotalCities Time (Sec) Nodes Nodes

Timing Results for Branch and Bound Algorithm (Average of 3 Instances.)

17 City Admission Tour: 1.05 seconds (710,050 nodes visited out of 2x1013)

Ant Colony Optimization

Ants find shortest path to food.

They deposit pheromone on trail.

Trails with most pheromone get most ants.

Basic searching behavior has random character.

Ant Colony Optimization Algorithm for TSP:

Place some ants at each city and iterate the following:

1. While at city i, an ant calculates a = c / d where c is the amount of pheromone and d is the distance

between i and j.

2. Each ant selects a city not already visited with probabilities proportional to the a .

3. After all ants have built a tour, evaporate some fraction of the existing pheromone.

4. Each ant deposits an amount of pheromone inversely proportional to the length of their tour.

ij ijij

ijijr s

ij

Ant Campus Tour: 1291 seconds

Elapsed time: 1.6 secondsAnt Tours: 34,000

Cutler

Armstrong

Mathais

Tutt Library

Herb ‘n Farm

El PomarSlocum

Line drawings with TSP routes: (Robert Bosch and Craig Kaplan)

References:

The Traveling Salesman Problem (1985) - Lawler, Lenstra, Rinnoooy Kan, Shmoys The Traveling Salesman Problem and its Variations (2002) - Gutin, Punnen The Traveling Salesman Problem: A Computational Study (2006) - Applegate, Bixby, Chvatal, Cook

Dynamic Programming Algorithm:

A – BCD EFG – AA – BCD EGF – AA – BCD FGE – AA – BCD FEG – AA – BCD GFE – AA – BCD GEF – A

A – CBD FEG – AFEG

g(i, S) = shortest path from i through vertices in S ending at A.

g(i, S) = min { c + g(j, S - {j}) }

Optimal tour = g(A, {all vertices}-A) Steps: n2 x 2n

i,jj in S

Optimal