Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer...

22
Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming & Combinatorial Optimization o Traveling salesman problem o Facility location o Network design o Traveling salesman problem o Facility location o Network design Three Lectures Three Lectures Games/Challenges Applications, Models, and Solution Methods Games/Challenges Applications, Models, and Solution Methods Traveling Salesman Problem Traveling Traveling Salesman Salesman Problem Problem 1 Thomas L. Magnanti Thomas L. Magnanti

Transcript of Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer...

Page 1: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Module on Large-ScaleInteger Programming & Combinatorial

Optimization

Module on Large-ScaleInteger Programming & Combinatorial

Optimization

o Traveling salesman problemo Facility locationo Network design

o Traveling salesman problemo Facility locationo Network design

Three LecturesThree Lectures

Games/ChallengesApplications, Models, and

Solution Methods

Games/ChallengesApplications, Models, and

Solution Methods

Traveling Salesman Problem

Traveling Traveling Salesman Salesman ProblemProblem

1Thomas L. MagnantiThomas L. Magnanti

Page 2: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

AgendaAgenda

o Originso Electronic Component Placemento TSP Modelo Turbine Vane Placemento Other Applications of TSPo Solution Methods

u Heuristic Methodsu Lagrangian relaxation (bounding methods)

o Originso Electronic Component Placemento TSP Modelo Turbine Vane Placemento Other Applications of TSPo Solution Methods

u Heuristic Methodsu Lagrangian relaxation (bounding methods)

2

Icosian game Icosian Icosian game game

3

William Rowan HamiltonWilliam Rowan Hamilton

Page 3: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Icosian game Icosian Icosian game game

Hamiltonian Path and the TSPHamiltonian Path and the TSPHamiltonian Path and the TSP4

1

20

19

2

3 45

6 7

89

10

131415

1617

18

11 12

Interest in TravelingSalesman Problem (TSP)

Interest in TravelingSalesman Problem (TSP)

o Arises in Many Applicationso Alluring (Captures Imagination)o Notoriously Difficult to Solveo Has Attracted Best Minds in

Math/CS/OR for 40 Years

o Arises in Many Applicationso Alluring (Captures Imagination)o Notoriously Difficult to Solveo Has Attracted Best Minds in

Math/CS/OR for 40 Years

5

Page 4: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

The Traveling Salesman Problem and Electronics Assembly

The Traveling The Traveling Salesman Salesman Problem and Problem and Electronics Electronics AssemblyAssembly

6

SMT Placement MachineSMT Placement Machine

Page 5: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

FeedersFeedersFeeders

Placement LocationsPlacement LocationsPlacement Locations

7

Placement ProblemPlacement Problem

8

Placement SequencePlacement Sequence

Page 6: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Home Position Home Home PositionPosition

9

Traveling Salesman InterpretationTraveling Salesman Interpretation

xjk =1 if placement k follows placement j

0 otherwise

xxjkjk = =11 if placement k follows if placement k follows placement jplacement j

0 otherwise0 otherwise

cjk cost of placing module k after placing module j

ccjkjk cost of placing module cost of placing module k after placing module jk after placing module j

10

Model IngredientsModel Ingredients

Page 7: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Minimize Σj Σk cjkxjkMinimize Minimize ΣΣj j ΣΣkk c cjkjkxxjkjk

Σkxjk = 1 for each jΣj xjk = 1 for each k

xjk > 0 for all j & k

ΣΣkkxxjkjk == 1 for each j1 for each jΣΣjj x xjkjk == 1 for each k1 for each k

xxjkjk >> 0 for all j & k0 for all j & k

subject tosubject tosubject to

Proper TSP Model?Proper TSP Model?Proper TSP Model?11

Assignment ProblemAssignment Problem

12

Subtour SolutionSubtour Solution

Page 8: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Assignment Model

+Subtour Breaking Constraints

Assignment ModelAssignment Model

++Subtour Breaking ConstraintsSubtour Breaking Constraints

Σj∈S Σk∈S xjk < |S| - 1 for all subsets S of nodes {2,3,…,n}

Σ Σjj∈∈SS Σ Σkk∈∈SS xxjkjk << |S| - 1 for all subsets |S| - 1 for all subsets S of nodes {2,3,…,n} S of nodes {2,3,…,n}

Proper TSP Model?Proper TSP Model?Proper TSP Model?1313

TSP ModelTSP Model

Implications for IC InsertionsImplications for IC Insertions

o Manual Designs ⇒ Long Timeu 10 hours for 70 to 100

components

o Better Solutionsu 10-25% improvements by

optimization

o Manual Designs ⇒ Long Timeu 10 hours for 70 to 100

components

o Better Solutionsu 10-25% improvements by

optimization

Other Applications SimilarOther Applications SimilarOther Applications Similar23

Page 9: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Feeder PlacementFeeder Placement

o Modeling?o Solution Methods?

u Heuristicu Optimization

o Modeling?o Solution Methods?

u Heuristicu Optimization

24

Other Applications of TSP?

Other Other Applications Applications of TSP?of TSP?

??

??

14

Page 10: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Machine SchedulingMachine SchedulingMachine Scheduling

Machine “Visits” JobsTravel Time = Set up TimeMachine “Visits” JobsMachine “Visits” JobsTravel Time = Set up TimeTravel Time = Set up Time

1515

Other Applications of TSPOther Applications of TSP

DepotDepotDepot

16

Other Applications of TSPOther Applications of TSP

Page 11: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

1717

Vane Turbine PlacementVane Turbine Placement

Vane Turbine PlacementVane Turbine PlacementVane Turbine Placement

o Placement of Vanes in NozzleAssembly

o Ideally Area Between Two AdjacentVanes Should be Equal

o Two Parameters A(i) and B(i) for EachVane

o Area Between Vane i and i+1 Dependson A(i) + B(i+1)

o Min ∑[d-A(i) -B(i+1)]2

{d = target area}

o Placement of Vanes in NozzleAssembly

o Ideally Area Between Two AdjacentVanes Should be Equal

o Two Parameters A(i) and B(i) for EachVane

o Area Between Vane i and i+1 Dependson A(i) + B(i+1)

o Min ∑[d-A(i) -B(i+1)]2

{d = target area}18

Page 12: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Other ApplicationsOther Applications

o Analyzing the structure of crystalso Material handling in a warehouseo Clustering of data arrayso Cutting stock problemso Genome sequencingo Starlight interferometer satellite

positioningo DNA universal stringso Collecting coins from payphones

o Analyzing the structure of crystalso Material handling in a warehouseo Clustering of data arrayso Cutting stock problemso Genome sequencingo Starlight interferometer satellite

positioningo DNA universal stringso Collecting coins from payphones

Solving the TSPSolving the TSP

Page 13: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

26 City Example26 City Example

2 5

3 0

3 5

4 0

4 5

5 0

7 08 09 01 0 01 1 01 2 0

Lo ng itude

L a t i t u d e

S E

P O

S F

L A

D E

T U

D A

H U N O

M E

A T

W A

P H

N Y

B O

P I

C L

D T

C NINS L

K S

O M

MN

C H

M L

2 5

3 0

3 5

4 0

4 5

5 0

7 08 09 01 0 01 1 01 2 0

Lo ng itude

L a t i t u d e

S E

P O

S F

L A

D E

T U

D A

H U N O

M E

A T

W A

P H

N Y

B O

P I

C L

D T

C NINS L

K S

O M

MN

C H

M L

2020

26 City Traveling Salesman Problem26 City Traveling Salesman Problem

LatitudeLatitude

Page 14: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Finding a Good SolutionFinding a Good Solution

q How?q How good is good?

uLP boundsuCombinatorial bounds

qq How?How?qq How good is good?How good is good?

uuLP boundsLP boundsuuCombinatorial boundsCombinatorial bounds

21

Solution MethodsSolution Methods

o Heuristicsu Growing solutions: nearest neighbor,

farthest neighbor, nearest insertionu Improvement procedures: 2-opt, 3-opt

o Optimization Methodsu Bounding: LP relaxation, Lagrangian

dualu Polyhedral methods (cutting planes)

o Heuristicsu Growing solutions: nearest neighbor,

farthest neighbor, nearest insertionu Improvement procedures: 2-opt, 3-opt

o Optimization Methodsu Bounding: LP relaxation, Lagrangian

dualu Polyhedral methods (cutting planes)

Page 15: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

HeuristicsHeuristics

o Build Touru Nearest Neighboru Nearest/Farthest Insertion

o Improve Touru Swapping Edges

o Build Touru Nearest Neighboru Nearest/Farthest Insertion

o Improve Touru Swapping Edges

22

Insertion HeuristicsInsertion Heuristics

1

5 2

3

4 6

1

5 2

3

4 6

x

x

NearestInsertionNearestInsertion

FarthestInsertionFarthestInsertion

Page 16: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Tour ImprovementsTour Improvements

1

5 2

3

4 6

1

5 2

3

4 6

2-opt

Eliminate 2 arcs and reconnect

Choose bestalternative at each step

2-opt

Eliminate 2 arcs and reconnect

Choose bestalternative at each step

Exploiting Embedded StructureExploiting Embedded Structure

{ }

1 1

1

1

2 2

minimize

subject to 1 for all 1, 2, ,

1 for all 1, 2, ,

2

1 for all 2,3, ,

n n

ij iji j

nijj

n

iji

n n

iji j

iji S j S

ij

c x

x i n

x j n

x n

x S S n

x

= =

=

=

= =

∈ ∈

= =

= =

= −

≤ − ⊆

∑ ∑∑∑

∑ ∑∑ ∑

K

K

K

0 and integerRedundant ConstraintRedundant Constraint

Minimum spanning tree on nodes 2 to nMinimum spanning tree on nodes 2 to n

Page 17: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Underlying StructureUnderlying Structure

1

5 2

3

4 6

Decomposed TourDecomposed Tour

Arc in and outof node 1

Arc in and outof node 1

Path (hence tree) on nodes 2 to nPath (hence tree) on nodes 2 to n

Lagrangian RelaxationLagrangian Relaxation

1 1 2 1

2 1

11

11

2 2

( , ) minimize 1

1

subject to 1

1

2

n n n nij ij i iji j i j

n nj ijj i

njj

nii

n niji j

ij

L u v c x u x

v x

x

x

x n

x

= = = =

= =

=

=

= =

= + − + −

=

=

= −

∑ ∑ ∑ ∑∑ ∑

∑∑

∑ ∑{ }1 for all 2,3, ,

0 and integeri S j S

ij

S S n

x∈ ∈

≤ − ⊆

∑ ∑ K

Page 18: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Improving Lagrangian Lower BoundImproving Lagrangian Lower Bound

1

5 2

3

4 6

Solution x* to Lagrangian RelaxationSolution x* to Lagrangian Relaxation

Too many arcs outIncrease u5c5j+ u5 + vj more expensive

Too few arcs outDecrease u6c6j+ u6 + vj less expensive

Solution Approach (Dual Ascent)Solution Approach (Dual Ascent)

o Solve L(u,v)u Use costs cij+ vi + uj

u Select least cost arc out of and into node 1u Find minimal spanning tree on nodes 2 to n

(easy)o Let x* be optimal solution to L(u,v)o Using x* alter u and v to increase lower

bound L(u,v)o Iterate to solve Lagrangian dual

d = max u,v L(u,v)

o Solve L(u,v)u Use costs cij+ vi + uj

u Select least cost arc out of and into node 1u Find minimal spanning tree on nodes 2 to n

(easy)o Let x* be optimal solution to L(u,v)o Using x* alter u and v to increase lower

bound L(u,v)o Iterate to solve Lagrangian dual

d = max u,v L(u,v)

Page 19: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Dual Ascent in GeneralDual Ascent in General

* = minimize subject to

v cxAx bx X

=∈

ComplicatingConstraintsComplicatingConstraints

[ ]( ) minimize

subject to

L u cx u Ax b

x X

= + −

{ }: , 0X x Dx d x= = ≥e.g.,

Lagrangian Dual

Dual Ascent in GeneralDual Ascent in General

If ( ) ( ) ( )k k k kL u cx u u Ax u b = + − ( ) ( )k kL u Ax u b∇ ≈ −

1 ( )k k kku u Ax u bθ+ = − −

2

( ) *

( )

kk

k k

cx u v

Ax u b

λθ

− =−

Page 20: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Dual Ascent ConvergenceDual Ascent Convergence

Theorem

If

and are bounded,

then c(xk) converges to v*.

1 20 2kε λ ε< ≤ ≤ −

( )kAx u b−

In practice, continue with fixed λk except half λk after some number (50?) of iterations if c(xk) doesn’t decrease

Solving Minimum Spanning TreeSolving Minimum Spanning Tree

o Order arcs from smallest to highestcosts

o Choose arcs in orderu If arc does not form an undirected

circuit with arcs already chosen, thenchoose arc;

u Otherwise eliminate arc fromconsideration

o Order arcs from smallest to highestcosts

o Choose arcs in orderu If arc does not form an undirected

circuit with arcs already chosen, thenchoose arc;

u Otherwise eliminate arc fromconsideration

Greedy (Kruskal Algorithm)Greedy (Kruskal Algorithm)

Page 21: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Amazing Facts!Amazing Facts!

o Greedy algorithm (and several variants)solves the minimum spanning problem

o The linear programming relaxation of theformulation we have given for theminimum spanning problem always has aninteger solution (the underlyingpolyhedron has integer extreme points)

o Greedy algorithm (and several variants)solves the minimum spanning problem

o The linear programming relaxation of theformulation we have given for theminimum spanning problem always has aninteger solution (the underlyingpolyhedron has integer extreme points)

Solving Network Flow ProblemsSolving Network Flow Problems

o Giden (Graphical Environment forNetwork Optimization)u http://giden.nwu.edu/download/index.ht

mlu Demonstration of

• Minimum spanning trees• Maximum flows• Minimum cost flows

(Based on Ahuja, Magnanti, Orlin’s book Network Flows)(Based on Ahuja, Magnanti, Orlin’s book Network Flows)

Page 22: Module on Large-Scale Integer Programming & Combinatorial ... · Module on Large-Scale Integer Programming & Combinatorial Optimization Module on Large-Scale Integer Programming &

Traveling salesman problem through 15,112 cities in Germany

22.6 years ofcomputation on network of 110 Processors

See http://www.math.princeton.edu/tsp/

Traveling salesman problem through 15,112 cities in Germany

22.6 years ofcomputation on network of 110 Processors

See http://www.math.princeton.edu/tsp/

Today’s LessonsToday’s Lessons

o Traveling salesman problem arises innumerous applications

o Problem is a large-scale integer programo Many heuristic methods: often find good

solutionso Lagrangian dual (bounding) exploits

special problem structure (embeddedminimal spanning tree)

o MST is easy to solve

o Traveling salesman problem arises innumerous applications

o Problem is a large-scale integer programo Many heuristic methods: often find good

solutionso Lagrangian dual (bounding) exploits

special problem structure (embeddedminimal spanning tree)

o MST is easy to solve

We did NOT examine polyhedral (cutting plane) methodsWe did NOT examine polyhedral (cutting plane) methods