Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken...

47
Section #2: Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes)

Transcript of Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken...

Page 1: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Section #2:

Linear and Integer Programming

Prof. Dr. Sven Seuken

8.3.2012

(with most slides borrowed from David Parkes)

Page 2: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Housekeeping

• Game Theory homework submitted?

• HW-00 and HW-01 returned

• Feedback on Comprehension Questions later

today or tomorrow

• BitTorrent assignment � online this afternoon

• Next chapter � online this afternoon

• Questions? Concerns?

Prof. Dr. Sven Seuken - University of Zurich

Page 3: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Outline

1. O()-Notation

2. NP vs. P

3. Linear Programming

4. Integer Programming

Prof. Dr. Sven Seuken - University of Zurich

Page 4: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Big O()-Notation

• Analysis of algorithms

• Asymptotic running time

• Input Size…?

Prof. Dr. Sven Seuken - University of Zurich

Page 5: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Polynomial vs. Exponential

Prof. Dr. Sven Seuken - University of Zurich

Page 6: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

The Complexity Class “P”

• Decision Problems

• A problem is in P, if there exists a worst-case

polynomial time algorithm for solving the

problem.

• Efficient

Prof. Dr. Sven Seuken - University of Zurich

Page 7: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

The Complexity Class “NP”

• Decision Problems! (YES vs NO)

• A problem is in NP, if, given an answer (or a

proof), there exists a worst-case polynomial

time algorithm that can verify that the answer

is YES.

Prof. Dr. Sven Seuken - University of Zurich

Page 8: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

NP-hard vs. NP-complete

Prof. Dr. Sven Seuken - University of Zurich

Page 9: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

P vs. NP

• Important Conjecture: P ≠NP

• Why do we believe this?

• Consequence:

– Fastest algorithms for problems that are NP-hard probably require exponential time in the worst case.

� Inefficient algorithms (for medium-sized problems)

Prof. Dr. Sven Seuken - University of Zurich

Page 10: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Mathematical Optimization

• What is “optimization”?

– problem of making decisions to maximize or minimize an objective in the presence of complicating constraints

• Examples: scheduling aircraft, designing a jet engine, sourcing goods, predicting who will win a football game.

Prof. Dr. Sven Seuken - University of Zurich

Page 11: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Example: Going Shopping

• I can buy pizza x_1, and chicken x_2

• Pizza costs $5 per unit, and gives me 3 carbohydrates and 2 proteins per unit

• Chicken costs $10 per unit, and gives me 5 proteins and 1 carbohydrate per unit

• Task:– Maximize the carbohydrates

– Do not spend more than $30

– Need at least 10 protein units

• Variables?

• Objective?

• Constraints?

Prof. Dr. Sven Seuken - University of Zurich

Page 12: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Example: Marketing campaign• Comedy– 7 million high-income women, 2

million high-income men. Cost $50,000

• Football – 2 million high-income women and 12

million high-income men. Cost $100,000

• Goal: reach at least 28 million high-income

women and 24 million high-income men at

MINIMAL cost

Prof. Dr. Sven Seuken - University of Zurich

Page 13: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

graphical version of problem

(solution is x1=3.6, x2=1.4, value 320)

Prof. Dr. Sven Seuken - University of Zurich

Page 14: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Example: Unbounded objective

How would you show this algebraically?Prof. Dr. Sven Seuken - University of Zurich

Page 15: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Example: Infeasible problem

• ..\..\Desktop\17.bmp

How would you show this algebraically?Prof. Dr. Sven Seuken - University of Zurich

Page 16: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Post Office ProblemDay Number of full-time

employees required

1=Monday 17

2=Tuesday 13

3=Wednesday 15

4=Thursday 19

5=Friday 14

6=Saturday 16

7=Sunday 11

Union rules state that each

full-time employee must work

5 consecutive days and then

receive 2 days off.

Formulate an LP to minimize

the number of full-time

employees who must be

hired.

Prof. Dr. Sven Seuken - University of Zurich

Page 17: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Note: will need solution to be integral!Prof. Dr. Sven Seuken - University of Zurich

Page 18: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Prof. Dr. Sven Seuken - University of Zurich

Page 19: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Polyhedron

• Definition. A polyhedron is a set that can be described in form P={x in Rn | Ax>=b}

• Fact: feasible set of any LP can be described in this way, and in particular {x in Rn | Ax=b, x>=0} is also a polyhedron.

• Theorem. Every polyhedron is a convex set.

• Why?

Prof. Dr. Sven Seuken - University of Zurich

Page 20: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Extreme points

P

vw

u

y

zx

Prof. Dr. Sven Seuken - University of Zurich

Page 21: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Algorithms for LPs

• Idea: Check every extreme point

– With n constraints: 2^n extreme points

– � takes exponential time

• Simplex Algorithm:

– Move from extreme point to extreme point

– Stop when local optimum (= global optimum)

– Very fast in practice (e.g., more than 10,000 constraints)

– Worst-case time complexity: exponential

Prof. Dr. Sven Seuken - University of Zurich

Page 22: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Complexity of LPs?

• Solving an LP is in P!

• Thus, there exists a worst-case polynomial

time algorithm for solving LPs:

� interior-point method

• In the worst case faster than simplex method

• In the average case, slower than simplex…

• In practice:

– CPLEX (IBM)

Prof. Dr. Sven Seuken - University of Zurich

Page 23: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Integer Programming

Prof. Dr. Sven Seuken - University of Zurich

Page 24: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Some uses of IP• Scheduling problems

– e.g., air taxi scheduling (assign each air taxi to a particular route)

• Strategic procurement– e.g., hospital system determining which suppliers to use for

sourcing of medical/surgical equipment

• Electricity generation planning– e.g., developing a schedule to determine when to start-up

plants, and levels to run at

• Clearing kidney exchanges– e.g., finding “donor chains”

• Computational biology– e.g. determining evolutionary trees (“phylogenetic trees”)

from population variation data

Prof. Dr. Sven Seuken - University of Zurich

Page 25: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Shouldn’t IPs be easier to solve

• “There are fewer feasible solutions”

– What is misleading about this argument?

Prof. Dr. Sven Seuken - University of Zurich

Page 26: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

max 5x1 + 8x2

s.t. x1 + x2 <= 6

5x1 + 9x2 <= 45

x1, x2 >= 0, integer

isoprofit

x2 6

4

2

00 4 6 8 x1

optimal continuous solution

x*=(9/4, 15/4), z*=41.25

optimal integer solution

x*=(0,5), z*=40

cont round off

nearest feas

integer

x1 2.25 2 2 0

x2 3.75 4 3 5

z 41.25 infeas 34 40

Prof. Dr. Sven Seuken - University of Zurich

Page 27: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Example: Weighted Knapsack

• E.g., Budget b available for investment in

projects. Project j in {1,…,n} has cost (“size”) aj

and value (“weight”) cj.

• Let xj = 1 if project j selected, xj = 0 otherwise

• max ∑j cj xj

s.t. ∑j aj xj <= b

xj in {0,1}

(budget constraint)

(maximize value)

(no fractional projects)

Prof. Dr. Sven Seuken - University of Zurich

Page 28: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Example: Traveling Salesman Problem (TSP)

• Salesman must visit each of N={1,…,n} cities once

and then return to starting point. Cost cij¸0 to

travel from i to j. Goal: find tour that minimizes

total cost.

1

2

3 5

4

6

7

A feasible tour in a

seven-city TSP

Examples: FedEx pick-up,

machine placing modules on a

circuit board, visiting

colleges,…Prof. Dr. Sven Seuken - University of Zurich

Page 29: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• How many solutions?

• Starting at city 1, there are n-1 choices

• For the next city, n-2 choices,…

• (n-1)! feasible tours

n log n n0.5 n2 2n n!

10 3.32 3.16 102 1.02x103 3.6x106

100 6.64 10.00 104 1.27x1030 9.33x10157

1000 9.97 31.62 106 1.07x10301 4.02x102567

Prof. Dr. Sven Seuken - University of Zurich

Page 30: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Branch and Bound: Solving IPs

• Ignore integrality constraints, solve the LP relaxation and hope the solution is integer!

• Example:max 5x1 + 8x2

s.t. x1 + x2 <= 65x1 + 9x2 <= 45

x1, x2 >= 0, integer

• Let S0 denote feasible solution space of IP

• Obtain the initial LP relaxation by dropping the integrality constraints

Prof. Dr. Sven Seuken - University of Zurich

Page 31: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

6 90

x2

x1

0

5

6x1+x2<=6

5x1+9x2<=45

2 4

2

4LP optimal x0=(2.25, 3.75)z0=41.25

Prof. Dr. Sven Seuken - University of Zurich

Page 32: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Let P0=LP(S0)={x in R2: x1+x2 <= 6, 5x1+9x2 <= 45,x>=0}. z0=41.25, x0=(2.25,3.75)

• Consider variable x2. Will be integer in solution to

IP. Add constraints x2 <= 3, x2 >= 4.

• Let S1 denote S0 Å {x2<= 3}; P1=LP(S1)

• Let S2 denote S0 Å {x2 >= 4}; P2=LP(S2)

• Can plot the feasible region for P1 and P2

Caution: the use of subscripts (e.g. Sk, Pk) in BnB can be to denote the index of a subproblem, not the index of a variable

Prof. Dr. Sven Seuken - University of Zurich

Page 33: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

6 90

x2

x1

0

5

6x1+x2 >= 6

5x1+9x2 >= 45

2 4

2

4

P1

P2

Prof. Dr. Sven Seuken - University of Zurich

Page 34: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Study two new IPs, with feasible solution spaces S1 and S2.

• Call S1 and S2 subproblems and this process branching.

• Consider S1 first. Optimal solution to LP(S1) is x1=(3,3), z1=39. Especially helpful! Our first feasible solution; write x*

1=(3,3), z*1=39.

• Update the incumbent, z:=max(z, z*1) = max(-

1,39) = 39, and x:=(3,3).

Prof. Dr. Sven Seuken - University of Zurich

Page 35: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Now consider S2. Optimal solution to LP(S2) is

fractional.

• x2=(1.8,4), z2=41

• If z2· z = 39 would be done… any integer solution

in S2 would be smaller yet.

• But, not the case. Divide S2 into two subproblems

• S3 = S0 Å {x2 >= 4, x1 >= 1}

• S4 = S0 Å {x2 >= 4, x1 >= 2}

• S=S1 [ S3 [ S4, S1Å S3Å S4 = ;

Prof. Dr. Sven Seuken - University of Zurich

Page 36: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Develop a search tree of IP subproblems

S0

S1 S2

x2 <= 3 x2 >= 4

39

39solvedx*1=(3,3)

S3 S4

x1 >= 2x1 <= 1

41.25

41

• Consider left branch. Solve LP(S3)

Prof. Dr. Sven Seuken - University of Zurich

Page 37: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

6 90

x2

x1

0

5

6x1+x2<=6

5x1+9x2<=45

2 4

2

4

P1

P3

P4P4

Prof. Dr. Sven Seuken - University of Zurich

Page 38: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Consider left branch. Solve LP(S3)

• x3=(1, 4 4/9), z3 = 40 5/9

• Fractional, and z3 > z = 39… again subdivide and

continue

• S5 = S0 Å {x2¸4, x1·1, x2·4} = S0 Å {x1· 1, x2=4}

• S6 = S0 Å {x2¸4, x1·1, x2¸5} = S0 Å {x1· 1, x2¸5}

Prof. Dr. Sven Seuken - University of Zurich

Page 39: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

S0

S1 S2

X2<=3 X2>=4

39

39solvedx*1=(3,3)

S3 S4

X1>=2X1<=1

S5 S6

X2>=5X2<=4

40 5/9

41.25

41

Prof. Dr. Sven Seuken - University of Zurich

Page 40: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• At this point have 3 possible directions.

• Could consider subproblems S4, S5 or S6

• To be specific in the example, consider depth first

search and consider a child of the most recently

considered subproblem; and take the left branch.

• Solve LP(S5).

• x5= (1,4), z5 = 37

• z5 · z = 39. Ignore branch, any feasible solution is

bounded above by 37.

• Say that this node is “pruned by bound.”

• (Hence “branch and bound”).

Prof. Dr. Sven Seuken - University of Zurich

Page 41: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

6 90

x2

x1

0

5

6x1+x2=6

5x1+9x2=45

2 4

2

4

P1

P5

P6

P4P4

Prof. Dr. Sven Seuken - University of Zurich

Page 42: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

S0

S1 S2

X2<=3 X2>=4

39

39solvedx*1=(3,3)

S3 S4

X1>=2X1<=1

S5 S6

X2>=5X2<=4

40 5/9

41.25

41

37

37prunedby boundProf. Dr. Sven Seuken - University of Zurich

Page 43: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Now at a leaf. Back up, find the first node with an unsolved problem.

• Right branch under S3 (i.e., S6)

• z6=40, x6=(0,5). Another feasible solution!

• Write z*6=40, x*

6=(0,5)

• Since z*6>z, z:=max(z,z*

6) =max(39,40)=40, x:=(0,5). A new incumbent.

• Finally, solve the problem under S2 (i.e. S4), and including any subproblems thereof. Infeasible.

Prof. Dr. Sven Seuken - University of Zurich

Page 44: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

6 90

x2

x1

0

5

6x1+x2=6

5x1+9x2=45

2 4

2

4

P1

P6

P4

P5

P4

Prof. Dr. Sven Seuken - University of Zurich

Page 45: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

S0

S1 S2

X2<=3 X2>=4

39

39solvedx*1=(3,3)

S3 S4

X1>=2X1<=1

S5 S6

X2>=5X2<=4

40 5/9

41.25

41

37

37prunedby bound

40

40solved x*6=(0,5)

-1

infeasible

Final search tree. Completely fathomed!

Page 46: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

• Now completely fathomed. No branches left to

explore.

• Can conclude that x*=x=(0,5), z*=z=40 is the

optimal solution.

Prof. Dr. Sven Seuken - University of Zurich

Page 47: Section #2: Linear and Integer Programming · Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping • Game Theory

Complexity of IPs

• Integer Programming Problems are NP-hard

• Thus, it is likely that even the best possible

algorithms for solving IPs will need

exponential time in the worst case!

• Depending on the amount of structure in the

problem, powerful algorithms (CPLEX) can

solve problems with hundreds of variables and

constraints

Prof. Dr. Sven Seuken - University of Zurich