Mixed Integer Linear Programming

8/10/2019 Mixed Integer Linear Programming

1/42

6. Mixed Integer Linear Programming

Javier Larrosa

Albert Oliveras

Enric Rodrguez-Carbonell

Problem Solving and Constraint Programming (RPAR)

Session 6 p.1/40


2/42

Mixed Integer Linear Programming

Amixed integer linear program(MILP,MIP) is of the form

min cTx

Ax=b

x0

xiZ

i IIf all variables need to be integer,it is called a(pure) integer linear program(ILP, IP)

If all variables need to be0or1(binary, boolean),it is called a0 1linear program

Session 6 p.2/40


3/42

Applications of MIP

Used in contexts where, e.g.:

it only makes sense to take integral quantities

of certain goods or resources, e.g.:men(human resources planning)

power stations(facility location)

binary decisions need to be taken

producing a product(production planning)

assigning a task to a worker(assignment problems)assigning a slot to a course(timetabling)

And many many more...

Session 6 p.3/40


4/42

Computational Complexity: LP vs. IP

Including integer variables increases enourmously themodeling power,at the expense of more complexity

LPs can be solved inpolynomial timewith interior-pointmethods (ellipsoid method, Karmarkars algorithm)

Integer Programming is anNP-completeproblem. So:

There isno known polynomial-time algorithm

There arelittle chancesthat one will ever be found

Even small problems may be hard to solveWhat follows is one of the many approaches(and one of the most successful) for attacking IPs

Session 6 p.4/40


5/42

LP Relaxation of a MIP

Given a MIP

(IP)

min cTx

Ax=b

x0

xi Z i I

itslinear relaxationconsists in the LP obtained bydropping the integrality constraints:

(LP)

min cTx

Ax=b

x0

Can we solve IPby solvingLP? By rounding?Session 6 p.5/40


6/42

Branch & Bound (1)

The optimal solution of

max x+y

2x+ 2y 1

8x+ 10y 13

x, y

Z

is(x, y) = (1, 2)with objective3

The optimal solution of its LP relaxation

is(x, y) = (4, 4.5)with objective9.5

No direct way of getting from(x, y) = (4, 4.5)to(x, y) = (1, 2)by rounding!

Something more elaborate is needed: branch & boundSession 6 p.6/40


7/42

Branch & Bound (2)

(1, 2)

y

x

x 0

y0

max x+y

(4, 4.5)

2x + 2y1

8x + 10y13

Session 6 p.7/40


8/42

Branch & Bound (3)

Assume integer variables have lower and upper boundsLetP0 initial problem,LP(P0)LP relaxation of P0

If in optimal solution of LP(P0)all integer variables takeinteger values then it is also an optimal solution to P0

Else

Roundingthe solution ofLP(P0)mayyieldtonon-optimalornon-feasiblesolutions for P0!

Letxj be integer variable whose value j at optimal

solution ofLP(P0)satisfies j Z. Define

P1:=P0 xj j

P2:=P0 xj j

Feasible solutions to P0 =feasible solutions to P1 orP2Session 6 p.8/40


9/42

Branch & Bound (4)

Letxj be integer variable whose value j at optimalsolution ofLP(P0)satisfies j Z.

P1:=P0 xj j P2 :=P0 xj j

Pi can besolved recursively

We can build abinary tree of subproblemswhoseleaves correspond topendingproblems still to be solved

Terminates as integer vars have finite bounds and, ateach split, range of one var becomes strictly smaller

IfLP(Pi)has optimal solution where integer variablestake integer values then solution is stored

IfLP(Pi)is infeasible then Pi can be discarded

(pruned, fathomed)

Session 6 p.9/40


10/42

Example (1)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

End

====================================================================

Status: OPTIMAL

Objective: obj = 8.5 (MAXimum)

No. Column name St Activity Lower bound Upper bound

------ ------------ -- ------------- ------------- -------------

1 x B 4 0

2 y B 4.5 0

Session 6 p.10/40


11/42


12/42

Example (3)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

y < = 4

End

====================================================================

Status: OPTIMAL


No. Column name St Activity Lower bound Upper bound

------ ------------ -- ------------- ------------- -------------1 x B 3.5 0

2 y NU 4 0 4

Session 6 p.12/40


13/42


14/42

Example (5)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 3

y < = 4

End

====================================================================

Status: OPTIMAL


No. Column name St Activity Lower bound Upper bound------ ------------ -- ------------- ------------- -------------

1 x NU 3 0 3

2 y B 3.7 0 4

Session 6 p.14/40


15/42

Example (6)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 3

y = 4

End

====================================================================

GLPSOL: GLPK LP/MIP Solver 4.38

...

PROBLEM HAS NO PRIMAL FEASIBLE SOLUTION

Session 6 p.15/40


16/42

Example (7)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 3

y < = 3

End

====================================================================

Status: OPTIMAL



1 x B 2.5 0 3

2 y NU 3 0 3

Session 6 p.16/40


17/42

Example (8)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x = 3

y < = 3

End

====================================================================


...


Session 6 p.17/40


18/42

Example (9)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 2

y < = 3

End

====================================================================

Status: OPTIMAL



1 x NU 2 0 2

2 y B 2.9 0 3

Session 6 p.18/40


19/42

Example (10)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 2

y = 3

End

====================================================================


...


Session 6 p.19/40


20/42

Example (11)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 2

y < = 2

End

====================================================================

Status: OPTIMAL



1 x B 1.5 0 2

2 y NU 2 0 2

Session 6 p.20/40


21/42

Example (12)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x = 2

y < = 2

End

====================================================================


...


Session 6 p.21/40


22/42

Example (13)

Max obj: x + y

Subject To

c1: -2 x + 2 y >= 1

c 2 : - 8 x + 1 0 y < = 1 3

Bounds

x < = 1

y < = 2

End

====================================================================

Status: OPTIMAL

Objective: obj = 3 (MAXimum)


1 x NU 1 0 1

2 y NU 2 0 2

Session 6 p.22/40


23/42

Pruning in Branch & Bound

We have already seen that if relaxation is infeasible,the problem can be pruned

Now assume an (integral) solution has been foundIf solution has costZthen any pending problem Pjwhose relaxation has optimal value> Zcan be ignored

cost(Pj)cost(LP(Pj))> Z

The optimum will not be in any descendant of Pj!

Thispruningof the search tree has a huge impacton the efficiency of Branch & Bound

Session 6 p.23/40


24/42

Unboundedness in Branch & Bound

We assumed integer variables are bounded

In mixed problems,

we allow non-integer variables to be unboundedAssumeLP(Pi)isunbounded. Then:

If in basic solution integer variables take integervalues then the problem is unbounded(assuming that problem data are rational numbers)

Else we proceed recursivelyas if an optimal solution toLP(Pi)had been found.

Whats different wrtLP(Pi)having optimal solution?

Session 6 p.24/40


25/42

Unboundedness in Branch & Bound

We assumed integer variables are bounded

In mixed problems,

we allow non-integer variables to be unboundedAssumeLP(Pi)isunbounded. Then:

If in basic solution integer variables take integervalues then the problem is unbounded(assuming that problem data are rational numbers)

Else we proceed recursivelyas if an optimal solution toLP(Pi)had been found.

Whats different wrtLP(Pi)having optimal solution?IfLP(P

i)is unbounded then P

icannot be pruned:

no need to check this!Session 6 p.24/40


26/42

Branch & Bound: Algorithmic Description

S:={P0

} /* set of pending problems */Z := + /* best cost found so far */

whileS=do

removeP fromS; solveLP(P)

ifLP(P)is feasiblethen

Letbe basic solution obtained after solving LP(P)

ifsatisfies integrality constraintsthen

ifis optimal forLP(P)then

ifcost()< Z thenstore; updateZ

else returnUNBOUNDED

else

ifis optimal forLP(P)Pcan be prunedthen continue

Letxj be integer variable such thatj Z

S:=S {P xj j, P xj j}returnZ

Session 6 p.25/40


27/42

Heuristics in Branch & Bound

Possible choices in Branch & Bound

Choosing apending problem

Depth-firstsearchBreadth-firstsearchBest-firstsearch (select node with best cost value)

Choosing abranching variable

Thatclosest to halfway two integervaluesThat withleast costcoefficientThat which isimportant inthemodel(0-1 variable)That which is biggest in avariable ordering

No known strategy is best for all problems!Session 6 p.26/40


28/42

Remarks on Branch & Bound

If integer variables not bounded, B&B may not terminate

min 0

13x 3y2

x, y Z

is infeasible but B&B loops forever looking for solutions!

New problems neednotbe solvedfrom scratchbutstarting from optimal solutionof parent problem

Dual Simplex Methodcan be used:dual feasibility preserved if change bounds of basic vars

Often Dual Simplex needs few iterations to obtain an

optimal solution to new problem(reoptimization)

Session 6 p.27/40


29/42

Lower Bounding Procedures

Pruning at a node is achieved here bysolving the LP relaxation with dual simplex

But there exist other procedures for giving lower bounds

on best objective value:Lagrangian relaxation

(M IP)

min cTx

Axb

xu

xi Z i I

(LG)

min cTx+(Ax b)

xuxi Z i I

with0

LGis a relaxation: gives lower bound on M IPLGcan be trivially solved

For good , LGis as good as LPrelax. but cheaper

Concrete problems have ad-hoc lower bounding procs

Session 6 p.28/40


30/42

C i Pl (2)


31/42

Cutting Planes (2)

(1, 2)

y

xy0

max x+y

(4, 4.5)

2x + 2y1

x+ y6

8x + 10y13

x0

(0, 1)

Session 6 p.30/40

U i C f S l i MIP


32/42

Using Cuts for Solving MIP

LetaTx bbe a cut. Then the MIP

min cTx

xS where S =

x Rn

Ax=b

aTx b

xu

xi Z i I

has thesame set of feasible solutions Sbut its LPrelaxation is strictly more constrained

Rather than splitting into subproblems as in Branch &Bound, one can add the cut and solve the relaxation

Used together with Branch & Bound: Branch & Cut

If after adding cuts no solution is found, then branch

Session 6 p.31/40

G C t (1)


33/42

Gomory Cuts (1)

There are several techniques for deriving cuts

Some are problem-specific (e.g, travelling salesman)

Here we will see a generic technique: Gomory cuts

Let us consider a tableau with a row of the form

xi=i+

jR aijxj (i B)

Letbe an associated basic solution such that

1. i I

2. (xi) Z

3. For allj Rwe have (xj) =j or(xj) =uj

Can think that it is the optimal tableau of the relaxationSession 6 p.32/40

G C t (2)


34/42

Gomory Cuts (2)

Let=(xi) (xi). Then0<


35/42

Gomory Cuts (3)

xi=i+

jR aijxj

(xi) =i+jR aij(xj)Subtracting

xi (xi) = jR aij(xj (xj))=

jL aij(xj j)

jUaij(uj xj)

Finally

xi (xi)=+

jL aij(xj j)

jUaij(uj xj)

Let us define

L

+

={j L |aij 0} L

={j L |aij


36/42

Gomory Cuts (4)

xi (xi)=+

jL aij(xj j)

jUaij(uj xj)

Assume

jLaij(xj j) jU

aij(uj xj)0. Then

+jL

aij(xj j) jU

aij(uj xj)1

jL+

aij(xj j) jU

aij(uj xj)1

jL+

aij

1 (xj j) +jU

aij1

(uj xj)1

Moreover

jLaij

(xj j) +

jU+

aij

(uj xj)0

Session 6 p.35/40

G C t (5)


37/42

Gomory Cuts (5)

xi (xi)=+

jL aij(xj j)

jUaij(uj xj)

Assume

jLaij(xj j) jU

aij(uj xj)


38/42

Gomory Cuts (6)

In any case

jL

aij

(xj j)+jU+

aij

(uj xj)+

jL+

aij

1 (xj j)+

jU

aij1 (uj xj)1

for any xS.However,does not satisfy this inequality

(setxj =j for j L, andxj =uj j U)

Session 6 p.37/40

E i All V ti A I t (1)


39/42

Ensuring All Vertices Are Integer (1)

Let us assume A, bhave coefficients in ZSometimes it is possible to ensure for an IP thatall vertices of the relaxation are integer

For instance, when the matrix Aistotally unimodular:the determinant of every square submatrix is 0or1

Sufficient condition:property K

Each element of Ais0or1

No more than two non-zeros appear in each columm

Rows can be partitioned in two subsets R1 andR2 s.t.If a column contains two non-zeros of the samesign, one element is in each of the subsets

If a column contains two non-zeros of differentsigns, both elements belong to the same subset

Session 6 p.38/40

A i t P bl


40/42

Assignment Problem

m= #of workers= #of tasksEach worker must be assigned to exactly one task

Each task is to be performed by exactly one worker

cij =cost when worker iperforms task j

Session 6 p.39/40


41/42



42/42


Several kinds of IPs satisfy property K:

Assignment

TransportationMaximum flow

Shortest path

...

Usually specialized network algorithms are moreefficient for these problems than simplex techniques

But simplex techniques are general and can be used ifno implementation of network algorithms is available

Mixed Integer Linear Programming

Documents

Transcript of Mixed Integer Linear Programming