Discrete Optimization 2010 Lecture 8 Lagrangian Relaxation ...
Lagrangian Relaxation of Integer...
Transcript of Lagrangian Relaxation of Integer...
Lagrangian Relaxation of Integer Programs
John E. Mitchell
Department of Mathematical SciencesRPI, Troy, NY 12180 USA
April 2019
Mitchell Lagrangian Relaxation of Integer Programs 1 / 19
An example
Outline
1 An example
2 Theoretical results
Mitchell Lagrangian Relaxation of Integer Programs 2 / 19
An example
An example problem
We consider the integer optimization problem
maxx∈R2 x1subject to 2x1 − x2 ≤ 2 } A1x ≤ b1
x1 + 2x2 ≤ 8 }x1 ≤ 3 } A2x ≤ b2
x2 ≤ 3 }
x ≥ 0, integer
(1)
Mitchell Lagrangian Relaxation of Integer Programs 3 / 19
An example
LP relaxation of the example
0 x1
x2
2x1 + x2 = 8
2x1 − x2 = 2
Optimal solution to LPrelaxation: xLP = (2.4,2.8)
Mitchell Lagrangian Relaxation of Integer Programs 4 / 19
An example
Dualize the first constraint
We relax the constraints A1x ≤ b1, placing it in the objective functionwith a penalty for violation.
The Lagrangian relaxation for any λ ≥ 0 is
maxx∈R2 x1 + λ(2− 2x1 + x2)subject to x1 + 2x2 ≤ 8
x1 ≤ 3 LR(λ)x2 ≤ 3x ≥ 0, integer
(2)
We let Q denote the feasible solutions to this relaxation.
Mitchell Lagrangian Relaxation of Integer Programs 5 / 19
An example
Feasible region for Lagrangian relaxation
0 x1
x2
conv(Q)
Mitchell Lagrangian Relaxation of Integer Programs 6 / 19
An example
Solution to Lagrangian relaxationFor a particular λ, the optimal solution to the Lagrangian relaxation isachieved at one of the points x = (0,0), (3,0), (3,2), (2,3), (0,3);these are the five extreme points of conv(Q), the convex hull of integerpoints feasible in the relaxation.
The objective function to the Lagrangian relaxation can be written
2λ + (1− 2λ)x1 + λx2.
Thus, the five extreme points lead to the following linear functions of λ:
x = (0,0) : 2λx = (3,0) : 3− 4λx = (3,2) : 3− 2λx = (2,3) : 2 + λx = (0,3) : 5λ
Thus, the value of the Lagrangian relaxation is
zLR(λ) = max{2λ,3− 4λ,3− 2λ,2 + λ,5λ}Mitchell Lagrangian Relaxation of Integer Programs 7 / 19
An example
The Lagrangian dual function
0 λ
y
1
2
3
16
13
12
y = 5λ
y = 2λ
y = 2 + λ
y = 3− 2λ
y = 3− 4λ
(12 ,2
12)(1
3 ,213)
zLR(λ) = max{2λ,3− 4λ,3− 2λ,2 + λ,5λ}
=
3− 2λ if 0 ≤ λ ≤ 1
32 + λ if 1
3 ≤ λ ≤12
5λ if λ ≥ 12
Mitchell Lagrangian Relaxation of Integer Programs 8 / 19
An example
The Lagrangian dual function
0 λ
y
1
2
3
16
13
12
y = 5λ
y = 2λ
y = 2 + λ
y = 3− 2λ
y = 3− 4λ
(12 ,2
12)(1
3 ,213)
zLR(λ) = max{2λ,3− 4λ,3− 2λ,2 + λ,5λ}
=
3− 2λ if 0 ≤ λ ≤ 1
32 + λ if 1
3 ≤ λ ≤12
5λ if λ ≥ 12
Mitchell Lagrangian Relaxation of Integer Programs 8 / 19
An example
The Lagrangian dual problem
The Lagrangian dual problem is
zLD := minλ≥0
zLR(λ)
The optimal solution is to take λ = 13 , giving zLD = 21
3 .
This optimal value is the solution to the problem
maxx∈R2 x1subject to 2x1 − x2 ≤ 2
x ∈ conv(Q)
Mitchell Lagrangian Relaxation of Integer Programs 9 / 19
An example
Interpreting the Lagrangian value
0 x1
x2
conv(Q)
maxx∈R2 x1subject to 2x1 − x2 ≤ 2
x ∈ conv(Q)
Mitchell Lagrangian Relaxation of Integer Programs 10 / 19
An example
Interpreting the Lagrangian value
0 x1
x22x1 − x2 = 2
conv(Q)
Point giving value of Lagrangian
dual: xLR = (213 ,2
23)
maxx∈R2 x1subject to 2x1 − x2 ≤ 2
x ∈ conv(Q)
Mitchell Lagrangian Relaxation of Integer Programs 10 / 19
An example
Values of relaxations
The optimal value of the integer program is zIP = 2.
Thus, for this problem, we have
2 = zIP < 213
= zLR < 2.4 = zLP .
Mitchell Lagrangian Relaxation of Integer Programs 11 / 19
Theoretical results
Outline
1 An example
2 Theoretical results
Mitchell Lagrangian Relaxation of Integer Programs 12 / 19
Theoretical results
The general formWe let zIP denote the optimal value of the integer program
zIP := max cT xsubject to A1x ≤ b1 (complicating constraints)
A2x ≤ b2 (nice constraints)x ∈ Zn
+
(3)
The matrices A1 ∈ Rm1×n and A2 ∈ Rm2×n, and all vectors aredimensioned appropriately.
For any λ ∈ Rm1+ , we obtain the Lagrangian relaxation
zLR(λ) := max cT x + λT (b1 − A1x)subject to A2x ≤ b2
x ∈ Zn+
(4)
Mitchell Lagrangian Relaxation of Integer Programs 13 / 19
Theoretical results
We do have a relaxation
PropositionProblem (4) is a relaxation of problem (3) for any λ ≥ 0, in that any xfeasible in (3) is also feasible in (4) with no smaller objective functionvalue.
Proof.If x is feasible in (3) then it is feasible in (4) with objective function value
cT x + λT (b1 − A1x) ≥ cT x
since λ ≥ 0 and A1x ≤ b1.
Mitchell Lagrangian Relaxation of Integer Programs 14 / 19
Theoretical results
We do have a relaxation
PropositionProblem (4) is a relaxation of problem (3) for any λ ≥ 0, in that any xfeasible in (3) is also feasible in (4) with no smaller objective functionvalue.
Proof.If x is feasible in (3) then it is feasible in (4) with objective function value
cT x + λT (b1 − A1x) ≥ cT x
since λ ≥ 0 and A1x ≤ b1.
Mitchell Lagrangian Relaxation of Integer Programs 14 / 19
Theoretical results
The Lagrangian dual problem
Thus, the optimal value of problem the Lagrangian relaxation (4) givesan upper bound on the optimal value of the integer program (3) for anyλ ≥ 0.
In the Lagrangian dual problem, this upper bound is minimized:
zLD := min zLR(λ)subject to λ ≥ 0.
(5)
Mitchell Lagrangian Relaxation of Integer Programs 15 / 19
Theoretical results
Relating the Lagrangian dual to the original problem
We immediately obtain the following proposition:
PropositionThe optimum value of the Lagrangian dual problem (5) is no smallerthan the optimum value of the integer program (3).
Mitchell Lagrangian Relaxation of Integer Programs 16 / 19
Theoretical results
Characterizing the optimal dual valueWe can use polyhedral theory to characterize the optimal value of theLagrangian dual (5) more precisely. First, let
Q := {x ∈ Zn+ : A2x ≤ b2}, (6)
the feasible region for the Lagrangian relaxation (4) for any λ ≥ 0.
Theorem
The optimum value zLD of the Lagrangian dual (5) is given by
zLD = max{cT x : A1x ≤ b1, x ∈ conv(Q)},
a linear program.
This is Theorem 6.2 on page 327, and the proof can be found in thetextbook. In the proof, conv(Q) is represented in terms of its extremepoints and rays and then linear programming duality is exploited toderive the result.
Mitchell Lagrangian Relaxation of Integer Programs 17 / 19
Theoretical results
Comparing the values
The LP relaxation of (3) has value
zLP := max cT xsubject to A1x ≤ b1
A2x ≤ b2
x ∈ Rn+
(7)
The three values are compared in the following theorem:
TheoremzIP ≤ zLD ≤ zLP , and all of the inequalities can be strict.
The proof of the inequalities follows directly from Theorem 1.
Our earlier example shows the inequalities can all be strict.
Mitchell Lagrangian Relaxation of Integer Programs 18 / 19
Theoretical results
Making the relaxation too easy
TheoremzLD = zLP if all the extreme points of {x ∈ Rn
+ : A2x ≤ b2} are integral.
Proof.In this case, we have
conv(Q) = {x ∈ Rn+ : A2x ≤ b2},
so the result follows from Theorem 1.
So: don’t want to choose the nice constraints so that the Lagrangianrelaxation is too easy to solve, because then may get no improvementover the LP relaxation. For example, don’t choose the nice constraintsso that the constraint matrix A2 is totally unimodular.
Mitchell Lagrangian Relaxation of Integer Programs 19 / 19
Theoretical results
Making the relaxation too easy
TheoremzLD = zLP if all the extreme points of {x ∈ Rn
+ : A2x ≤ b2} are integral.
Proof.In this case, we have
conv(Q) = {x ∈ Rn+ : A2x ≤ b2},
so the result follows from Theorem 1.
So: don’t want to choose the nice constraints so that the Lagrangianrelaxation is too easy to solve, because then may get no improvementover the LP relaxation. For example, don’t choose the nice constraintsso that the constraint matrix A2 is totally unimodular.
Mitchell Lagrangian Relaxation of Integer Programs 19 / 19