Post on 28-Aug-2018
CSCI5654 (Linear Programming, Fall 2013)Lecture-8
Lecture 8 Slide# 1
Today’s Lecture
1. Recap of dual variables and strong duality.
2. Complementary Slackness Theorem.
3. Interpretation of dual variables.
4. Primal and Dual Simplex.
Lecture 8 Slide# 2
Primal-Dual correspondences
Variable to constraint correspondence:
Dual variable (yj) ↔ Primal constraint Aj~x ≤ bj
Primal variable (xi ) ↔ Dual constraint At
i ~y ≥ ci
Correspondence in dictionary:
Dual decision variable yj ↔ Primal slack variable xn+j .
Primal decision variable (xi ) ↔ Dual dictionary slack variable ym+i .
Illustrate in class with an example.
Lecture 8 Slide# 3
Strong Duality Theorem: If x∗1 , . . . , x∗m is primal optimal then there
exists a dual optimal solution y∗1 , . . . , y∗m that satisifies∑
i
cix∗i =
∑j
bjy∗j .
(∴ if primal has optimal solution then the dual has optimum with thesame value as primal).Proof: This is theorem 5.1 in Chvatal (pages 58-59).
Lecture 8 Slide# 4
Example
maximize x1 + 2x2s.t. x1 ≤ 3
x2 ≤ 3−x1 + x2 ≤ 1x1 + x2 ≤ 5
x1, x2 ≥ 0
Lecture 8 Slide# 5
Example (final dictionary)
x1 = 2 + x52 − x6
2x2 = 3 − x5
2 + x62
x3 = · · ·x4 = · · ·z = 8 − 1
2x5 − 32x6
Insight: Dual variables correspond to primal slack variables.
x3 ↔ y1, x4 ↔ y2, x5 ↔ y3, x6 ↔ y4 .
Read off dual solution by from objective row of final dictionary:
y1 : 0, y2 : 0, y3 :1
2, y4 :
3
2.
Lecture 8 Slide# 6
Proof of Strong Duality
Final Dictionary:
xi = bi +∑
j∈Independent aijxj...
z = z∗ + c∗1x1 + · · ·+ c∗n+mxn+m
Note: c∗i = 0 if i ∈ Basisc∗i ≤ 0 if i ∈ Independent.
Claim: Set variable yi = −(c∗n+i ). This is a dual feasible solution withoptimal value z∗.
Lecture 8 Slide# 7
Relationship Between Primal/Dual
Primal
Optimal Infeasible UnboundedOptimal Possible Impossible Impossible
Dual Infeasible Impossible Possible PossibleUnbounded Impossible Possible Impossible
Lecture 8 Slide# 8
Primal and Dual Infeasible
Primal:max. x2
s.t. x1 ≤ −1−x2 ≤ −1x1, x2 ≥ 0
Dual:min. −y1 − y2
s.t. y1 ≥ 0−y2 ≥ 1y1, y2 ≥ 0
Lecture 8 Slide# 9
Complementary Slackness Theorem I
Let x1, . . . , xn, xn+1, . . . , xn+m︸ ︷︷ ︸slack
be a primal feasible.
Let y1, . . . , ym, ym+1, . . . , ym+n︸ ︷︷ ︸slack
be dual feasible.
Slack Variables:xn+i = bi −
∑nk=1 aikxj
ym+j = ck −∑m
k=1 akjyj
The following conditions are necessary and sufficient for ~x and ~y to beoptimal solutions to the primal and dual:
xiym+i = 0, j = 1, . . . , nyjxn+j = 0, i = 1, . . . ,m
Proof: ~x is a primal feasible solution. Therefore,
A~x ≤ ~b, ~x ≥ 0 .
Lecture 8 Slide# 10
Complementary Slackness Theorem II
Similarly, ~y is dual feasible. Therefore,
At~y ≥ ~c , ~y ≥ 0 .
Applying these facts, we obtain
~ct~x ≤ (A
t~y)
t~x(≡ ~y t
A~x)
≤ ~yt~b
~x and ~y can be optimal solutions for primal/dual resp. iff the inequalitiesabove all hold with equalities.Therefore,
~yt~b = ~y
tA~x (1)
~ct~x = ~y
tA~x (2)
Lecture 8 Slide# 11
Complementary Slackness Theorem IIIFrom the equation (1), we obtain
~yt(~b − Ax) = 0, or
m∑i=1
yi (bi − Ai~x) = 0 .
Note that yi ≥ 0 and bi − Ai~x ≥ 0. Sum of non-negative terms is zero iffeach of the individual terms are zero. Therefore
yi (bi − Ai~x) = 0
Note that bi − Ai~x is just the slack variable xn+i .Considering the first equation will derive the remainder of the requiredcomplementary slackness conditions.
Lecture 8 Slide# 12
Duality: Interpretation
Diet problem: Minimize cost, while satisfying dietary constraints.FX ($2) FY ($5) FZ ($ 15)
Carbs 20 1 1Protein 1 30 40
Vitamins 1 10 5
1. At least 200units of carbs.
2. At least 50 units of protien.
3. At least 40 units of vitamins.
4. No more than 20 units of FX.
5. No more than 10 units of FY.
6. No more than 5 units of FZ.
Lecture 8 Slide# 13
Diet Problem
max. −2x1 − 5x2 − 15x3−20x1 − x2 − x3 ≤ −200 ← CARBS−x1 − 30x2 − 40x3 ≤ −50 ← PROT−x1 − 10x2 − 5x3 ≤ −40 ← VIT
x1 ≤ 20 ← LIM FXx2 ≤ 10 ← LIM FYx3 ≤ 5 ← LIM FZ
x1, x2, x3 ≥ 0
Optimal: x1 ∼ 9.84, x2 ∼ 3, x3 : 0.Dual: y1 : 0.07, y2 : 0, y3 : 0.5, y4, . . . , y6 : 0.
Lecture 8 Slide# 14
Shadow costs
max. −2x1 − 5x2 − 15x3−20x1 − x2 − x3 ≤ −200 ← CARBS(0.07)−x1 − 30x2 − 40x3 ≤ −50 ← PROT(0)−x1 − 10x2 − 5x3 ≤ −40 ← VIT(0.5)
x1 ≤ 20 ← LIM FX 0x2 ≤ 10 ← LIM FY 0x3 ≤ 5 ← LIM FZ 0
x1, x2, x3 ≥ 0
Experiment: Play around with RHS of inequalities and observe solution.
Lecture 8 Slide# 15
Shadow Cost (Margin Cost)
max. −2x1 − 5x2 − 15x3−20x1 − x2 − x3 ≤ −200 ← CARBS(0.07)−x1 − 30x2 − 40x3 ≤ −50 ← PROT(0)−x1 − 10x2 − 5x3 ≤ −40 ← VIT(0.5)
x1 ≤ 20 ← LIM FX 0x2 ≤ 10 ← LIM FY 0x3 ≤ 5 ← LIM FZ 0
x1, x2, x3 ≥ 0
CARBS: $0.07, PROT: free, VIT: $0.5
Q1: If dual var. corresponding to a constraint is 0 does that mean that wecan drop the constraint and still get the same answer?A1: YES.Q2: Can we, then, change the RHS of the inequality to arbitrary values?A2: NO.
Lecture 8 Slide# 16
Sensitivity TheoremConsider Primal Problem: decision variables are x1, . . . , xn.
maximize c1x1 + · · · + cnxns.t. aj1x1 + · · · + ajnxn ≤ bj j ∈ {1, 2, . . . ,m}
x1, . . . , xn ≥ 0
Optimal Value: z∗.Primal Optimal: x∗1 , . . . , x
∗n . Dual Optimal: y∗1 , . . . , y
∗n .
Final simplex dictionary is non-degenerate.Modified LP:
maximize c1x1 + · · · + cnxns.t. aj1x1 + · · · + ajnxn ≤ bj+tj j ∈ {1, 2, . . . ,m}
x1, . . . , xn ≥ 0
Optimal Value: There is ε > 0 s.t., if |ti | ≤ ε, z ′ = z∗ +∑
i tiyi .Shadow Cost: The dual decision variables are interpreted as “shadowcost”.Lecture 8 Slide# 17
Primal-Dual Simplex
Lecture 8 Slide# 18
Primal and Dual
Consider the following LP:
max. −x1 −x2−2x1 −x2 ≤ 4−2x1 +4x2 ≤ −8−x1 +3x2 ≤ −7
x1, x2 ≥ 0
Lecture 8 Slide# 19
Dual LP
The dual LP for our example:
max. −4y1 +8y2 +7y32y1 +2y2 +y3 ≤ 1y1 −4y2 −3y3 ≤ 1
y1, y2, y3 ≥ 0
Lecture 8 Slide# 20
Primal-Dual Dictionaries
x3 4 +2x1 +x2x4 −8 +2x1 −4x2x5 −7 +x1 −3x2z −x1 −x2
y4 1 −2y1 −2y2 −y3y5 1 −y1 +4y2 +3y3−w −4y1 +8y2 +7y3
Note: Dual dictionary y2 enters and y4 leaves.
Lecture 8 Slide# 21
Primal-Dual Dictionaries
Complementary pairs:
x3 ↔ y1, x4 ↔ y2, x5 ↔ y3x1 ↔ y4, x2 ↔ y5
Rule:
I v1 enters and v2 leaves dual dictionary.v c1 leaves and v c2 enters primal dictionary.
I v c1 enters and v c2 leaves primal dictionary.v1 leaves and v2 enters dual dictionary.
Lecture 8 Slide# 22
Primal Dual Example
y2 .5 −y1 −.5y3 −.5y4y5 3 −5y1 +y3 −2y4−w 4 −12y1 +3y3 −4y4
x3 12 +x4 +5x2x5 −3 +.5x4 −x2x1 4 +.5x4 +2x2z −4 −.5x4 −3x2
Lecture 8 Slide# 23
Primal Dual Examples
y3 1 −2y1 −2y2 −y4y5 4 −7y1 −2y2 −3y4−w 7 −18y1 −6y2 −7y4
Can you write down the primal dictionary??
Lecture 8 Slide# 24
Dual Simplex
I Easy transformation of primal to dual and vice versa.
I Entering variable for dual ⇒ complementary variable leaves primaldictionary.
I Leaving variable ⇒ complementary variable enters primal.
I Unbounded dual ⇒ infeasible primal (and vice versa).
Lecture 8 Slide# 25
Why Dual Simplex?
1. When primal objective coefficients are all non-positive.Avoid need for an initialization phase.
2. When primal has more variables than constraints.Simplex usually observed to be sensitive to number of variables.
3. As an alternative to the initialization phase!
Lecture 8 Slide# 26
Initialization using Dual Simplex
Lecture 8 Slide# 27
Example
maximize −x1 +4x2subjectto −2x1 −x2 ≤ 4
−2x1 +4x2 ≤ −8−x1 +3x2 ≤ −7
x1, x2 ≥ 0
Dual:max −4y1 +8y2 +7y3
2y1 +2y2 +y3 ≤ 1y1 −4y2 −3y3 ≤ −4
y1, y2, y3 ≥ 0
Problem: Neither primal nor dual initial dictionary is feasible.
Lecture 8 Slide# 28
Initialization using Dual
Idea: Change of primal objective function can make dual initial dictionaryfeasible!!
−x1 + 4x2 → −x1 − x2
Dual:max −4y1 +8y2 +7y3
2y1 +2y2 +y3 ≤ 1y1 −4y2 −3y3 ≤ 1
y1, y2, y3 ≥ 0
The change ensures that dual starts at a feasible dictionary.
Lecture 8 Slide# 29
Initialization Using Dual
I Dualize given problem.
I If dual starting dictionary is infeasible, then change primal objectiveto make initial dual dictionary feasible.
I Perform dual simplex until a dual optimal dictionary is reached.
I Change final dual dictionary into corresponding primal dictionary.
I Replace the objective function, taking into account new basisvariables.
Lecture 8 Slide# 30