CSCI5654 (Linear Programming, Fall 2013)Lecture-8

Lecture 8 Slide# 1

Today’s Lecture

1. Recap of dual variables and strong duality.

2. Complementary Slackness Theorem.

3. Interpretation of dual variables.

4. Primal and Dual Simplex.

Lecture 8 Slide# 2

Primal-Dual correspondences

Variable to constraint correspondence:

Dual variable (yj) ↔ Primal constraint Aj~x ≤ bj

Primal variable (xi ) ↔ Dual constraint At

i ~y ≥ ci

Correspondence in dictionary:

Dual decision variable yj ↔ Primal slack variable xn+j .

Primal decision variable (xi ) ↔ Dual dictionary slack variable ym+i .

Illustrate in class with an example.

Lecture 8 Slide# 3

Strong Duality Theorem: If x∗1 , . . . , x∗m is primal optimal then there

exists a dual optimal solution y∗1 , . . . , y∗m that satisifies∑

i

cix∗i =

∑j

bjy∗j .

(∴ if primal has optimal solution then the dual has optimum with thesame value as primal).Proof: This is theorem 5.1 in Chvatal (pages 58-59).

Lecture 8 Slide# 4

Example

maximize x1 + 2x2s.t. x1 ≤ 3

x2 ≤ 3−x1 + x2 ≤ 1x1 + x2 ≤ 5

x1, x2 ≥ 0

Lecture 8 Slide# 5

Example (final dictionary)

x1 = 2 + x52 − x6

2x2 = 3 − x5

2 + x62

x3 = · · ·x4 = · · ·z = 8 − 1

2x5 − 32x6

Insight: Dual variables correspond to primal slack variables.

x3 ↔ y1, x4 ↔ y2, x5 ↔ y3, x6 ↔ y4 .

Read off dual solution by from objective row of final dictionary:

y1 : 0, y2 : 0, y3 :1

2, y4 :

3

2.

Lecture 8 Slide# 6

Proof of Strong Duality

Final Dictionary:

xi = bi +∑

j∈Independent aijxj...

z = z∗ + c∗1x1 + · · ·+ c∗n+mxn+m

Note: c∗i = 0 if i ∈ Basisc∗i ≤ 0 if i ∈ Independent.

Claim: Set variable yi = −(c∗n+i ). This is a dual feasible solution withoptimal value z∗.

Lecture 8 Slide# 7

Relationship Between Primal/Dual

Primal

Optimal Infeasible UnboundedOptimal Possible Impossible Impossible

Dual Infeasible Impossible Possible PossibleUnbounded Impossible Possible Impossible

Lecture 8 Slide# 8

Primal and Dual Infeasible

Primal:max. x2

s.t. x1 ≤ −1−x2 ≤ −1x1, x2 ≥ 0

Dual:min. −y1 − y2

s.t. y1 ≥ 0−y2 ≥ 1y1, y2 ≥ 0

Lecture 8 Slide# 9

Complementary Slackness Theorem I

Let x1, . . . , xn, xn+1, . . . , xn+m︸ ︷︷ ︸slack

be a primal feasible.

Let y1, . . . , ym, ym+1, . . . , ym+n︸ ︷︷ ︸slack

be dual feasible.

Slack Variables:xn+i = bi −

∑nk=1 aikxj

ym+j = ck −∑m

k=1 akjyj

The following conditions are necessary and sufficient for ~x and ~y to beoptimal solutions to the primal and dual:

xiym+i = 0, j = 1, . . . , nyjxn+j = 0, i = 1, . . . ,m

Proof: ~x is a primal feasible solution. Therefore,

A~x ≤ ~b, ~x ≥ 0 .

Lecture 8 Slide# 10

Complementary Slackness Theorem II

Similarly, ~y is dual feasible. Therefore,

At~y ≥ ~c , ~y ≥ 0 .

Applying these facts, we obtain

~ct~x ≤ (A

t~y)

t~x(≡ ~y t

A~x)

≤ ~yt~b

~x and ~y can be optimal solutions for primal/dual resp. iff the inequalitiesabove all hold with equalities.Therefore,

~yt~b = ~y

tA~x (1)

~ct~x = ~y

tA~x (2)

Lecture 8 Slide# 11

Complementary Slackness Theorem IIIFrom the equation (1), we obtain

~yt(~b − Ax) = 0, or

m∑i=1

yi (bi − Ai~x) = 0 .

Note that yi ≥ 0 and bi − Ai~x ≥ 0. Sum of non-negative terms is zero iffeach of the individual terms are zero. Therefore

yi (bi − Ai~x) = 0

Note that bi − Ai~x is just the slack variable xn+i .Considering the first equation will derive the remainder of the requiredcomplementary slackness conditions.

Lecture 8 Slide# 12

Duality: Interpretation

Diet problem: Minimize cost, while satisfying dietary constraints.FX ($2) FY ($5) FZ ($ 15)

Carbs 20 1 1Protein 1 30 40

Vitamins 1 10 5

1. At least 200units of carbs.

2. At least 50 units of protien.

3. At least 40 units of vitamins.

4. No more than 20 units of FX.

5. No more than 10 units of FY.

6. No more than 5 units of FZ.

Lecture 8 Slide# 13

Diet Problem

max. −2x1 − 5x2 − 15x3−20x1 − x2 − x3 ≤ −200 ← CARBS−x1 − 30x2 − 40x3 ≤ −50 ← PROT−x1 − 10x2 − 5x3 ≤ −40 ← VIT

x1 ≤ 20 ← LIM FXx2 ≤ 10 ← LIM FYx3 ≤ 5 ← LIM FZ

x1, x2, x3 ≥ 0

Optimal: x1 ∼ 9.84, x2 ∼ 3, x3 : 0.Dual: y1 : 0.07, y2 : 0, y3 : 0.5, y4, . . . , y6 : 0.

Lecture 8 Slide# 14

Shadow costs

max. −2x1 − 5x2 − 15x3−20x1 − x2 − x3 ≤ −200 ← CARBS(0.07)−x1 − 30x2 − 40x3 ≤ −50 ← PROT(0)−x1 − 10x2 − 5x3 ≤ −40 ← VIT(0.5)

x1 ≤ 20 ← LIM FX 0x2 ≤ 10 ← LIM FY 0x3 ≤ 5 ← LIM FZ 0

x1, x2, x3 ≥ 0

Experiment: Play around with RHS of inequalities and observe solution.

Lecture 8 Slide# 15

Shadow Cost (Margin Cost)

max. −2x1 − 5x2 − 15x3−20x1 − x2 − x3 ≤ −200 ← CARBS(0.07)−x1 − 30x2 − 40x3 ≤ −50 ← PROT(0)−x1 − 10x2 − 5x3 ≤ −40 ← VIT(0.5)

x1 ≤ 20 ← LIM FX 0x2 ≤ 10 ← LIM FY 0x3 ≤ 5 ← LIM FZ 0

x1, x2, x3 ≥ 0

CARBS: $0.07, PROT: free, VIT: $0.5

Q1: If dual var. corresponding to a constraint is 0 does that mean that wecan drop the constraint and still get the same answer?A1: YES.Q2: Can we, then, change the RHS of the inequality to arbitrary values?A2: NO.

Lecture 8 Slide# 16

Sensitivity TheoremConsider Primal Problem: decision variables are x1, . . . , xn.

maximize c1x1 + · · · + cnxns.t. aj1x1 + · · · + ajnxn ≤ bj j ∈ {1, 2, . . . ,m}

x1, . . . , xn ≥ 0

Optimal Value: z∗.Primal Optimal: x∗1 , . . . , x

∗n . Dual Optimal: y∗1 , . . . , y

∗n .

Final simplex dictionary is non-degenerate.Modified LP:

maximize c1x1 + · · · + cnxns.t. aj1x1 + · · · + ajnxn ≤ bj+tj j ∈ {1, 2, . . . ,m}

x1, . . . , xn ≥ 0

Optimal Value: There is ε > 0 s.t., if |ti | ≤ ε, z ′ = z∗ +∑

i tiyi .Shadow Cost: The dual decision variables are interpreted as “shadowcost”.Lecture 8 Slide# 17

Primal-Dual Simplex

Lecture 8 Slide# 18

Primal and Dual

Consider the following LP:

max. −x1 −x2−2x1 −x2 ≤ 4−2x1 +4x2 ≤ −8−x1 +3x2 ≤ −7

x1, x2 ≥ 0

Lecture 8 Slide# 19

Dual LP

The dual LP for our example:

max. −4y1 +8y2 +7y32y1 +2y2 +y3 ≤ 1y1 −4y2 −3y3 ≤ 1

y1, y2, y3 ≥ 0

Lecture 8 Slide# 20

Primal-Dual Dictionaries

x3 4 +2x1 +x2x4 −8 +2x1 −4x2x5 −7 +x1 −3x2z −x1 −x2

y4 1 −2y1 −2y2 −y3y5 1 −y1 +4y2 +3y3−w −4y1 +8y2 +7y3

Note: Dual dictionary y2 enters and y4 leaves.

Lecture 8 Slide# 21

Primal-Dual Dictionaries

Complementary pairs:

x3 ↔ y1, x4 ↔ y2, x5 ↔ y3x1 ↔ y4, x2 ↔ y5

Rule:

I v1 enters and v2 leaves dual dictionary.v c1 leaves and v c2 enters primal dictionary.

I v c1 enters and v c2 leaves primal dictionary.v1 leaves and v2 enters dual dictionary.

Lecture 8 Slide# 22

Primal Dual Example

y2 .5 −y1 −.5y3 −.5y4y5 3 −5y1 +y3 −2y4−w 4 −12y1 +3y3 −4y4

x3 12 +x4 +5x2x5 −3 +.5x4 −x2x1 4 +.5x4 +2x2z −4 −.5x4 −3x2

Lecture 8 Slide# 23

Primal Dual Examples

y3 1 −2y1 −2y2 −y4y5 4 −7y1 −2y2 −3y4−w 7 −18y1 −6y2 −7y4

Can you write down the primal dictionary??

Lecture 8 Slide# 24

Dual Simplex

I Easy transformation of primal to dual and vice versa.

I Entering variable for dual ⇒ complementary variable leaves primaldictionary.

I Leaving variable ⇒ complementary variable enters primal.

I Unbounded dual ⇒ infeasible primal (and vice versa).

Lecture 8 Slide# 25

Why Dual Simplex?

1. When primal objective coefficients are all non-positive.Avoid need for an initialization phase.

2. When primal has more variables than constraints.Simplex usually observed to be sensitive to number of variables.

3. As an alternative to the initialization phase!

Lecture 8 Slide# 26

Initialization using Dual Simplex

Lecture 8 Slide# 27

Example

maximize −x1 +4x2subjectto −2x1 −x2 ≤ 4

−2x1 +4x2 ≤ −8−x1 +3x2 ≤ −7

x1, x2 ≥ 0

Dual:max −4y1 +8y2 +7y3

2y1 +2y2 +y3 ≤ 1y1 −4y2 −3y3 ≤ −4

y1, y2, y3 ≥ 0

Problem: Neither primal nor dual initial dictionary is feasible.

Lecture 8 Slide# 28

Initialization using Dual

Idea: Change of primal objective function can make dual initial dictionaryfeasible!!

−x1 + 4x2 → −x1 − x2

Dual:max −4y1 +8y2 +7y3

2y1 +2y2 +y3 ≤ 1y1 −4y2 −3y3 ≤ 1

y1, y2, y3 ≥ 0

The change ensures that dual starts at a feasible dictionary.

Lecture 8 Slide# 29

Initialization Using Dual

I Dualize given problem.

I If dual starting dictionary is infeasible, then change primal objectiveto make initial dual dictionary feasible.

I Perform dual simplex until a dual optimal dictionary is reached.

I Change final dual dictionary into corresponding primal dictionary.

I Replace the objective function, taking into account new basisvariables.

Lecture 8 Slide# 30