Lecture 6:

Duality Theory

Reading: Chapter 4

1

Bounds on the Objective Function

Consider the canonical max LP:

(P )

max z = c1x1 + c2x2 + . . . + cnxna11x1 + a12x2 + . . . + a1nxn ≤ b1a21x1 + a22x2 + . . . + a2nxn ≤ b2

... ...

am1x1 + am2x2 + . . . + amnxn ≤ bmx1 ≥ 0 x2 ≥ 0 . . . xn ≥ 0

or

max z = cx

Ax ≤ b

x ≥ 0

Let z∗ denote the (as yet unknown) optimal solution objec-

tive value for this LP.

Question: How can we find lower and upper bounds

on z∗?

2

lower bounds: The objective function value z for any

feasible solution x is an lower bound on z∗

upper bounds: Write (P ) in inequality max form:

(P ′)

max z = c1x1 + c2x2 + . . . + cnxna11x1 + a12x2 + . . . + a1nxn ≤ b1a21x1 + a22x2 + . . . + a2nxn ≤ b2

... ...

am1x1 + am2x2 + . . . + amnxn ≤ bm−x1 ≤ 0

−x2 ≤ 0. . .

−xn ≤ 0

We want to derive a surrogate constraint for the

objective function of the sort

c1x1 + c2x2 + . . . + cnxn ≤ zU

by using the m + n inequalities in (P ′). The rules for

doing this are:

1. We can multiply any inequality by a nonnega-

tive scalar.

2. We can add any set of inequalities together.

If we can obtain an inequality whose left-hand-side co-

efficients exactly match those of the objective function,

then the right-hand-side constant is an upper bound

on the optimal value of z.

3

Example

Consider Woody’s LP:

max z = 35x1 + 60x2 + 75x38x1 + 12x2 + 16x3 ≤ 120

15x2 + 20x3 ≤ 60

3x1 + 6x2 + 9x3 ≤ 48

x1 ≥ 0 x2 ≥ 0 x3 ≥ 0

Lower bound: the feasible solution x1 = 11, x2 = x3 = 1

has objective function value 520, which is then a lower

bound on z∗.

Upper bound: We can construct the following surrogate

constraint

15/4× (8x1 + 12x2 + 16x3 ≤ 120) ⇒ 30x1 + 45x2 + 60x3 ≤ 450

1/5× (15x2 + 20x3 ≤ 60) ⇒ 3x2 + 4x3 ≤ 12

2× (3x1 + 6x2 + 9x3 ≤ 48) ⇒ 6x1 + 12x2 + 18x3 ≤ 96

1× (−x1 ≤ 0) ⇒ −x17× (−x3 ≤ 0) ⇒ −7x3 ≤ 0

Adding rows: 35x1 + 60x2 + 75x3 ≤ 558

The number 558 − 520 gives you a rough idea of how

close you are to finding the true value z∗.

4

The Primal and Dual LPs

greatest lower bound (primal LP): find the feasi-

ble solution with the largest objective function

value.

least upper bound (dual LP): construct the implied

constraint for the objective function having the small-

est bound on z.

5

Mathematical Formulation of the Dual LP

Let y1, . . . , ym be the multipliers of the first m rows of

(P ′), and let u1, . . . , un the multipliers of the last n

rows (corresponding to the nonnegativity constraints).

Multiplying each inequality by the appropriate multi-

plier and adding the inequalities gives the surrogate

inequality

m∑i=1

yiai1 − u1

x1+. . .+ m∑i=1

yiain − un

xn ≤m∑i=1

yibi

(In matrix form, (yA− u)x ≤ yb)

This is an objective function upper bound constraint if

(B1) The yi’s and uj’s are nonnegative,

(B2) cj =m∑i=1

yiaij − uj for each j = 1, . . . , n.

and the upper bound for z implied by this constraint ism∑i=1

yibi.

Goal: To obtain such a constraint whose right-hand-side

value is as small as possible.

6

Example

y1 × (8x1 + 12x2 + 16x3 ≤ 120)

⇒ (8y1)x1 + (12y1)x2 + (16y1)x3 ≤ 120y1y2 × (15x2 + 20x3 ≤ 60)

⇒ (15y2)x2 + (20y2)x3 ≤ 60y2y3 × (3x1 + 6x2 + 9x3 ≤ 48)

⇒ (3y3)x1 + (6y3)x2 + (9y3)x3 ≤ 48y3u1 × (−x1 ≤ 0) ⇒ −u1x1 ≤ 0

u2 × (−x2 ≤ 0) ⇒ −u2x2 ≤ 0

u3 × (−x3 ≤ 0) ⇒ −u3x3 ≤ 0

Adding these, we get the surrogate constraint:

(8y1 + 3y3 − u1)x1 + (12y1 + 15y2 + 6y3 − u2)x2

+ (16y1+20y2+9y3−u3)x3 ≤ 120y1+60y2+48y3

y and u satisfy (B1), (B2) if and only if 8y1+3y3−u1 = 35,

12y1+15y2+6y3−u2 = 60, and 16y1+20y2+9y3−u3 = 75,

or equivalently if the y’s satisfy

8y1 + 3y3 ≥ 35

12y1 + 15y2 + 6y3 ≥ 60

16y1 + 20y2 + 9y3 ≥ 75

y1 ≥ 0, y2 ≥ 0, y3 ≥ 0

and we want to choose y so as to minimize

120y1 + 60y2 + 48y3

7

Dual Linear Programs

The dual to a canonical max LP

max z = c1x1 + c2x2 + . . . + cnxna11x1 + a12x2 + . . . + a1nxn ≤ b1a21x1 + a22x2 + . . . + a2nxn ≤ b2

(P ) ... ... ... ...

am1x1 + am2x2 + . . . + amnxn ≤ bmx1 ≥ 0 x2 ≥ 0 . . . xn ≥ 0

is the canonical min LP

minw = b1y1 + b2y2 + . . . + bmyma11y1 + a21y2 + . . . + am1ym ≥ c1a12y1 + a22y2 + . . . + am2ym ≥ c2

(D) ... ... ... ...

a1ny1 + a2ny2 + . . . + amnym ≥ cny1 ≥ 0 y2 ≥ 0 . . . ym ≥ 0

8

Lemma 6.1 Solving (D) is equivalent to finding the set

of multipliers that gives us a surrogate objective con-

straint having the least upper bound for z.

Proof: If we use the matrix form of the LP:

max z = cx

Ax ≤ b

−x ≤ 0

and let y = (y1, . . . , ym) be the vector of row multipliers

and u = (u1, . . . , un) the vector of nonnegativity constraint

multipliers, then the implied equality constraint will be

yAx− ux = (yA− u)x ≤ yb.

In order that (B1) and (B2) be satisfied we need y ≥ 0,

u ≥ 0, and c = yA− u, or equivalently,

yA ≥ c

y ≥ 0

Further, to get the least upper bound on z we need to

minimize the right-hand-side term yb. Putting this together

gives LPminw = yb

yA ≥ c

y ≥ 0

which is the matrix form of (D).

9

Weak Duality Theorem

Theorem 6.1 Let x = (x1, . . . , xn) be a feasible so-

lution to (P ) with objective function value z, and let

y = (y1, . . . , ym) be a feasible solution to (D) with objec-

tive function value w. Then z ≤ w, and if z = w then

x and y are both optimal to their respective LPs.

Proof: Since we know that z provides an lower bound

on z∗ and w provides a upper bound on that same num-

ber, then clearly z ≤ w. Further, the only way that z = w

is when they are both equal to z∗, that is, x and y are

optimal to (P ) and (D), respectively.

Question: When can we actually get an x and y such that

the upper and lower bounds given above are the same?

10

A Fundamental Insight

Lemma 6.2 Suppose we solve (P ) using the Simplex Method,

starting with tableau

basis z x s rhs

xB 0 A I b

z 1 −c 0 0

where s is the vector of slack variables. A basic tableau

for (P ) corresponding to basis (xB, xN) will look like:

basis z x s rhs

xB 0 A S b

z 1 −c + yA y z0

Then

• The associated BFS x = (xB, xN) = (b, 0) is a feasible

solution for (P ) whenever b ≥ 0.

• The primal objective function value for x is z0 =

yb.

• If we let y be the dual solution value, then the slack

variables values for the dual problem will be

u = −c + yA

Therefore,

• If the objective function row is nonnegative, then y

is a feasible solution for (D).

• The dual objective function value for y is z0.

11

Fundamental Theorem of Duality

Theorem 6.2 For any primal-dual pair of linear pro-

grams, if either has an optimal solution, then both

have optimal solutions, and their optimal objective func-

tion values are equal.

Proof: Assume that (P ) has an optimal solution (the case

where (D) has an optimal solution will be shown later to

be symmetric). Then the Simplex Method will produce

optimal tableau

basis z x s rhs

xB 0 A∗ S∗ b∗

z 1 u∗ y∗ z∗0

where b∗, u∗ and y∗ are nonnegative. It follows from the

Fundamental Insight that

• The associated basic solution x∗ = (xB, xN) = (b∗, 0)

for this tableau is feasible to (P ), with objective value

z∗0 .

• The solution y∗ is feasible to (D), also with objective

value z∗0 .

By the Weak Duality Theorem, both solutions are optimal

to their respective LPs.

12

Example

The dual for Woody’s LP is

(D)

minw = 120y1 + 60y2 + 48y38y1 + 3y3 ≥ 35

12y1 + 15y2 + 6y3 ≥ 60

16y1 + 20y2 + 9y3 ≥ 75

y1 ≥ 0, y2 ≥ 0, y3 ≥ 0

Now the optimal simplex tableau for Woody’s problem is

basis z x1 x2 x3 s1 s2 s3 rhs

s2 0 0 0 −10 15/4 1 −10 30

x2 0 0 1 2 −1/4 0 2/3 2

x1 0 1 0 −1 1/2 0 −1 12

z 1 0 0 10 5/2 0 5 540

so if we set

(y1, y2, y3) = (5/2, 0, 5)

then it can be checked that the slack variables for (D) are

(u1, u2, u3) = (0, 0, 10)

Thus y = (5/2, 0, 5) is feasible to (D) and has objective

function value w = 540, the same as that for the optimal

primal solution.

13

The Symmetry of the Primal and Dual LPs

Consider the two dual LPs (in matrix form):

(P )

max z = cx

Ax ≤ b

x ≥ 0

(D)

minw = yb

yA ≥ c

y ≥ 0

Theorem 6.3 The dual of (D) is (P ).

Proof: First put (D) in canonical max form, by negating

the objective (to put it into max form) and all of the con-

straints (to put them into ≤ form):

(D′)

max −w = −yb

−yA ≤ −c

y ≥ 0

or

max z = cx

Ax ≤ b

x ≥ 0

where x = yT , c = −bT , A = −AT , and b = −cT .

Now take the dual of (D′):

min w = yb

yA ≥ c

y ≥ 0

or

min −cx

−Ax ≥ −b

x ≥ 0

which is exactly (P ).

14

The Fundamental Duality Equation

Recall that a general LP can have

• objective function either minimized or maximized

• constraints either =, ≥, or ≤• variables either ≥ 0, ≤ 0, or unrestricted.

How will this affect the dual?

To study this problem, we will formulate an important

equation relating the primal and dual objective values pre-

cisely. Let x and y be any n- and m-vectors, and set

s = b− Ax, u = yA− c.

(For canonical max and min LPs these are the slack vari-

ables.) Now multiply the first equation on the left by y

and the second equation on the right by x, so that both

equations have yAx in them. Equating these, we get

yb− ys = yAx = cx + ux

This gives the Fundamental Duality Equation

yb = cx + ux + ys

(FDE) or

w = z +n∑

j=1ujxj +

m∑i=1

yisi

15

Duality for General LPs

Let’s consider the dual program for a general LP whose

objective is to be maximized. The Fundamental Duality

Equation

w = z +n∑

j=1ujxj +

m∑i=1

yisi

applies to a general LP by setting

si = bi −n∑

j=1aijxj

uj = cj −n∑

i=1ajiyi

Now the nature of the xj’s, yi’s, si’s and uj’s will depend

upon what the nature of the variables and constraints are

for P . What we know, however, is that w comprises an

upper bound on z if

n∑j=1

ujxj +m∑i=1

yisi ≥ 0,

or more precisely,

ujxj ≥ 0, j = 1, . . . , n

yisi ≥ 0, i = 1, . . . ,m.

16

The Dual of the General LP

The dual of a general LP will always have as variables

y1, . . . , ym, one for each constraint, objective function value

w = b1y1 + . . . + bmym

and as constraints

aj1yi + . . . + ajmym

≥≤=

cj, j = 1, . . . , n.

In terms of the uj’s, this is equivalent to

uj

≥ 0

≤ 0

= 0

Now let’s see what this means in terms of (FDE) implying

w ≥ z. For the variables:

• If xj ≥ 0, then in order that ujxj ≥ 0 we must have

uj ≥ 0, that is,m∑i=1

yiaij ≥ cj.

• If xj ≤ 0, then in order that ujxj ≥ 0 we must have

uj ≤ 0, that is,m∑i=1

yiaij ≤ cj.

• If xj is unrestricted, in order that ujxj ≥ 0 we must

have uj = 0, that is,m∑i=1

yiaij = cj.

17

and for the constraints:

• If the jth constraint is∑nl=1 ailxl ≤ bi, then si ≥ 0, and

so in order that yisi ≥ 0 we must have yi ≥ 0.

• If the jth constraint is∑nl=1 ailxl ≥ bi, then si ≤ 0, and

so in order that yisi ≥ 0 we must have yi ≤ 0.

• If the jth constraint is∑nl=1 ailxl = bi, then si = 0, and

so there is no restriction on yi.

For aminimization problem, we need (FDE) to imply

w ≤ z, and so ujxj and yisi must be ≤ 0. Therefore all

of the final inequalities are opposite those given above,

namely,

• xj ≥ 0 implies∑mi=1 yiaij ≤ cj,

• xj ≤ 0 implies∑mi=1 yiaij ≥ cj,

• xj unrestricted implies∑mi=1 yiaij = cj,

• ∑nl=1 ailxl ≤ bi implies yi ≤ 0,

• ∑nl=1 ailxl ≥ bi implies yi ≥ 0,

• ∑nl=1 ailxl = bi implies yi is unrestricted.

18

Summary

ofDualConstru

ction

foraGenera

lLP

Maxim

izationPro

blem

Minim

izationPro

blem

objective:

max

z=

c 1x1+c 2x2+...+

c nxn

objective:

minw

=b 1y 1

+b 2y 2

+...+

b my m

constra

int:

ai1x1+ai2x2+...+

ainxn

≤b i

≥b i

=b i

variable:

y i

≥0

≤0

unrestricted

variable:

xj

≥0

≤0

unrestricted

constra

int:

a1jy 1

+a2jy 2

+...+

amjy m

≥c j

≤c j

=c j

Coro

llary

:Thedual

ofthedual

istheprimal.

Pro

of:

Therelationship

betweentheconstraints/dualvariablesandvariables/dualconstraints

goes

bothwaysin

theabovediagram

.

19

Example: The Dual to the Standard Form LP

The dual to the standard equality LP

max z = c1x1 + c2x2 + . . . + cnxna11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2

(PS) ... ... ...

am1x1 + am2x2 + . . . + amnxn = bmx1 ≥ 0 x2 ≥ 0 . . . xn ≥ 0.

is the inequality form min LP

minw = b1y1 + b2y2 + . . . + bmyma11y1 + a21y2 + . . . + am1ym ≥ c1a12y1 + a22y2 + . . . + am2ym ≥ c2

(DS) ... ... ... ...

a1ny1 + a2ny2 + . . . + amnym ≥ cn(yi unrestricted)

Now an optimal tableau for (PS) will look like

basis z x rhs

xB 0 S∗A S∗b

z 1 −c + y∗A y∗b

where S∗ is the inverse of the optimal basis matrix B∗,

y∗ = cB∗S∗ (no sign restriction), and −c + y∗A ≥ 0. It

follows that y∗ is optimal to (DS).

20

A General Example: The dual of the general linear

program

(GP )

max z = 17x1 − 12x2 − 25x3 + 16x48x1 + 12x2 + 16x3 + 20x4 ≤ 68

15x2 + 20x3 + 2x4 = 11

3x1 + 6x2 + 9x3 + 12x4 ≥ 18

x1 ≥ 0 x2 unres. x3 ≥ 0 x4 ≤ 0

is

(GD)

min w = 68y1 + 11y2 + 18y38y1 + 3y3 ≥ 17

12y1 15y2 + 6y3 = −12

16y1 + 20y2 + 9y3 ≥ −25

20y1 + 2y2 + 12y3 ≤ 16

y1 ≥ 0 y2 unres. y3 ≤ 0

These have optimal solutions (x∗1, x∗2, x

∗3, x

∗4) = (12, 1, 0,−2)

and (y∗1, y∗2, y

∗3) = (4,−2,−5), respectively, with common

objective value 160.

21

Complete Version of the

Fundamental Theorem of Duality

Theorem 6.4 Let (P ) and (D) be a primal-dual pair of

LPs. Then

(i) If either LP is unbounded, then the other LP

is infeasible.

(ii) If either LP is infeasible, then the other LP is

either infeasible or unbounded.

(iii) If either (P ) or (D) has an optimal solution,

then both LPs have an optimal solution, and their

objective function values are equal.

Proof: (iii) is the original FTD. (ii) follows from the orig-

inal FTD, since if, say, (P ) is infeasible, then (D) cannot

have an optimal solution, and so it must be either infeasible

or unbounded. (i) follows from the Weak Duality Theorem,

since if, say, (P ) is unbounded, then (D) cannot have a

feasible solution, since the objective function value for this

solution would immediately provide an upper bound on the

objective function value for (P ).

22

Two Economic Interpretations of Duality

Woody’s Problem

Recall Woody’s problem of maximizing the profits of mak-

ing x1 chairs, x2 tables, and x3 desks, subject to limits on

the amount of pine, mahogany, and carpenter hours

max z = 35x1 + 60x2 + 75x38x1 + 12x2 + 16x3 ≤ 120 (pine)

+ 15x2 + 20x3 ≤ 60 (mahogany)

3x1 + 6x2 + 9x3 ≤ 48 (carpenter-hours)

x1 ≥ 0 x2 ≥ 0 x3 ≥ 0

The dual to this LP is

minw = 120y1 + 60y2 + 48y38y1 + + 3y3 ≥ 35 (chairs)

12y1 15y2 + 6y3 ≥ 60 (tables)

16y1 + 20y2 + 9y3 ≥ 75 (desks)

y1 ≥ 0 y2 ≥ 0 y3 ≥ 0

23

Interpretation

A takeover artist is interested in assuming the resources

of Woody’s company — namely, pine, mahogany, and car-

penters — and selling them separately. Woody is not in-

terested in selling off his resources unless they are worth at

least as much as they were in making his three products. If

we let yi be the price paid for each unit of resource i, then

the three dual constraints calculate the worth of each prod-

uct (chairs, tables, desks) in terms of the price the takeover

artist is paying Woody, and this must be greater than the

profit Woody makes from actually making the product out

of these resources.

The objective of the takeover artist in setting prices is to

minimize the total amount paid to Woody.

Conclusion from FTD: The amount the takeover artist

pays to Woody is exactly the same as Woody’s profits would

be, were he to make the furniture himself.

24

The Transportation Problem:

A manufacturer makes a single product in factories at

m locations, and wishes to ship them to n distribution

centers. Each factory i makes si units of this product,

and each distribution center j has a demand dj for the

product, with∑i si ≥

∑j dj. Further, there is a cost

cij for shipping each unit of the product from factory

i to distribution center j. The manufacturer wishes to

determine a shipping schedule that ships from avail-

able supply to satisfy demand and has minimum total

shipping cost.

Mathematical Description:

Let xij represent the amount shipped from factory i to

distribution center j. Then the LP is

minm∑i=1

n∑j=1

cijxijn∑

j=1xij ≤ si, i = 1, . . . ,m (supplies)

m∑i=1

xij ≥ dj, j = 1, . . . , n (demands)

xij ≥ 0, i = 1, . . . ,m, j = 1, . . . , n

25

Example

An example with m = 3 and n = 2 is given below:

Dist.

Center

production

capacity

costs 1 2

1 20 25 100

Factory 2 10 14 250

3 30 15 100

demands 295 150

and the LP is

min z = 20x11 + 25x12 + 10x21 + 14x22 + 30x32 + 15x32x11 + x12 ≤ 100

x21 + x22 ≤ 250x31 + x32 ≤ 100

x11 + x21 + x31 ≥ 295x12 + x22 + x32 ≥ 150

x11 ≥ 0, x12 ≥ 0, x21 ≥ 0, x22 ≥ 0, x31 ≥ 0, x32 ≥ 0

26

The Dual LP

First we put the LP in canonical min form

minm∑i=1

n∑j=1

cijxij

−n∑

j=1xij ≥ −si, i = 1, . . . ,m (supplies)

m∑i=1

xij ≥ dj, j = 1, . . . , n (demands)

xij ≥ 0, i = 1, . . . ,m, j = 1, . . . , n

Now let pi be the dual variable corresponding to the supply

constraint at factory i, and qj the dual variable correspond-

ing to the demand constraint at distribution center j. Then

the dual LP is a canonical max LP:

maxw =n∑

j=1djqj −

m∑i=1

sipi

qj − pi ≤ cij, i = 1, . . . ,m, j = 1, . . . , n (route (i, j))

pi, qj ≥ 0, i = 1, . . . ,m, j = 1, . . . , n

For our example the dual is

maxw = 295q1 + 150q2 − 100p1 − 250p2 − 100p3q1 − p1 ≤ 20

q2 − p1 ≤ 25

q1 − p2 ≤ 10

q2 − p2 ≤ 14

q1 − p3 ≤ 30

q2 − p3 ≤ 15

pi, qj ≥ 0, i = 1, 2, j = 1, 2, 3

27

Interpretation

The marketplace will naturally make transportation ar-

rangements by decoupling the manufacturer from the

distribution centers, and having an independent trucking

company serve as the middleman between these centers.

The trucker will buy all of the product made at each man-

ufacturing center i for price pi, and then will sell all of the

product demanded by distribution center j for price qj. The

manufacturers and distribution centers are willing to accept

this arrangement so long as the net loss of value along each

route (buy-back price – selling price) does not exceed the

cost of shipping along that particular route.

The Trucker’s objective in setting prices is to maximize

profits while staying competitive with current transporta-

tion costs.

Conclusion from FTD: The profit the trucker gets from

taking over the transportation portion of the process is ex-

actly the same as the cost the manufacturer would incur by

doing it in-house.

28

Complementary Slackness

One might be surprised that the competitive pricing situa-

tions given above actually gives the same total profit/cost

values, for the following reasons:

Woody’s problem: There are two ways Woody would

benefit here.

• The takeover artist is likely to be forced to over-

charge in pricing one or more of Woody’s pieces

of furniture.

• The takeover artist must purchase all of Woody’s

resources, even if Woody does not actually use all

of that resource.

Won’t either of these situations immediately give more

profit to Woody than he makes producing the furniture

himself?

Transportation problem:

• The trucker, in setting only prices at the source and

destination points, will in all likelihood be forced to

undercharge along certain of the routes.

• The trucker buys all of the goods available at the

factory and sells only goods demanded at the de-

mand point, even if these are not what the trucker

was actually going to pick up or drop off there.

29

The Complementarity Slackness Conditions

Let’s consider (FDE) for feasible solutions x and y to a

canonical dual pair of LPs:

w = z +n∑

j=1ujxj +

m∑i=1

yisi

where x and y are feasible to (P ) and (D) with objective

function values z and w, respectively, and s = b−Ax and

u = c − yA. Since ujxj and yisi will always be nonnega-

tive, then the following four statements are equivalent:

CSC1:n∑

j=1ujxj +

m∑i=1

yisi = 0

CSC2: ujxj = 0 for every j = 1, . . . , n and

yisi = 0 for every i = 1, . . . ,m

CSC3: For each j = 1, . . . , n either xj = 0 or

uj = 0 (or both), and for each i = 1, . . . ,m ei-

ther yi = 0 or sj = 0 (or both).

CSC4: For each j = 1, . . . , n either xj = 0 or∑mi=1 aijyi = cj (or both), and for each i = 1, . . . ,m

either yi = 0 or∑mj=1 aijxj = bi (or both).

A pair of solutions satisfying any of the above criteria is

called complementary.

30

Example: For (SP ) and (DP ) given above, the opti-

mal primal and dual solutions x∗ = (12, 2, 0) and

y∗ = (5/2, 0, 5) satisfy

j 1 2 3 1 2 3 i

x∗j 12 2 0 0 30 0 bi −∑nj=1 aijx

∗j

cj −∑mi=1 y

∗i aij 0 0 10 5/2 0 5 y∗i

and so these solutions are complementary.

The Complementary Slackness Theorem

Theorem 6.5 A pair x∗ and y∗ are optimal to (P ) and

(D) if and only if

FP: x∗ is feasible to (P ),

FD: y∗ is feasible to (D),

CSC: x∗ and y∗ are complementary.

Proof: Let z∗ = cx∗, w∗ = y∗b, and s∗ = c − y∗A.

From the Fundamental Theorem of Duality, we get that x∗

and y∗ are optimal iff (FP ) and (FD) holds and z∗ =

w∗, and from (FDE) we get that z∗ = w∗ if and only if∑nj=1 u

∗jx

∗j+

∑mi=1 y

∗i s

∗i = 0, that is, x∗ and y∗ satisfy (CSC).

31

Complementary Slackness for General LPs

The Complementary Slackness Conditions for general LPs

follows by exactly the same argument as for canonical

LPs. This manifests itself as follows:

• Complementary slackness conditions only occur be-

tween inequality constraints and their paired

restricted variables.

• Two solutions x∗ and y∗ are complementary if for

every “restricted” pair of inequalities (this includes

≥ 0 and ≤ 0 constraints), at least one of the

inequalities is met at equality.

Example: For the general pair of LPs (GP ) and (GD)

given above, the following five conditions constitute

the CSC conditions for this LP:

Either 8x1 + 12x2 + 16x3 + 20x4 = 68 or y1 = 0

Either 3x1 + 6x2 + 9x3 + 12x4 = 18 or y3 = 0

Either 8y1 + 3y3 = 17 or x1 = 0

Either 16y1 + 20y2 + 9y3 = −25 or x3 = 0

Either 20y1 + 2y2 + 12y3 = 16 or x4 = 0

(no conditions for 2nd const. (P )/y2 and 2nd const. (D)/x2).

It can be checked that the optimal solutions (x∗1, x∗2, x

∗3, x

∗4) =

(12, 1, 0,−2) and (y∗1, y∗2, y

∗3) = (4,−2,−5) satisfy this.

32

Economic Interpretation of the

Complementary Slackness Theorem

Woody’s Problem:

• Whenever the takeover artist overcharges for the

resources of a particular piece of furniture, Woody

would be making none of that particular piece of

that furniture.

• Whenever Woody does not use up all of a par-

ticular resource in his solution, then the takeover

artist pays nothing for that particular resource.

Transportation problem:

• Whenever the trucker undercharges the actual

cost along any route, the manufacturer would ship

no goods along that route.

• Whenever the manufacturer does not ship all

of her goods out of a factory, or whenever she

oversupplies a distribution center, the price set

at that point by the trucker is 0.

33

A Physical Interpretation of

Duality/Complementarity

Consider the two variable version of Woody’s problem,

in inequality max form:

max z = 35x1 + 60x2(W1) 8x1 + 12x2 ≤ 120

(W2) 15x2 ≤ 60

(W3) 3x1 + 6x2 ≤ 48

(W4) −x1 ≤ 0

(W5) −x2 ≤ 0

Suppose the feasible region represents a box with walls

(W1), (W2), (W3), (W4), (W5) in which lies a ball

bearing. A magnet applies a uniform force on the

plane represented by the vector

35

60

. The ball bear-ing is allowed to roll around the box until it comes to

rest at some point of equilibrium.

Physical property of equilibrium point:

net force =

sum of all force vectors

applied to the ball bearing

= 0

34

Forces applied to the ball:

1. the force of the magnet

2. the force applied by — and perpendicular to — the

walls on which the ball bearing rests.

The force applied by the magnet is represented by vec-

tor

35

60

. The force applied by a wall is propor-

tional to the vector perpendicular to the wall and

facing into the region. These vectors are

(W1) :

−8

−12

(W2) :

0

−15

(W3) :

−3

−6

(W4) :

1

0

(W5) :

0

1

.Let f1, . . . , f5 be the proportion of force actually ap-

plied by each of the five walls, in terms of themultiple

of the appropriate vector above.

35

Note: Force is only applied by a wall on which the

ball bearing rests, i.e., fi = 0 whenever the ball does

not rest on that constraint.

Equilibrium of forces equation:

35

60

+ −8

−12

f1+ 0

−15

f2+ −3

−6

f3+ 1

0

f4+ 0

1

f5 = 0

0

Moving the final five columns to the other side of the

equation and putting into matrix form, we have

8 0 3 −1 0

12 15 6 0 −1

f1f2f3f4f5

=

35

60

The proportions f1, . . . , f5 must of course be nonnega-

tive, and the conditions of force being applied only by

touching walls can be written as

f1 = 0 whenever 8x1 + 12x2 < 120

f2 = 0 whenever 15x2 < 60

f3 = 0 whenever 3x1 + 6x2 < 48

f4 = 0 whenever x1 > 0

f5 = 0 whenever x2 > 0

This is exactly the feasibility and complementarity con-

ditions (with y1 = f1, y2 = f2, y3 = f3, y4 = f4, and

y5 = f5) for the primal-dual pair.

36

Using Duality and Complementarity to

Solve LPs

By the Complementary Slackness Theorem we know that

we have solved an LP if we can find x∗ and y∗ satisfying

(FP ), (FD), and (CSC). Most LP methods proceed

by maintaining solutions x and y satisfying two of the

above three conditions, and then modify them until

they eventually satisfy the third condition, and are

hence optimal.

Example 1: the Phase II Simplex Method

For any tableau, let x be the current bfs, and let y be

read off from the objective row slack columns. Then x

and y are complementary (Why?) but y is infeasi-

ble precisely where its objective function row elements

are negative. Thus the Phase II simplex method al-

ways maintains (FP) and (CSC), and stops at optimal-

ity precisely when y becomes feasible, that is, (FD) is

satisfied.

Example 2: the Interior Point Method.

Here we maintain x and y satisfying (FP) and (FD)

but not (CSC). When enough iterations are performed

solutions x∗ and y∗ are found that satisfy all three

conditions, and hence are optimal.

The details of this method will be presented later.

37

Example 3: The Dual Simplex Method

The Dual Simplex Method maintains a basic tableau

having the objective function row nonnegative,

that is, y feasible, and pivots until x becomes feasible,

at which time we again have an optimal tableau. Thus

the Dual Simplex Method maintains (FD) and (CSC),

and stops when (FP) is satisfied.

Example: Consider solving the dual toWoody’s (3-variable)

LP. The initial tableau for this problem is

−z x1 x2 x3 x4 x5 x6 rhs

0 8 0 3 −1 0 0 35

0 12 15 6 0 −1 0 60

0 16 20 9 0 0 −1 75

1 120 60 48 0 0 0 0

By negating the rows, we get the basic tableau

basis −z x1 x2 x3 x4 x5 x6 rhs

x4 0 −8 0 −3 1 0 0 −35

x5 0 −12 −15 −6 0 1 0 −60

x6 0 −16 −20 −9 0 0 1 −75

−z 1 120 60 48 0 0 0 0

Although this is not a feasible tableau, the bottom row

is nonnegative, and so the dual solution — which can

be read off from the reduced costs in the slack rows as

(y1, y2, y3) = (0, 0, 0) — satisfies (FD) (and is feasible

to Woody’s primal LP!).

38

Outline of the Dual Simplex Method

Steps of a simplex pivot: starting with a given basic

dual feasible tableau:

1. First choose a pivot row i whose basic (leav-

ing) variable xBiviolates primal feasibility. That is,

choose a pivot row i with negative right-hand-

side value bi. (Heuristic best choice: Choose

row with most negative right-hand-side coeffi-

cient.)

2. Determine the pivot column j having the prop-

erty that a pivot on that entry aij will preserve

the nonnegativity of the objective func-

tion row. In particular, the pivot element aij must

be negative and satisfy

cj/aij = min{|cs/ais| : s such that ais < 0}.

3. Replace the leaving variable by the entering vari-

able in the basis, compute the associated basic fea-

sible tableau for the new basis by pivoting on the

associated tableau entry, and go back to Step 1.

39

This results in a new tableau whose dual solution re-

mains feasible, and whose associated objective function

value is no greater than that of the previous

tableau — and strictly less if the minimum ratio

is not 0 (dual nondegeneracy).

Stopping Rules:

No negative bi values: STOP, current tableau is

primal feasible, and hence the primal and dual

solutions are optimal.

No negative aij entry in the ith row: STOP, LP

is infeasible (Farkas’ Lemma alternative solution

exists).

Note: There is a Phase I version of the Dual Simplex

Method, and also rules to prevent cycling, but we will

not present these here.

40

Dual Simplex Applied to Woody’s Dual

Problem

basis −z x1 x2 x3 x4 x5 x6 rhs

x4 0 −8 0 −3 1 0 0 −35x5 0 −12 −15 −6 0 1 0 −60x6 0 −16 −20 −9 0 0 1 −75

−z 1 120 60 48 0 0 0 0

x4 0 −8 0 −3 1 0 0 −35x5 0 0 0 3/4 0 1 −3/4 −15/4x2 0 4/5 1 9/20 0 0 −1/20 15/4

−z 1 72 0 21 0 0 3 −225

x3 0 8/3 0 1 −1/3 0 0 35/3x5 0 −2 0 0 1/4 1 −3/4 −25/2x2 0 −2/5 1 0 3/20 0 −1/20 −3/2

−z 1 16 0 0 7 0 3 −470

x3 0 8/3 0 1 −1/3 0 0 35/3x6 0 8/3 0 0 −1/3 −4/3 1 50/3x2 0 −4/15 1 0 2/15 −1/15 0 −2/3

−z 1 8 0 0 8 4 0 −520

x3 0 0 10 1 1 −2/3 0 5x6 0 0 10 0 1 −2 1 10x1 0 1 −15/4 0 −1/2 1/4 0 5/2

−z 1 0 30 0 12 2 0 −540

Note: The numbers in the tableau are exactly the

same as if the dual problem were being solved using

the Primal Simplex Method.

41