OR-1 2009 1

Relation between (P) & (D)

Dual of dual is primal.

Dual of a standard LP can be expressed as

max

m

iii yb

1)(

m

ijiij njcya

1,...,1,)(s. t.

miyi ,...,2,1,0

min

n

jjj xc

1)(

n

jijij mibxa

1,...,1,)(s. t.

njx j ,...,2,1,0

and its dual is

max

n

jjj xc

1

n

jijij mibxa

1,...,1,s. t.

njx j ,...,2,1,0

OR-1 2009 2

Hence strong duality theorem can be extended to :

Primal LP has an optimal solution Dual LP has an optimal solution.

Possible status between (P) and (D) (Important)

optimal solution

Infeasible Unbounded

Optimal solution

O X X

Infeasible X O O

Unbounded X O X

(D)

(P)

OR-1 2009 3

Complementary Slackness Theorem

Let x* = (x1*, …, xn

*) be (P) feasible and y* = (y1*, …, ym

*) be (D)

feasible solutions. Then x* and y* are optimal solutions to (P) and (D) respectively if and only if

Note that the conditions can be stated as follows:

miyx

njyx

iin

jmj

,...,1,0

,...,1,0

**

**

mibxxaxabx

njcyyayacy

iinnj jij

nj jijiin

mi

mi jjmiijiijjjm

,...,1),(,

,...,1),(

11

1 1

where

njxorcyami jjiij ,...,2,101

** every for both) (or

miyorbxanj iijij ,...,2,101

** every for both) (or

OR-1 2009 4

The conditions say that if one of the primal (dual) variable is positive, then the other corresponding dual (primal) variable must be 0, i.e. at least one of the corresponding pair of primal, dual variables must be 0 in an optimal solution. (primal, dual variables must be feasible, respectively)

OR-1 2009 5

ex)

3532

55835

13

0354

74321

64321

54321

4321

xxxxx

xxxxx

xxxxx

xxxxz

1112195

52

143542

296112

76531

75431

75321

7531

xxxxx

xxxxx

xxxxx

xxxxz

0) 2, 0, 1, 6, 0, 11,

0) 1, 0, 5, 0, 14,

(

,0(

*

*

y

x

OR-1 2009 6

Pf of complementary slackness theorem)

Suppose x* is a feas. solution to (P) and y* is a feas. solution to (D).

From weak duality, we know that

) Suppose x*, y* are optimal solutions to (P) and (D) respectively, then from strong duality theorem, we have

Since x*, y* are feasible solutions to (P) and (D)

Hence we have sum of nonnegative terms which is equal to 0. It implies that each term in the summation is 0.

So we have

nj

mi ii

mi

nj ijij

nj

mi jiijjj ybyxaxyaxc1 1

*1 1

**1 1

*** )()(

nj

mi iijj ybxc1 1

**

nj

nj

mi jiijjj xyaxc1 1 1

*** )( nj j

mi iijj xyac1

*1

* 0)(

mi jiijj xyac 1

** 0,0

njyx jmj ,...,1,0**

mi

nj

mi iiniiijij miyxybyxa1 1 1

***** ,...,1,0)( from Similarly,

OR-1 2009 7

) From weak duality relation, nj

nj

mi jiijjj xyaxc1 1 1

*** )(

nj j

mi iijj xyac1

*1

* 0)(

nj

mi jiijjjjm xyacjxy 1 1

**** 0)(,0 Since

nj

mi jiij

njjj xyaxc1 1

**1

* )(

mi

nj

mi iiijij ybyxa1 1 1

*** )( Similarly,

nj

mi iijj yxybxc1 1

**** ,, (D) and (P) to feasible are Hence

x* is optimal to (P) and y* is optimal to (D)

OR-1 2009 8

Remarks

CS conditions are necessary and sufficient conditions for optimality. But the condition that the coefficients in the z-row are nonpositive in the simplex tableau is only a sufficient condition for optimality, but not necessary.

CS optimality conditions also hold for more general forms of primal, dual pair of problems if dual is defined appropriately.

In the CS theorem, the solutions x* and y* need not be basic solutions (in equality form LP). Any primal and dual feasible solutions that satisfy the CS conditions are optimal.

CS conditions can be used to design algorithms for LP or network problems.

Most powerful form of interior point method tries to find solutions that satisfy the CS conditions with some modifications. (Logic to derive the conditions is different though)

OR-1 2009 9

Interior point method and CS conditions

0

subject to

' maximize

x

bAx

xc

0

'' subject to

' minimize

y

cAy

yb

0,

subject to

' maximize

wx

bwAx

xc

0,

''' subject to

' minimize

zy

czAy

yb

0,,,

,...,1,0

,...,1,0

'

zywx

miwy

njzx

czyA

bwAx

ii

jj

0,,,

)0(,...,1,

)0(,...,1,

'

zywx

miwy

njzx

czyA

bwAx

ii

jj

CS conditions Interior Point Method

OR-1 2009 10

Characteristics of the Interior Point MethodInterior point method finds a solution to the system iteratively with 0.

(path following method)Solve system of nonlinear equations.Newton’s method is used to find the solution.Try to find solution that satisfies the equations and positivity of the points

examined is always maintained.Strict positivity is maintained and 0 is obtained as small positive number.

Hence obtained solution is not a basic feasible solution (We do not know the basis).

To identify a basic feasible solution, a post-processing stage is needed.Although we mentioned the path following method, there are other types

of the interior point methods.

OR-1 2009 11

CS theorem may enable us to derive optimal y*, given an optimal x*. Hence can verify the optimality of x*.

Thm: A feasible solution x* is optimal to (P)

mi jjiij xcyay 1

*** 0 if that such

nj jiji bxay 1

** 0 if

and y* satisfies dual constraints and nonnegativity.

33

14..

34max

321

21

321

xxx

xxts

xxxex)

3

14

43..

3min

2

21

21

21

y

yy

yyts

yy

0,, 321 xxx0, 21 yy

Prove that x* = (0, 1/4, 13/4) is optimal to (P)

(5.22)

OR-1 2009 12

33

14..

34max

321

21

321

xxx

xxts

xxx

3

14

43..

3min

2

21

21

21

y

yy

yyts

yy

0,, 321 xxx0, 21 yy

x* = (0, 1/4, 13/4)

)0140 *4

*2

*1

*2 yyyx (

)030 *5

*2

*3 yyx (

Hence y* = (1, 3), and it satisfies dual feasibility. So it is optimal.

OR-1 2009 13

Does the system we obtained always have unique solution?

Thm: (sufficient condition for unique solution)

If x* is a nondegenerate basic feasible solution (5.22) has a unique solution y*.

pf) homework later.

OR-1 2009 14

Economic significance of dual variables Can interpret optimal dual variable as marginal value of adding

one unit of resource to the r.h.s of constraint i (interpret the constraint as limiting the availability of resource i ).

Thm: If (P) has at least one nondegenerate basic optimal solution, then

*iy

problem the for , that such ,,...,1,0 miti

max

n

jjj xc

1

n

jiijij mitbxa

1,...,1,s. t.

njx j ,...,2,1,0

(D) osolution t optimal is and (P) of valueobjective optimal is where

ith valuesolution w optimalan has**

1**

yz

tyz mi ii

pf) homework later.

OR-1 2009 15

yi* called the marginal value of the i-th resource, shadow price of the

i-th resource.

Ex) Forester’s Problem :

Forester has 100 acres of hardwood timber and $4,000.

(1) Fell the hardwood and let the area regenerate:

cost $10/acre, subsequent return $50/acre, net profit $40/acre

(2) Fell the hardwood and plant the area with pine:

cost $50/acre, subsequent return $120/acre, net profit $70/acre.

maximize 40x1 + 70x2

subject to x1 + x2 100

10x1 + 50X2 4,000

x1, x2 0

OR-1 2009 16

maximize 40x1 + 70x2

subject to x1 + x2 100

10x1 + 50X2 4,000

x1, x2 0

Optimal solution is x1* = 25, x2* = 75, value = $6,250

Optimal dual solution is y1* = 32.5, y2* = 0.75

If interest is lower than 75 cents per dollar, it is better to borrow money and invest it to the forestry.

If interest is greater than 75 cents per dollar, it is better to deposit money in the bank rather than invest it in forestry.

Note that if we invest too much money, the value of money may not remain 75 cents per dollar. Above analysis only holds for small changes of t, and the value of t such that the value of money remains 75 cents per dollar need to be examined (sensitivity analysis)