Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel...

Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions

Ashish Goel

Department of Management Science and Engineering

Stanford University

Stanford, CA 94305, U.S.A.

http://www.stanford.edu/~ashishg

http://www.stanford.edu/class/msande211/

1Lecture #3; Based on slides by Yinyu Ye

LP Feasible Region in the Inequality Form

2

n

jmjmj

Tm

n

jjj

T

n

jjj

T

bxaxa

bxaxa

bxaxa

1

1222

1111

...

x simultaneously satisfy

This is the intersection of the m Half-spaces, and it isa convex (polyhedron) set

Lecture #3; Based on slides by Yinyu Ye

Corner or Extreme Points

5

Convex Hull:

Extreme Points: A point in a set that does not belong to the hull (convex combination) of the other pointsFor LP in inequality form, an extreme point is the intersection of n hyperplanes associated with the inequality constraints.


Feasible Direction

6

Direction: A direction is notated by a vector dIt is always associated with a “location” or point xTogether a point and a direction define a ray:

x + ϵd, for all ϵ > 0where d and kd are considered the same direction for all k > 0

Feasible Direction: A direction, d, is said to be “feasible” (relative to a given feasible point x) if x + ϵd is feasible for some ϵ> 0

For LP, all feasible directions at a feasible point form a convex (cone) set


Extreme Feasible Direction

7

A feasible direction d is extreme if d cannot be written as an convex combination of other feasible directions

Interior Point is a point x where every direction is feasible


LP Problem in Inequality Form

8

n

jmjmj

Tm

n

jjj

T

n

jjj

T

n

jjj

T

bxaxa

bxaxa

bxaxa

xcxc

1

1222

1111

1

...

s.t.

max


9

Recall the Production Problem

Objective contour

c

00

10

5.1

10

00 s.t.

2 max

21

21

21

21

21

21

xx

xx

xx

xx

xx

xx


Basic Theorems of Linear Programming

10

All LP problems fall into one of three cases:•Problem is infeasible: Feasible region is empty.

•Problem is unbounded: Feasible region is unbounded towards the optimizing direction.•Problem is feasible and bounded; and in this case:

– there exists an optimal solution or optimizer.– There may be a unique optimizer or multiple optimizers.– All optimizers form a convex set and they are on a face of

the feasible region.

– There is always at least one corner (extreme) optimizer if the feasible region has a corner point.

Moreover, Local optimality implies global optimality


Sketch Proof of Local Optimality Implies Global

11

(P) minimize f (x)subject to x ∈ Ω ⊂ Rn,

is a Convex Optimization problem if f (x) is a convex function over a convex feasible region Ω.

Proof by contradiction. Suppose x’ is a local minimizer but not a global minimizer x∗, that is, x’ ∈ Ω and x ∗ ∈ Ω but f (x∗) < f (x’). Now the convex combination point αx’ + (1 − α)x ∗must be feasible (why?), and

f (αx’+ (1 − α)x∗) ≤ αf (x’)+ (1 − α)f (x∗) < f (x’)

for any 0 ≤ α < 1. This contradicts the local optimality as α can be arbitrarily close to 1 so that αx’+ (1 − α)x ∗ can be arbitrarily close to x’.


12

00

10

5.1

10

00 s.t.

2 max

21

21

21

21

21

21

xx

xx

xx

xx

xx

xx

Optimality Certification of the Production Problem

a4

a5

a1

a2a3

a2

a3

a4

Objective contour

c


13

Feasible Directions at a Corner

a4

a5

a1

a2a3

a2

a3

a4

Objective contour

c


How to Certify a Corner being an Optimizer

14

• Every feasible direction at the corner is a worsening (descent in this case) direction, that is, cTd ≤0 in this case.

• Or, equivalently, c is a conic combination of the normal directions at the corner point, that is, there are multipliers α1 ≥0 and α2 ≥0, such as c= α1 ai1+ α2 ai2, in the 2-dimensional case.

• In the n-dimensional case: c= α1 ai1+…+ αn ain

where ai1,…, ain are the normal directions associated with the corner point.


LP in Standard (Equality) Form

15

0

...

s.t.

min

1

1222

1111

1

x

bxaxa

bxaxa

bxaxa

xcx c

n

jmjmjm

n

jjj

n

jjj

n

jjj

T

0

, s.t.

min

x

bAx

xcT


Reduction to Standard Form

16

•max cTx to min – cTx

•Eliminating ”free” variables: substitute with the

difference of two nonnegative variables

x := xl − xll, (xl, xll) ≥ 0.

•Eliminating inequalities: add a slack variableaT x ≤ b = ⇒ aT x + s = b, s ≥ 0

aT x ≥ b = ⇒ aT x − s = b, s ≥ 0


Reduction of the Production Problem

17

min -x1 -2x2

s.t. x1 +x3 = 1

x2 +x4 = 1

x1 +x2 +x5 = 1.5

(x1, x2, x3, x4, x5) ≥ 0

0

0

5.1

1

1 s.t.

2max

2

1

21

2

1

21

x

x

xx

x

x

xx

x3, x4,and x5 are called slack variables


Basic and Basic Feasible Solution

18

In the LP standard form, select m linearly independent columns, denoted by the variable index set B, from A. Solve

AB xB = b

for the dimension-m vector xB . By setting the variables, xN , of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b.

Then, x is said to be a basic solution to (LP) with respect to the basic variable set B. The variables in xB are called basic variables, those in xN are nonbasic variables, and AB is called a basis.

If a basic solution xB ≥ 0, then x is called a basic feasible solution, or BFS. Note that AB and xB follow the same index order in B.Two BFS are adjacent if they differ by exactly one basic variable.


BS of the Production Problem in Equality Form

19

x1 +x3 = 1

x2 +x4 = 1

x1 +x2 +x5 = 1.5

(x1, x2, x3, x4, x5) ≥ 0

Basis 3,4,5 1,4,5 3,4,1 3,2,5 3,4,2 1,2,3 1,2,4 1,2,5

Feasible? √ √ √ √ √

x1 , x2 0, 0 1, 0 1.5, 0 0, 1 0, 1.5 .5, 1 1, .5 1, 1

x1

x2


BFS and Corner Point Equivalence Theorem

20

Theorem Consider the feasible region in the standard

LP form. Then, a basic feasible solution and a corner

(extreme) point are equivalent; the formal is algebraic

and the latter is geometric.

Algorithmically, we need only to look at BFSs for finding

an optimizer. But still too many to look…


Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel...

Documents

Transcript of Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions Ashish Goel...