Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.
-
Upload
joanna-hancock -
Category
Documents
-
view
216 -
download
0
Transcript of Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.
![Page 1: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/1.jpg)
Basic Feasible Solutions: Recap
MS&E 211
![Page 2: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/2.jpg)
WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION
![Page 3: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/3.jpg)
![Page 4: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/4.jpg)
![Page 5: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/5.jpg)
![Page 6: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/6.jpg)
TEN STEPS TOWARDS UNDERSTANDING BASIC FEASIBLE SOLUTIONS
THESE SLIDES: ONLY INTUITIONDETAILS ALREADY COVERED
![Page 7: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/7.jpg)
Step 0: Notation
• Assume an LP in the following formMaximize cTxSubject to:
Ax ≤ b x ≥ 0
• N Variables, M constraints• U = Set of all feasible solutions
![Page 8: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/8.jpg)
Step 1: Convex Sets and Convex Combinations
• Convex set: If two points belong to the set, then any point on the line segment joining them also belongs to the set
• Convex combination: weighted average of two or more points, such that the sum of weights is 1 and all weights are non-negative– Simple example: average
![Page 9: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/9.jpg)
Convex Combination
• Given points x1, x2, …, xK , in N dimensions• For any scalars a1, a2, …, aK such that– Each ai is non-negative
– a1 + a2 + … + aK = 1
a1x1 + a2x2 + … + aKxK is a convex combination of x1, x2, …, xK
![Page 10: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/10.jpg)
Example: Convex combination of two points
![Page 11: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/11.jpg)
Exercise
• If w is a convex combination of x and y, and z is a convex combination of x and w, then must z be a convex combination of x and y?
![Page 12: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/12.jpg)
Example: Convex combination of three points
![Page 13: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/13.jpg)
Exercise
Is the set of all convex combinations of N points always convex?
![Page 14: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/14.jpg)
Exercise
Can any convex set be represented as a convex combination of N points, for finite N?
![Page 15: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/15.jpg)
Step 2: Half spaces are convex
• aTx ≤ b, aTy ≤ b
• Choose z = (x+y)/2• Then, we must have aTz ≤ b
![Page 16: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/16.jpg)
Step 3: Intersection of convex sets is convex
• Two convex sets S, T• Suppose x, y both belong to S ∩T• z = (x+y)/2
(1) z belongs to S since S is convex(2) z belongs to T since T is convex
• Hence, z belongs to S ∩ T• Implication: S∩T is convex
![Page 17: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/17.jpg)
Step 4: Feasible solutions to LP
• The set U of feasible solutions to a Linear Program is convex
• Proof: U is the intersection of Half Spaces
![Page 18: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/18.jpg)
Exercise
• Must every convex set be bounded?
![Page 19: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/19.jpg)
Step 5: Basic Feasible Solution
• x is a basic feasible solution to a LP, if
(1) x is a feasible solution
(2) There do not exist two other feasible solutions y, z such that x = (y+z)/2
ALSO known as vertex solution, extreme point solution, corner-point solution
![Page 20: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/20.jpg)
![Page 21: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/21.jpg)
Step 6: BFS and Bounded Polytopes
• If U is bounded, then any point in U is the convex combination of its basic feasible solutions
Use b1, b2, …, bK to denote the K basic feasible solutions
![Page 22: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/22.jpg)
![Page 23: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/23.jpg)
Step 7: BFS and Optimality for Bounded Polytopes
• If U is bounded and non-empty, then there exists an optimal solution that is also basic feasible
![Page 24: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/24.jpg)
Step 8: BFS and Optimality for General LPs
• If an LP has a basic feasible solution and an optimum solution, then there exists an optimal solution that is also basic feasible
• Will call these solutions “Vertex Optimal” or “Optimum Basic Feasible”
![Page 25: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/25.jpg)
Step 9: Two Implications
(1) SOLUTION OF LPs: The Simplex method which walks from BFS to BFS is correct for bounded polytopes
(2) STRUCTURAL CONSEQUENCES: The Vertex Optimal Solutions often have very interesting properties. Example: For the matching LP, every vertex optimal solution is integral
![Page 26: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/26.jpg)
![Page 27: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/27.jpg)
In Practice
• Most LP Solvers return an optimum basic feasible solution, when one exists.– Either, they use Simplex– Or, they transform the solution that they do find
to a basic feasible solution• Hence, when we solve a problem using Excel
we get an optimum basic feasible solution, when one exists.
![Page 28: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/28.jpg)
Step 10: A useful property
• At every basic feasible solution, at least N constraints are tight
• More specifically: Exactly N Linearly Independent Constraints are tight
• Basic solution: Not necessarily feasible, but exactly N Linearly Independent Constraints are tight
![Page 29: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/29.jpg)
Exercise
You are given a LP with three decision variables where some of the constraints are
0 ≤ x ≤ 10 ≤ y ≤ 10 ≤ z ≤ 1
Is it possible to add other constraints and an objective function such that the LP has an optimum solution but not an optimum basic feasible solution?
![Page 30: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/30.jpg)
LP in standard form
Minimize cTxSubject to:
Ax = b x ≥ 0
• N Variables, M constraints, assume that the M constraints are Linearly Independent
• At any BFS, at most M variables are strictly positive
![Page 31: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/31.jpg)
Recap
• A feasible solution is basic feasible if it is not the average of two other feasible solutions
• If the feasibility region U for a LP is bounded and non-empty, then there exists an optimal solution that is also basic feasible
• If an LP has a basic feasible solution and an optimum solution, then there exists an optimal solution that is also basic feasible
• Commercial LP solvers return an optimum basic feasible solution by default, where one exists
• If there are N decision variables, then a basic feasible solution has N linearly independent constraints tight
![Page 32: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/32.jpg)
LP Example: Transportation
32
Retailer 1 Retailer 2 Retailer 3 Retailer 4 SUPPLY
Warehouse 1 12 (c11) 13 4 6 500 (s1)
Warehouse 2 6 4 10 11 700 (s2)
Warehouse 3 10 9 12 14 (c34) 800 (s3)
DEMAND 400 (d1) 900 (d2) 200 (d3) 500 (d4)
jix
jdx
isx
xc
ij
ji
ij
ij
ij
i jijij
,,0
4,3,2,1 ,
3,2,1,s.t.
min
3
1
4
1
3
1
4
1
1
2
3
1
2
3
4
x11
x34
Abstract Model
![Page 33: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/33.jpg)
LP Example: Transportation, N warehouses, M retailers
33
At most M + N non-zero variables at a BFS
Can we get to M + N -1?
1
2
N
1
2
3
M
x11
xNM
![Page 34: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/34.jpg)
LP Example: Transportation, N warehouses, M retailers
34
Demand = -si on the left, dj on the rightEdges directed from left to rightCosts cij
Capacities 1
MIN COST FLOW!
Hence, BFSs are going to be integral if demands and supplies are integral
1
2
N
1
2
3
M
x11
xNM
![Page 35: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/35.jpg)
LP Example: Transportation, N warehouses, M retailers
35
MIN COST FLOW!
No cycles at BFS (else can send flow in both directions)
=> At most M+N-1 non-zero values (the maximum number of edges in a an acyclic graph on M+N vertices)
Demand = -si on the left, dj on the rightEdges directed from left to rightCosts cij
Capacities 1
1
2
N
1
2
3
M
x11
xNM
![Page 36: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/36.jpg)
Exercise
• What if the demands and supplies don’t match up? For example, supply is bigger than the demand?
![Page 37: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/37.jpg)
Exercise
• What if the demands and supplies don’t match up? For example, supply is bigger than the demand?
• Solution: Add a fake nodewith the residual demand,and add 0 cost, infinitecapacity edges from all supplynodes to this fake node
1
2
N
1
2
3
M
x11
xNM
![Page 38: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/38.jpg)
Exercise
• What if there is also a cost to disposal of excess supply?
![Page 39: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/39.jpg)
Exercise
• What if there is also a cost to disposal of excess supply?
• Solution: Add a cost on theedges from the supply nodesto the fake node
• Same idea works for excessdemand
1
2
N
1
2
3
M
x11
xNM
![Page 40: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/40.jpg)
THE SIMPLEX METHOD
![Page 41: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/41.jpg)
Basic and Basic Feasible Solution
41
In the LP standard form, select m linearly independent columns, denoted by the variable index set B, from A. Solve
AB xB = b
for the dimension-m vector xB . By setting the variables, xN , of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b.
Then, x is said to be a basic solution to (LP) with respect to the basic variable set B. The variables in xB are called basic variables, those in xN are nonbasic variables, and AB is called a basis.
If a basic solution xB ≥ 0, then x is called a basic feasible solution, or BFS. Note that AB and xB follow the same index order in B.Two BFS are adjacent if they differ by exactly one basic variable.
![Page 42: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/42.jpg)
Simplex MethodGeorge B. Dantzig’s Simplex Method for linear programming stands as one of the most significant algorithmic achievements of the 20th century. It is now over 60 years old and still going strong.
The basic idea of the simplex method to confine the search to corner points of the feasible region (of which there are only finitely many) in a most intelligent way, so the objective always improves
The key for the simplex method is to make computers see corner points; and the key for interior-point methods is to stay in the interior of the feasible region.
42
x1
x2
![Page 43: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/43.jpg)
From Geometry to Algebra
•How to make computer recognize a corner point? BFS
•How to make computer terminate and declare optimality?
•How to make computer identify a better neighboring corner?
43
![Page 44: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/44.jpg)
Feasible Directions at a BFS and Optimality Test
44
Recall at a BFS: AB xB +ANxN = b, and xB > 0 and xN = 0 .
Thus the feasible directions, d, are the ones to satisfyAB dB +ANdN = 0, dN ≥ 0.
For the BFS to be optimal, any feasible direction must be an ascent direction, that is,
cTd= cTB dB + cT
NdN ≥ 0.
From dB = -(AB )-1ANdN, we must have for all dN ≥ 0,
cTd= -cTB (AB )-1ANdN +cT
NdN =(cTN - cT
B (AB )-1AN) dN ≥ 0
Thus, (cTN - cT
B (AB )-1AN) ≥ 0 is necessary and sufficient. It is called the reduced cost vector for nonbasic variables.
![Page 45: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/45.jpg)
Computing the Reduced Cost Vector
45
We compute shadow prices, yT = cT
B (AB )-1 , or yT AB = cT
B, by solving a system of linear equations.
Then we compute rT=cT-yTA, where rN is the reduced cost vector for nonbasic variables (and rB=0 always).
If one of rN is negative, then an improving feasible direction is found by increasing the corresponding nonbasic variable value
Increase along this direction till one of the basic variables becomes 0 and hence non-basic
We are left with
The process will always converge and produce an optimal solution if one exists (special care for unbounded optimum and when two basic variables become 0 at the same time)
T
![Page 46: Basic Feasible Solutions: Recap MS&E 211. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649cf95503460f949c9fa6/html5/thumbnails/46.jpg)
Summary
• The theory of Basic Feasible Solutions leads to a solution method
• The Simplex algorithm is one of the most influential and practical algorithms of all time
• However, we will not test or assign problems on the Simplex method in this class (a testament to the fact that this method has been so successful that we can use it as a basic technology)