Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms...
-
Upload
stephanie-stephany-logan -
Category
Documents
-
view
243 -
download
3
Transcript of Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms...
![Page 1: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/1.jpg)
Discrete Optimization Lecture #3
2008/3/4 1Shi-Chung Chang, NTUEE, GIIE, GICE
Last Time1. Algorithms and Complexity
» Problems, algorithms, and complexity» Polynomial time algorithms» Intractability» NP-complete problems
2. Basic Properties of Linear Programming» Forms of LP» Basic feasible solutions» Geometry of LP
3. The Revised Simplex Method
![Page 2: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/2.jpg)
Today1. The R evised Simplex Algorithm
» Basics» The algorithm» Getting an Initial Feasible Solution
2. Computational Aspects» Updating the basis» Convergence and Degeneracy» The Revised Simplex Method
3. Duality» Dual LP» The Duality Theorem» Complementary Slackness» The Dual Simplex Algorithm
2008/3/4 2Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 3: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/3.jpg)
Reading Assignments:1. Sections 3.1-3.3, 4.1-4.2 of [PaS 8 3]2. Sections 4.1-4.5 of [Lue 8 4]
Next Time: sections 4.3,4.4 and Chapter 5 of [PaS 83]
2008/3/4 3Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 4: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/4.jpg)
§2.2 The Revised Simplex MethodThe simplex method for LP : G.B. Dantzig 1951Key idea:
Phase1:find a bfs of
Note that an LP may not have a solution
Example min s.t.
0x
bxA
21 xx
0,
42
21
21
xx
xx
2008/3/4 4Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 5: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/5.jpg)
Phase 2Allow one of the zero components of the bfs to become positive and force one of the original positive components to become
zero.=> How to pick “entering” and “leaving” component Cost
Traditional form of the simplex method: Tableau=> read by yourself
Here we consider matrix form for conciseness of presentation and later developments.
xcT
2008/3/4 5Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 6: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/6.jpg)
SLP: s.t.
Assume P(A)=m.Partition A=[B:D] B: m linearly indep. columns of A (Assume the first m
cols.)
(SLP)
subject to (3.1)
xcmin
0
x
bxA
bx
xDB
A
B
:
mn
D
m
B
Rx
Rx
ZxCxC DDBB min
0,0
DB
DB
xx
bxDxB
2008/3/4 6Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 7: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/7.jpg)
If and a basic solutionWhen from (3.1)
(note that may not ≥0 ) (3.2)
Substituting(3.2) into the cost function
(3.3)
Define (3.4) as the relative cost vector=> To minimize Z, we need only adjust
Q: How?
0Dx bBx B
1
,0Dx
BxDB xDBbBx 11
DBDBDDDBxDBCCbBCxCxDBbBCZ 1111
DBccrBDD
1
Dx
2008/3/4 7Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 8: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/8.jpg)
Example
subject to =2 + =5
+ =6
In matrix form.
32133min xxx
4321 2 xxxx
32132 xxx 5
x321
22 xxx 6
x6,,1,0 ix
i
6100122
5010321
2001112654321 baaaaaa
000313 c
0B
0
Bc
2008/3/4 8Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 9: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/9.jpg)
0
0
0
0
0
0
1
1
1
)( 100BCB
6
5
2
6
5
4
0
x
x
x
x B
0
0
0
3
2
1
0
x
x
x
xD
2008/3/4 9Shi-Chung Chang, NTUEE, GIIE, GICE
3130 Dcr D
![Page 10: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/10.jpg)
Define the vector of simplex multipliers
Compute in two steps:
Assume that is non degenerate If is optimal and it does not pay to increase
If then the cost Z can possible be decreased by increasing
1 BcB
'r
Dcr
cB
D
B
D
B
x
xx
xr ,0
,0Dxr 0
Dx
)0)(( iBx
,0 kr
kmx
2008/3/4 10Shi-Chung Chang, NTUEE, GIIE, GICE
Dx
![Page 11: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/11.jpg)
Q: (1) any limitations on increasing ? (2) If there are more than one component of r < 0 => which should we change to increase?
Simplex method chooses one entering variable, Normally one with the most negative
Q: which component should leave?the first one to reach 0 => satisfied again. bxA
kr
kmx
2008/3/4 11Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 12: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/12.jpg)
If we consider bringing into the basis, i.e. , as the entering variable
=>
=>
1x
362
55
122
161
151
141
xxx
xxx
xxx
4
4
1
4,4,0,1
6
5
1
1
6541
x
x
x
x
xxxx
B 0
4
3
2
1
x
x
x
x D
2008/3/4 12Shi-Chung Chang, NTUEE, GIIE, GICE
1a
![Page 13: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/13.jpg)
Mathematically, let be the new value vector of=>
=>
The ith component of is zero when
i.e. when
Bx
Bx
BkkB
kBb
xbBxaBx
bxaxB
11
yxxbBx
xbByxx
kBB
BBB
.
.1
1
Bx iikbByx 1
i
Bi
i
i
k y
x
y
bBx
1
2008/3/4 13Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 14: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/14.jpg)
To maintain feasibility, the smallest of these ratios for i where > 0 determines how large can be. Say, it happens at l th components of
The new bfs has and
leaves the basis while enters
iyk
x
i
Bi
i
l
Bl
y
x
ymi
y
x
01
min
x 0
lx 0
kx
la ka
0Bx
R earranging , w e can get x
2008/3/4 14Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 15: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/15.jpg)
Revised Simplex Algorithm( one iteration)Step 1: Giving is the basis B such thatStep 2: Solve to get Step 3 : Calculate If r ≥ 0 => optimal solution achieved, STOP
Find K = arg
Step 4 : Solve for y Step 5 : Find
If , => unbounded solution, STOPStep 6 : Update the basic solution
Step 7 : Update the basis Return to step 2
01 bBx B
BcB Dcr D
iri
min
kayB
i
Bi
il
Bl
y
x
ymiy
x
0,1
min
k
iii
x
kiyxx ,
0i
y
2008/3/4 15Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 16: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/16.jpg)
How to Update BasisNote that in the simplex algorithm we need to solve
(step 2) (step 4)
Where B’s differ by only one column between any two subsequent iterations.=> How to calculate B’s efficiently?
B
T CB kayB
2008/3/4 16Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 17: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/17.jpg)
Recall
(rank one update)
Tllkoldkloldnew eaaBaaBB )( col.col.
11
1
1111 )(
)(1
)(
oldl
Tllkold
lkold
Tl
oldTllkold
oldnew By
eaaBI
aaBe
BeaaBBB
Tll
ll
Tl ee
yyey
IE1
2008/3/4 17Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 18: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/18.jpg)
llold
kold
eaB
yaB
1
1Note:
=>
1
0
010
01
001
2
1
l
m
l
l
l
yy
y
yy
yy
11 oldnew
EBB Product Form of the inverse(PFI)
2008/3/4 18Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 19: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/19.jpg)
Some Computational AspectsLet be the elementary matrices after K pivot iterations.
=>
• For large-scale problems, store ‘s as vector and update and y as followers:
kEEE ,,,
21
1
011
1
BEEEBkkk
iE
kk
kkB
T
aBEEy
BEEEc1
01
1
011)(
2008/3/4 19Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 20: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/20.jpg)
• What if is small in pivoting? Use LU or QR decompositions in computing If B is the current basis
Note that
ly
1B
kmllnew
oldoldoldm
aaaaaaB
BULaaaB
1121
21
newoldBLH 1
koldmll aLuuuuu 1
1121
=> upper Hessenberg
2008/3/4 20Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 21: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/21.jpg)
Using a sequence of elimination steps on H:
• Store • QR-decomposition
Do givens transformation on H
• The simplex method is theoretically an exponential algorithm In practice, 2(n + m) iterations, i.e., a linear function of (n + m)
newmoldnew
emnew
UMMLB
HMMU11
1
1
11
1
oldemnew LMML
)( 111 kmeenew aaaaaB
HBQ newT
old
1
1
meoldnew
newTe
Tm
QQQQ
RHQQ
m
n
2008/3/4 21Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 22: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/22.jpg)
Convergence and DegeneracyConvergence:
If the objective function value strictly decreases after every iteration, the simplex algorithm never repeats a basis and it converges in a finite number of iterations.
why?
Degeneracy:If for a such that
=> x is a degenerate
ixbfs ,
0)()( 1
iiB bBx
2008/3/4 22Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 23: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/23.jpg)
Property:in the pivoting step,
=> i.e. no change in cost.
=> Problem: cycling among degenerate=> no guarantee of convergence
=> need an anti-cycling algorithm
oldioldnewZYZZ
bfses
0)(
i
iB
y
x
2008/3/4 23Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 24: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/24.jpg)
How to get an initial feasible solution?
(p)
(PH1)
subject to
0
x
bxAmn
m
iii
zy1
min
0
0
0
z
y
x
bzIyIxAmm
mRzy ,
2008/3/4 24Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 25: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/25.jpg)
• (P) has a feasible solution in (PH1)Q: • can solve (PH1) using revised simplex with
as the initial feasible solution
izyii
,0,0
,0x
2008/3/4 25Shi-Chung Chang, NTUEE, GIIE, GICE
otherwise.
0 if
0ii
i
bby
otherwise.
0 if
0ii
i
bbz
![Page 26: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/26.jpg)
Solving S-P by the Revised Simplex
Subject to
* Flow balance eq. corresponding to * is dropped due to redundancy.
xxmix
52213312
0
0
0
0
1
111
111
111
111
11
4
3
2
1
)(0
8
7
6
5
4
3
2
1
87654321
x
x
x
x
x
x
x
x
eeeeeeees
8,,1,0 jxj
2008/3/4 26Shi-Chung Chang, NTUEE, GIIE, GICE
![Page 27: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/27.jpg)
Chapter 3 Duality§ 3 .1 Dual LP
Consider (SLP)
Subject to
2008/3/4 27Shi-Chung Chang, NTUEE, GIIE, GICE
minc x
x
0
,
.
Ax b
x
![Page 28: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/28.jpg)
Dual Viewpoint
• From the optimality condition, if optimal solution to SLP => corresponding partition of A matrix
such that
*x[ *: D*] = B A
(3.5)
where *' (3.6)
' ' 1* *
' 1*
( * ) 0
*
D BD
B
r c c B D
c B
Note that in Simplex Method, we have' (3.7)
where [ : and (
' 1
] ) .
mBc B R
B D A B m
![Page 29: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/29.jpg)
Dual Viewpoint (Cont.)
' '[
[
= [
' 1
' ' 1
: ]
: ]
: ]
B
B B
A B D
c B B D
c c B D
and .Dc D
' [
[ = ' (3.8)
' ' 1
' '
: ]
: ]
B B
B D
A c c B D
c c c
Note that
The ’ we are looking for must satisfy (3.5) for anyThe root of (3.5) comes from So,
Note that ’ in (3.5) and (3.6) satisfies (3.7) and (3.8).
0.x
![Page 30: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/30.jpg)
Dual Objective FunctionNow consider the objective function
From inequality (3.8) that then for any bfs
’ is the one that leads to the maximum value while satisfying (3.7) and (3.8)
' ' (3.9a)
(3.9b)
( = 0) (3.9c)
**' *'
*
*' *' *' *1 1
*' *' *' *
*' *
* [ : ]
* ( * )
( )
BB D
D
B D B D
D D
D
xc x c x c c
x
c B b c c B D x
b c D x
b x
'' cA 0xwe have
xcbxA ''' bxcbxA *'*''*' and from (3.9)
b'
![Page 31: Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity.](https://reader030.fdocuments.net/reader030/viewer/2022012910/56649f255503460f94c3c6e9/html5/thumbnails/31.jpg)
Dual SLP
(SLP)
Subject to
2008/3/4 31Shi-Chung Chang, NTUEE, GIIE, GICE
minc x
x
0
,
.
Ax b
x
(DSLP)
Subject to
max b
,
.
mR
A c