Sub Exponential Randomize Algorithm for Linear Programming
description
Transcript of Sub Exponential Randomize Algorithm for Linear Programming
![Page 1: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/1.jpg)
Sub ExponentialSub Exponential Randomize Algorithm for Linear Randomize Algorithm for Linear
ProgrammingProgramming
Paper by:
Bernd Gärtner and Emo Welzl
Presentation by :
Oz Lavee
![Page 2: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/2.jpg)
Linear programmingLinear programming
The linear programming problem is a well known problem in computational geometry
The last decade brought a progress in the efficiency of the linear programming algorithms
Most of the algorithms were exponential in the dimension of the problem
![Page 3: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/3.jpg)
Linear programmingLinear programming
The last progress is a randomized algorithm that solve linear programming problem with n inequalities and d variables (Rd ) in expected time of:
This algorithm that we will see is a combination of Matoušek and Kalai sub exponential bounds and Clarkson algorithms
![Page 4: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/4.jpg)
Definition : Linear programming Definition : Linear programming problemproblem
Find a non negative vector x Rd that minimize cTx subject to n linear inequalities Ax b
where x 0C – d-vector represent directionX – d-vector A[n,d] – n inequalities over d variables
![Page 5: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/5.jpg)
Example over RExample over R22
C
h1
h2
X
h1,h2 - inequalities
![Page 6: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/6.jpg)
DefinitionsDefinitions
• let H be the set of n half spaces defined by Ax b
•Let H+ be the set of d halfspaces defined by X0
•For a G H H+ we define vG as the lexicographically minimal point x minimizing
cTx over hG h
![Page 7: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/7.jpg)
Definitions : basisDefinitions : basis
•A set of halfspaces B H H+ is called a basis
if ,. vB is finite and for any subset B’ of B
vB’ < vB
•A basis of G is a minimal subset B G such that
vB = vG
![Page 8: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/8.jpg)
Definitions : violationsDefinitions : violations
a constraint h H H+ is violated by G if and only if vG < vG {h}
h is violated by vG if h is violated by G
h1
h2
h3
G= {h1,h3}
h = h2
vG
![Page 9: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/9.jpg)
Definitions : extremeDefinitions : extreme
a constraint h G is extreme in G if vG-{h} < vG
h1
h2
h3
G = {h1,h2,h3}
h2 is extreme vG
![Page 10: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/10.jpg)
Lemma 1
1. For F G H H+ , vF < vG
2. vF,vG are finite and vF = vG .
h is violated by F if and only if
h is violated by G.
3. If vG is finite then any basis of G has exactly d constraints , and G has at the most d extreme constraints
![Page 11: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/11.jpg)
The algorithmThe algorithm
Our algorithm is a combination of 3 algorithms that use each other:
1. Clarkson first algorithm (CL1) for n>>d 2. Clarkson second algorithm
(CL2) for 3. Subexponenial
algorithm(subex) for 6d2
CL1
CL2
subex
n>>d
nd
6d2
nd
![Page 12: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/12.jpg)
Clarkson First Algorithm (CL1)Clarkson First Algorithm (CL1) set H of n constraints where n>>d
We choose a random sample R H
of size ,compute vR and the set V of constraints from H that are violated by vR
If V is not too large we add it to initially set G = H+ , choose another sample R and
compute VR G and so on…
nd
![Page 13: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/13.jpg)
Clarkson First Algorithm (CL1)Clarkson First Algorithm (CL1) CL1 (H)
if n < 9d2 then
return CL2(H)
else
r = , G = H+
repeat
choose random
v = CL2 (R)
V = {h H | v violate h}
if then G = GV
until V = Ø
return v
nd
r
HR
nV 2
![Page 14: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/14.jpg)
Two important factsTwo important facts
Fact 1: The expected size of V is no more than
The probability that |V| >
is at the most
The number of attempts to get a small enough V
expected to be at the most 2
n
n2
2
1
![Page 15: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/15.jpg)
Three important factThree important fact
Fact 2:If any constraint is violated by v = vGR
then for any basis B of H there must be a constraint that is violated by v.
The number of expansion of G is at the most d
![Page 16: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/16.jpg)
CL1 Run timeCL1 Run time
CL1 compute vH H+ with expected
o(d2n) operations and at the most 2d
calls to CL2 with constraints)( ndO
![Page 17: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/17.jpg)
Clarkson Second Algorithm CL2Clarkson Second Algorithm CL2
This algorithm is very similar to the CL1
with the main different that instead of forcing V to be in the next iteration
We increase the probability of the elements
in v to be chosen for R in the next iteration
![Page 18: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/18.jpg)
Clarkson Second Algorithm CL2Clarkson Second Algorithm CL2
We will use factor µ for each constraint that will be initialized to 1 .
For any constraint in V we will double his factor
For any basis B the elements of B increase so quickly that after a logarithmic time they will be chosen with high probability
![Page 19: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/19.jpg)
Clarkson Second Algorithm CL2Clarkson Second Algorithm CL2
CL2 (H) if n 6d2 then
return subex(H) else
r = 6d2 repeat choose random v = subex (R)
V = {h H | v violate h}
if then for all h V do µh = 2µh until V = Ø return v
r
HR
)(3
1)( H
dV
![Page 20: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/20.jpg)
Lemma 4Lemma 4
Let k be a positive integer then after kd
Successful Iterations we have
For any basis B of H
nekk
B3/
)(2
![Page 21: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/21.jpg)
Run time CL2Run time CL2
Since and since 2 > e1/3
after big enough k 2k > nek/3
let k = 3ln(n)
2 k = n3ln2 > n2 = ne3ln(n)/3
There will be at the most 3dln(n) successful iterations
There will be at the most 6dln(n) iterations
nekk
B3/
)(2
![Page 22: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/22.jpg)
Run time CL2Run time CL2
In each iteration there are O(dn) arithmetic
operations and one call to subex
Totally O(d2nlogn) operations and 6dln(n) calls to subex
![Page 23: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/23.jpg)
The Sub Exponential Algorithm The Sub Exponential Algorithm (SUBEX)(SUBEX)
The idea :– H a group of n constraints .– Remove a random constraint h– Compute recursively vH-{h} – If h is not violated then done.– Else try again by removing
(hopefully different) constraintThe probability that h is violated is d/n
![Page 24: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/24.jpg)
The Subex AlgorithmThe Subex Algorithm
In order to get efficiency we will pass to the subex procedure in addition to the set G of constraints a candidate basis
We assume that we have the following primitive procedures:– Basis(B,h) : calculate the basis of B {h}– Violation test– Calculate VB of basis B
![Page 25: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/25.jpg)
The Subex AlgorithmThe Subex Algorithm
Subex(G,B)if G=B
return (VB,B) else choose random hG-B (v,B’) = Subex(G- {h},B)
if h violates v return Subex(G,basis(B’,h))
else return (v,B’)
![Page 26: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/26.jpg)
The Subex AlgorithmThe Subex Algorithm
The number of steps is finite since:– The first recursive call decrease the number of
constraints– The second recursive call increase the value of
the temporal solution
Inductively it can be shown that the step keeps the correctness of the temporal solution
![Page 27: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/27.jpg)
The Subex Algorithm Run TimeThe Subex Algorithm Run Time
The subex algorithm is computing vH H+
with We called the subex algorithm with 6d2
constraints
so the run time for each call to subex is
))(( )ln((2 ndOenddO
)ln()ln(3 )( ddOddO eedO
![Page 28: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/28.jpg)
Total Run TimeTotal Run TimeThe runtime of cl2 =
O(d2nlogn + 6dln(n)Tsubex(6d2) =
The runtime of cl1 =
)loglog( )ln(2 nenndO ddO
))(2( 22 nddTndO CL
)()log(
)loglog()ln(2)ln(2
)ln(22
ddOddO
ddO
endOnendO
nenndndO
![Page 29: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/29.jpg)
The Abstract FrameworkThe Abstract Framework
We can expand this algorithm to be used on larger range of problems.
Let H be a finite set
let (W{-},) linear ordered set of values
Let w : 2H W{-} value function
![Page 30: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/30.jpg)
The Abstract FrameworkThe Abstract Framework
Our goal :
to find a minimal subset BH
where w(B) = w(H)
We can use the algorithm that we saw
in order to solve this problem if 2 axiom are satisfied.
![Page 31: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/31.jpg)
The axiomsThe axioms
1. For any F ,G such that FG H
we have w(F)w(G) .(monotonicity)
2. For any FG H with w(F) = w(G)
if hH w(G) < w(G {h}) than
w(F) < w(F {h})
![Page 32: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/32.jpg)
LP-type problemsLP-type problems
If those axioms hold for a definition of a problem we will call it
LP-type problem
From lemma 1 we can see that the linear programming problem is LP-type.
![Page 33: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/33.jpg)
LP-type problemsLP-type problems
For the efficiency of the algorithm we need one more parameter for (H,w):
the maximum size of any basis of H which is referred to as the combinatorial dimension of (H,w) denote as
![Page 34: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/34.jpg)
LP-type problemsLP-type problems
Any LP-type problem can be solved using the above algorithm but it is not necessarily
be in sub exponential time In order to have sub exponential time the
problem should have the property of basis regularity
Basis regularity – all the basis has exactly constraints
![Page 35: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/35.jpg)
LP-type problems - ExamplesLP-type problems - Examples
Smallest closing ball- given a set of n points in Rd find the smallest closing ball (combinatorial dimension- d+1)
Polytope distance – given two polytopes P,Q.
compute pP q Q minimizing dist (p,q)(combinatorial dimension- d+2)
![Page 36: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/36.jpg)
SummerySummery
We have seen a randomized sub exponential in d-space algorithm for the linear programming problem
We have seen the family of LP-type problems
![Page 37: Sub Exponential Randomize Algorithm for Linear Programming](https://reader036.fdocuments.net/reader036/viewer/2022070419/56815a73550346895dc7d7da/html5/thumbnails/37.jpg)
ReferencesReferences
Linear-programming – Randomization and abstract frame work /
Bernd Gärtner and Emo welzl