Post on 09-Apr-2020
Wolfe’s Method
Jamie Haddock
April 6, 2017
Graduate Group in Applied Mathematics
UC Davis
Minimum Norm Point in Polytope
We are interested in solving the problem (MNP(P)):
minx∈P||x ||2
where P is a polytope.
Reminder: A polytope, P, is the convex hull of points p1, p2, ..., pm,
P =
{ m∑i=1
λipi :m∑i=1
λi = 1, λi ≥ 0 for all i = 1, 2, ...,m
}.
1
Minimum Norm Point in Polytope
We are interested in solving the problem (MNP(P)):
minx∈P||x ||2
where P is a polytope.
Reminder: A polytope, P, is the convex hull of points p1, p2, ..., pm,
P =
{ m∑i=1
λipi :m∑i=1
λi = 1, λi ≥ 0 for all i = 1, 2, ...,m
}.
1
Minimum Norm Point in Polytope
p1p2
p3
p4
p5
P
O
2
Minimum Norm Point in Polytope
p1p2
p3
p4
p5
P
O
2
Formulation as QP
This is a quadratic programming problem (QP).
Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.
Then our problem is which can be solved using interior-point methods.
3
Formulation as QP
This is a quadratic programming problem (QP).
Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.
Then our problem is which can be solved using interior-point methods.
3
Formulation as QP
This is a quadratic programming problem (QP).
Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.
Then our problem is
min ||Py ||22s.t. 1T y = 1
y ≥ 0
which can be solved using interior-point methods.
3
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
4
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
p1p2
p3
p4
p5
P
O
4
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
p1p2
p3
p4
p5
P
O
4
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
p1p2
p3
p4
p5
P
O
{x : pT4 x = ||p4||22}
4
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).5
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
6
Sketch of Method
x ∈ P = {p1,p2, ...,pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1,p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
Py
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C)
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
Py
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1,p2,p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O = y
Px
C = {p1, p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C)
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O = y
Px
C = {p1, p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z− y||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
z
p1
p3
p2
O = y
Px
C = {p1, p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C− {pi} where pi, z
are on different faces of
conv(C)
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
z
p1
p3
p2
O = y
Px
C = {p2,p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
x = z
p1
p3
p2
O = y
P
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
x = z
p1
p3
p2
O
P
y
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C)
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
x = z
p1
p3
p2
O
P
y
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
x = y
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
x
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
x
C = {p2, p3}
6
LP reduces to MNP
Proposition (De Loera, H., Rademacher)
An instance of LP can be reduced to an instance of MNP in strongly
polynomial time.
This means that if there is a polynomial time (or even better, strongly
polynomial time) algorithm for MNP then we have a polynomial time (or
strongly polynomial time) algorithm for LP.
7
LP reduces to MNP
Proposition (De Loera, H., Rademacher)
An instance of LP can be reduced to an instance of MNP in strongly
polynomial time.
This means that if there is a polynomial time (or even better, strongly
polynomial time) algorithm for MNP then we have a polynomial time (or
strongly polynomial time) algorithm for LP.
7
Experiments
8
Experiments
8