Quadratic Programswith Complementarity Constraints1
John E. Mitchell
Department of Mathematical SciencesRPI, Troy, NY 12180 USA
ISMP, BerlinAugust 23, 2012
1Joint work with Jong-Shi Pang and Lijie Bai. Supported by AFOSR.Mitchell (RPI) QPCCs ISMP, August 23, 2012 1 / 26
Outline
1 Introduction and Applications
2 Completely Positive Relaxation
3 Preprocessing using SDP
4 Conclusions
Mitchell (RPI) QPCCs ISMP, August 23, 2012 2 / 26
Introduction and Applications
Outline
1 Introduction and Applications
2 Completely Positive Relaxation
3 Preprocessing using SDP
4 Conclusions
Mitchell (RPI) QPCCs ISMP, August 23, 2012 3 / 26
Introduction and Applications
Quadratic Programs with Complementarity Constraints
Find x := (x0, x1, x2) ∈ IRn+ × IRm
+ × IRm+ to globally solve the quadratic
program with complementarity constraints (QPCC):
minimizex,(x0,x1,x2)≥0
cT x + 12 xT Qx
subject to Ax = b and ( x1 )T x2 = 0
We assume the quadratic form is convex.
Generalizes the linear complementarity problem:find y ∈ IRm with 0 ≤ y ⊥ q + My ≥ 0.
Can formulate as integer program if know bounds on x1 and x2.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 4 / 26
Introduction and Applications Applications
Eg: Bilevel programs
Bilevel program:
minx ,y f (x , y)
s.t. g(x , y) ≤ 0
y ∈ argminy{h(y) :
r(y) ≤ v(x)}
MPCC:
min f (x , y)
s.t. g(x , y) ≤ 0
∇h(y) + ∇r(y)λ = 0
0 ≤ v(x)− r(y) ⊥ λ ≥ 0
Mitchell (RPI) QPCCs ISMP, August 23, 2012 5 / 26
Introduction and Applications Applications
Eg: Bilevel programs
Bilevel program:
minx ,y f (x , y)
s.t. g(x , y) ≤ 0
y ∈ argminy{h(y) :
r(y) ≤ v(x)}
MPCC:
min f (x , y)
s.t. g(x , y) ≤ 0
∇h(y) + ∇r(y)λ = 0
0 ≤ v(x)− r(y) ⊥ λ ≥ 0
Mitchell (RPI) QPCCs ISMP, August 23, 2012 5 / 26
Introduction and Applications Applications
Eg: Bilevel programs
Bilevel program:
minx ,y f (x , y)
s.t. g(x , y) ≤ 0
y ∈ argminy{h(y) :
r(y) ≤ v(x)}
MPCC:
min f (x , y)
s.t. g(x , y) ≤ 0
∇h(y) + ∇r(y)λ = 0
0 ≤ v(x)− r(y) ⊥ λ ≥ 0
Equivalent if h(y) and r(y) are convex, and a constraint qualificationholds for the subproblems.
Get a Quadratic Program with Complementarity Constraints (QPCC) iff is convex quadratic and g, r , v are all linear and h linear or quadratic.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 5 / 26
Introduction and Applications Applications
Bilevel and Inverse Optimization Applications
• Selecting parameters to force a desired response.Eg: traffic flow.
• Parameter identification to construct better models.Eg: cross validation in data mining.
• Portfolio selection.Eg: VaR.
• System identification.Eg: terrain estimation from behavior of seismic waves.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 6 / 26
Completely Positive Relaxation
Outline
1 Introduction and Applications
2 Completely Positive Relaxation
3 Preprocessing using SDP
4 Conclusions
Mitchell (RPI) QPCCs ISMP, August 23, 2012 7 / 26
Completely Positive Relaxation
Expressing the QPCC as a QCQP
The QPCC is a quadratically constrained quadratic program (QCQP).
Complementarity constraint: (x1)T x2 ≤ 0.Generalize to quadratic constraint q(x) ≤ 0 where
Ax = b, x ≥ 0 implies q(x) ≥ 0.
Generalization subsumes a binary restriction:qi(x) = xi(1− xi) satisfies our condition,provided Ax = b, x ≥ 0 implies 0 ≤ xi ≤ 1 (Burer’s key assumption).
Multiple quadratic constraints satisfying the condition can be summed.So we work with just a single quadratic constraint.
We have a QCQP with no Slater point.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 8 / 26
Completely Positive Relaxation
Lifting the QCQPWe have a QCQP with a single quadratic constraint:
q(x) := h + gT x + 12xT Px ≤ 0,
such that Ax = b, x ≥ 0 implies q(x) ≥ 0.
Can be lifted to a completely positive program in a well-known manner:
minimizex ,X
cT x + 12 〈Q, X 〉
subject to Ai x = bi and Ai X ATi = b2
i , i = 1, · · · , k ,
h + gT x + 12 〈P,X 〉 = 0
and
(1 xT
x X
)∈ C∗1+n
(cone of completelypositive matrices
).
In general, the copositive program is a relaxation of the QCQP.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 9 / 26
Completely Positive Relaxation
Burer’s result
Theorem (Burer, Math Progg, 2009)If q(x) =
∑i∈B xi(1− xi) and if Ax = b, x ≥ 0 implies 0 ≤ xi ≤ 1 ∀i ∈ B
then the QCQP and its completely positive relaxation are equivalent.
Note that Burer imposes no convexity assumption on the objectivefunction.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 10 / 26
Completely Positive Relaxation
Our assumptions∗ Ax = b, x ≥ 0 implies q(x) ≥ 0.
∗ P is copositive on the recession cone of the linear constraints,
K , {d > 0 | A d = 0} ,
so dT Pd ≥ 0 for all d ∈ K .Note: for a QPCC, every entry in P is nonnegative.
∗ Let L ,{
d > 0 | A d = 0 and d T Pd = 0}⊆ K .
Assume objective function matrix Q is copositive on L.
∗ Note: No boundedness assumption on any of the variables.
∗ Note: Assumptions all hold if q(x) =∑
i∈B xi(1− xi) and0 ≤ xi ≤ 1∀i ∈ B. (Burer: third assumption holds providedoptimal value of BQP is finite.)
Mitchell (RPI) QPCCs ISMP, August 23, 2012 11 / 26
Completely Positive Relaxation
Completely positive relaxation is tight
TheoremUnder our three assumptions, the QCQP and its completely positiverelaxation are equivalent in the sense that
1. The QCQP is feasible if and only if the completely positiveprogram is feasible.
2. Either the optimal values of the QCQP and the completelypositive program are finite and equal, or both of them areunbounded below.
3. Assume both the QCQP and the completely positiveprogram are bounded below, and (x̄ , X̄ ) is optimal for thecompletely positive program, then x̄ is in the convex hullof the optimal solutions of the QCQP.
4. The optimal value of the QCQP is attained if and only ifthe same holds for the completely positive program.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 12 / 26
Completely Positive Relaxation
Completely positive relaxation is tight
TheoremUnder our three assumptions, the QCQP and its completely positiverelaxation are equivalent in the sense that
1. The QCQP is feasible if and only if the completely positiveprogram is feasible.
2. Either the optimal values of the QCQP and the completelypositive program are finite and equal, or both of them areunbounded below.
3. Assume both the QCQP and the completely positiveprogram are bounded below, and (x̄ , X̄ ) is optimal for thecompletely positive program, then x̄ is in the convex hullof the optimal solutions of the QCQP.
4. The optimal value of the QCQP is attained if and only ifthe same holds for the completely positive program.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 12 / 26
Completely Positive Relaxation
Completely positive relaxation is tight
TheoremUnder our three assumptions, the QCQP and its completely positiverelaxation are equivalent in the sense that
1. The QCQP is feasible if and only if the completely positiveprogram is feasible.
2. Either the optimal values of the QCQP and the completelypositive program are finite and equal, or both of them areunbounded below.
3. Assume both the QCQP and the completely positiveprogram are bounded below, and (x̄ , X̄ ) is optimal for thecompletely positive program, then x̄ is in the convex hullof the optimal solutions of the QCQP.
4. The optimal value of the QCQP is attained if and only ifthe same holds for the completely positive program.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 12 / 26
Completely Positive Relaxation
Completely positive relaxation is tight
TheoremUnder our three assumptions, the QCQP and its completely positiverelaxation are equivalent in the sense that
1. The QCQP is feasible if and only if the completely positiveprogram is feasible.
2. Either the optimal values of the QCQP and the completelypositive program are finite and equal, or both of them areunbounded below.
3. Assume both the QCQP and the completely positiveprogram are bounded below, and (x̄ , X̄ ) is optimal for thecompletely positive program, then x̄ is in the convex hullof the optimal solutions of the QCQP.
4. The optimal value of the QCQP is attained if and only ifthe same holds for the completely positive program.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 12 / 26
Completely Positive Relaxation
Completely positive relaxation is tight
TheoremUnder our three assumptions, the QCQP and its completely positiverelaxation are equivalent in the sense that
1. The QCQP is feasible if and only if the completely positiveprogram is feasible.
2. Either the optimal values of the QCQP and the completelypositive program are finite and equal, or both of them areunbounded below.
3. Assume both the QCQP and the completely positiveprogram are bounded below, and (x̄ , X̄ ) is optimal for thecompletely positive program, then x̄ is in the convex hullof the optimal solutions of the QCQP.
4. The optimal value of the QCQP is attained if and only ifthe same holds for the completely positive program.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 12 / 26
Completely Positive Relaxation
Notes regarding the proof
The structure of our proof is similar to that of Burer, and we examine
(1 xT
x X
)=∑j∈J+
v2j
1ξj
vj
1ξj
vj
T
+∑j∈J0
(0ξj
) (0ξj
)T
for points feasible in the completely positive program.
We are able to show that
* q(ξjvj
)= 0 ∀j ∈ J+, so these points are feasible in the QCQP
* ξTj Pξj = 0 ∀j ∈ J0, so these rays are in our set L.
The existence of such rays makes it necessary to define Land require Q to be copositive on L.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 13 / 26
Completely Positive Relaxation
Completely positive representations of QPCCs
CorollaryThe convex QPCC
minimizex,(x0,x1,x2)≥0
cT x + 12 xT Qx
subject to Ax = b and ( x1 )T x2 = 0
(with psd Q) is equivalent to a completely positive program.
Note that we impose no boundedness assumption on thecomplementary variables.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 14 / 26
Completely Positive Relaxation
Extensions to convex constraints
Burer extended his results to problems defined over convex cones(Handbook of Semidefinite, Cone, and Polyhedral Programming,2011). (See also Eichfelder and Povh.)
This requires the extension of the definition of a completely positivematrix to a matrix that is completely positive over a cone K:
C∗n(K) , conv{
M ∈ Sn | M = xxT , x ∈ K}.
Our results can be similarly extended.
For example, a convex QCQP with additional convex quadraticconstraints is equivalent to a completely positive program.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 15 / 26
Preprocessing using SDP
Outline
1 Introduction and Applications
2 Completely Positive Relaxation
3 Preprocessing using SDP
4 Conclusions
Mitchell (RPI) QPCCs ISMP, August 23, 2012 16 / 26
Preprocessing using SDP
An example
0
w
y
minw ,y y2 + 4w2
subject to y + w ≥ 50 ≤ y ⊥ w ≥ 0
Mitchell (RPI) QPCCs ISMP, August 23, 2012 17 / 26
Preprocessing using SDP
An example
0
w
y
minw ,y y2 + 4w2
subject to y + w ≥ 50 ≤ y ⊥ w ≥ 0
Solution to relaxation is not feasible in QPCC.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 17 / 26
Preprocessing using SDP
Modify objective function
yw = 0 in any feasible solution, so add multiple of yw to objective.
0
w
y
minw ,y y2 + 4w2 + 4ywsubject to y + w ≥ 5
0 ≤ y ⊥ w ≥ 0
Mitchell (RPI) QPCCs ISMP, August 23, 2012 18 / 26
Preprocessing using SDP
Modify objective function
yw = 0 in any feasible solution, so add multiple of yw to objective.
0
w
y
minw ,y y2 + 4w2 + 4ywsubject to y + w ≥ 5
0 ≤ y ⊥ w ≥ 0
Solution to relaxation is optimal for QPCC.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 18 / 26
Preprocessing using SDP
Generalize the example
minimizex,(x0,x1,x2)≥0
cT x + 12 xT Qx
subject to Ax = b and ( x1 )T x2 = 0
Mitchell (RPI) QPCCs ISMP, August 23, 2012 19 / 26
Preprocessing using SDP
Generalize the example
minimizex,(x0,x1,x2)≥0
cT x + 12 xT Qx + ( x1 )T Dx2 + xT M(Ax − b)
subject to Ax = b and ( x1 )T x2 = 0
D is a positive semidefinite diagonal matrix:so penalize violations of complementarity.
New formulation is equivalent to old formulation.
Choose D and M to ensure objective function remains convex.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 19 / 26
Preprocessing using SDP
Using SDP to find tight relaxationWant to get a good bound when relax complementarity.Solve maxmin problem:
maxD,M
minimizex,(x0,x1,x2)≥0
cT x + 12 xT Qx + ( x1 )T Dx2 + xT M(Ax − b)
subject to Ax = b and ( x1 )T x2 = 0//////////////////
D and M are constrained so that D is diagonal and psd,and the resulting quadratic term is psd.
Related work:Billionnet et al have proposed a quadratic convex reformulationmethod for a binary quadratic program, with the aim of obtaining aconvex relaxation that is as tight as possible.With binary variables, can always add x2
i − xi to objective, so easy toensure convexity.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 20 / 26
Preprocessing using SDP
Using Lagrangian relaxation
The previous problem is the Lagrangian dual of
minx,(x0,x1,x2)≥0
cT x + 12xT Qx
subject to Ax = b
(x1)i(x2)i = 0, i = 1, . . . ,m
xi(Ax − b)j = 0, i = 1, . . . ,n, j = 1, . . . , k
Write as:
minx≥0
cT x + 12xT Qx
subject to Ax = b
cTj x + xT Qjx = 0, j = 1, . . . , J
Mitchell (RPI) QPCCs ISMP, August 23, 2012 21 / 26
Preprocessing using SDP
Set up an SDP
Lift the problem and take its SDP relaxation:
minx≥0,X
cT x + 12 < Q,X >
subject to Ax = b
cTj x+ < Qj ,X > = 0, j = 1,2,3, · · ·, J.
and[
1 xT
x X
]� 0.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 22 / 26
Preprocessing using SDP
Get best relaxation by solving SDP
TheoremIf Q is positive definite then the SDP relaxation is equivalent to themax-min problem. Thus, the equivalent convex quadratic formulation ofthe QPCC with the best lower bound can be found by solving an SDP.
Proof exploits Lagrangian duality,and a result of Lemaréchal and Oustry.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 23 / 26
Preprocessing using SDP
Computational experiments
number of # constraints # variables % gap Timecomplementarities closed (secs)
50 eq: 55 110 96.74 3.15100 eq: 102 210 82.62 9.79300 eq: 305 610 67.19 163.63
90 ineq: 10, eq: 90 185 93.68 8.13
Average over ten problems of each size.
Problems generated using QPECgen.
Upper bound comes from known feasible solution.Optimal solution not known.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 24 / 26
Conclusions
Outline
1 Introduction and Applications
2 Completely Positive Relaxation
3 Preprocessing using SDP
4 Conclusions
Mitchell (RPI) QPCCs ISMP, August 23, 2012 25 / 26
Conclusions
Conclusions
Convex quadratic programs with complementarity constraints areequivalent to convex conic programs, even if the variables areunbounded.
Relaxations of convex QPCCs can be tightened through anSDP-based preprocessing approach.
Mitchell (RPI) QPCCs ISMP, August 23, 2012 26 / 26
Top Related