Graph-Theoretic Algorithms for Polynomial Optimization Problems

Somayeh Sojoudi, Ramtin Madani, Ghazal Fazelnia, and Javad Lavaei

Abstract— The objective of this tutorial paper is to study ageneral polynomial optimization problem using a semidefiniteprogramming (SDP) relaxation. The first goal is to show how theunderlying structure and sparsity of an optimization problemaffect its computational complexity. Graph-theoretic algorithmsare presented to address this problem based on the notions oflow-rank optimization and matrix completion. By building onthis result, it is then shown that every polynomial optimizationproblem admits a sparse representation whose SDP relaxationhas a rank 1 or 2 solution. The implications of these resultsare discussed in details and their applications in decentralizedcontrol and power systems are also studied.

I. INTRODUCTION

Optimization theory deals with the minimization of anobjective function subject to a set of constraints. This areaplays a vital role in the design, control, operation, andanalysis of real-world systems. The development of efficientoptimization techniques and numerical algorithms has beenan active area of research for many decades. The goal is todesign a robust and scalable method that is able to find aglobal solution in polynomial time. This question has beenfully answered for the class of convex optimization problemsthat includes all linear and some nonlinear problems [1]–[3].Convex optimization has found a wide range of applicationsacross engineering and economics [4]. In the past severalyears, a great effort has been devoted to casting real-world problems as convex optimization. Nevertheless, severalclasses of optimization problems, including polynomial op-timization and quadratically constrained quadratic program(QCQP) as a special case, are nonlinear, non-convex, andNP-hard in the worst case [5], [6]. In particular, there isno known effective optimization technique for integer andcombinatorial optimization as a small subclass of QCQP[7], [8]. Given a non-convex optimization, there are severaltechniques to find a solution that is locally optimal. However,seeking a global or near-global solution in polynomial timeis a daunting challenge. There is a large body of literatureon nonlinear optimization witnessing the complexity of thisproblem.

To reduce the computational complexity of a non-convexoptimization, several convex relaxation methods based onlinear matrix inequality (LMI), semidefinite programming

Somayeh Sojoudi is with the Langone Medical Center, New YorkUniversity. Ramtin Madani, Ghazal Fazelnia, and Javad Lavaei arewith the Electrical Engineering Department, Columbia University(Email: somayeh.sojoudi@nyumc.org, madani@ee.columbia.edu,gf2293@columbia.edu, and lavaei@ee. columbia.edu).

This work was supported by a Google Research Award, NSF CAREERAward and ONR YIP Award.

(SDP), and second-order cone programming (SOCP) havegained popularity [1], [9]. These techniques enlarge thepossibly non-convex feasible set into a convex set charac-terizable via convex functions, and then provide the exact ora lower bound on the optimal objective value associated witha global solution. The SDP relaxation technique provides alower bound on the minimum cost of the original problem,which can be used for various purposes such as the branchand bound algorithm [2]. To understand the quality of theSDP relaxation, its optimal objective value is shown to be atmost 14% different from the optimal cost for the MAXCUTproblem [10]. The maximum possible gap between thesolution of a graph optimization and its SDP relaxation isdefined as the Grothendieck constant of the graph [11], [12].This constant has been derived for some special cases in[13]. The paper [14] shows how a complex SDP relaxationmay solve the max-3-cut problem. This approach has beengeneralized in several papers [15]–[22]. If the SDP relaxationprovides the same optimal objective value as the originalproblem, the relaxation is said to be exact. The exactnessof the SDP relaxation has been verified for a variety ofproblems [23]–[26]. For instance, the work [27]–[30] hasexplored the SDP relaxation for the optimal power flow(OPF) problem, which is the most fundamental optimizationproblem for electrical power networks. That work shows thatthe relaxation is exact for a large class of OPF problems dueto the physics of a power gird. The exactness of an SDPrelaxation could be heavily formulation dependent. Indeed,a practical circuit optimization with four equivalent QCQPsis designed in [31], where only one of the formulations hasan exact SDP relaxation.

In the case where the SDP relaxation is not exact, theexistence of a low-rank SDP solution may still be helpful.To support this claim, a penalized SDP relaxation is proposedfor the OPF problem in [31], [32] and successfully used toderive near-global solutions for 7000 instances of OPF (anear-global solution is a near-optimal solution that is closeto a global solution with a known upper bound on its distanceto global optimality). In a general context, the existence ofa low-rank solution to matrix optimization problems withlinear and LMI constraints has been extensively studied inthe literature [33], [34]. The papers [35]–[37] provide anupper bound on the lowest rank among all solutions ofa feasible LMI problem. Based on the same approach, aconstructive method has been proposed in [38] to obtain alow-rank solution in polynomial time. Although the provenbound in [37] is tight in the worst case, many examples

are known to possess solutions with a lower rank due totheir underlying sparsity patterns [39], [40]. A rank-1 matrixdecomposition technique is developed in [41] to find a rank-1 solution whenever the number of constraints is small. Thistechnique is extended in [42] to the complex SDP problem.The paper [43] presents a polynomial-time algorithm forfinding an approximate low-rank solution.

This tutorial paper aims to study the SDP relaxation ofa polynomial optimization through graph-theoretic notions.To this end, three problems will be addressed here. InSection II, the objective is to investigate how the structure ofan optimization reduces the computational complexity. Forthis purpose, the structure of the optimization is mapped intoa weighted graph and it is shown that the SDP relaxation isexact if the graph possesses certain properties. In Section III,it is shown that the SDP relaxation of a sparse optimizationhas a solution whose rank can be characterized in termsof the sparsity level of the problem. In Section IV, it isexplained that every polynomial optimization admits a sparserepresentation whose SDP relaxation has a rank 1 or 2 matrixsolution. In other words, it is shown that the NP hardnessof polynomial optimization can be traced back to attaining arank-2 solution rather than a rank-1 solution. The techniquespresented in this paper are illustrated on two notoriousproblems of “optimal distributed control” and “nonlinearpower optimization” in Sections V and VI, respectively.

A. NotationsThe notations used throughout this tutorial paper will be

described below. R, C, Z+, Sn, and Hn denote the sets of realnumbers, complex numbers, nonnegative integer numbers,n × n symmetric matrices, and n × n Hermitian matrices,respectively. Sn+ and Hn+ denote the restrictions of Sn andHn to positive semidefinite matrices. ReW, ImW,rankW, and traceW denote the real part, imaginarypart, rank, and trace of a given scalar/matrix W, respectively.The notation W 0 means that W is Hermitian and positivesemidefinite. Given a matrix W, its (l,m) entry is denotedas Wlm. The superscript (·)opt is used to show the globallyoptimal value of an optimization parameter. The symbol (·)∗represents the conjugate transpose operator. ](x) representsthe phase of a complex number x. The imaginary unit isdenoted as “i”, while “i” is used for indexing. Given a setT , |T | denotes its cardinality. Given a graph G, |G| showsthe number of its vertices. Given a number (vector) x, |x|denotes its absolute value (2-norm). Given an undirectedgraph G, the notation i ∈ G means that i is a vertex ofG. Moreover, the notation (i, j) ∈ G means that (i, j) is anedge of G and besides i < j.

II. HIGHLY-STRUCTURED OPTIMIZATION

In this section, the objective is to investigate how theunderlying sparsity or structure of an optimization problemaffects its computational complexity. Our approach is to mapthe structure of the optimization into a weighted graph andrelate the exactness of a conic relaxation for the optimizationto certain properties of the graph.

1 2

3

4

1'2'

3'

4'

(a)

12

34

1'2'

3'

4'

(b)

Fig. 1: In Figure (a), there exists a line separating x’s(elements of T ) from o’s (elements of −T ) so the set Tis sign definite. In Figure (b), this is not the case.

A. Definitions

Before proceeding with this part, some definitions will beprovided below.

Definition 1: A finite set T ⊂ R is said to be sign definitewith respect to R if its elements are either all negative or allnonnegative. T is called negative if its elements are negativeand is called positive if its elements are nonnegative.

Definition 2: A finite set T ⊂ C is said to be sign definitewith respect to C if when the sets T and −T are mappedinto two collections of points in R2, then there exists a lineseparating the two sets (note that any or all elements of thesets T and −T are allowed to lie on the separating line).

To illustrate Definition 2, consider a complex set T withfour elements, whose corresponding points are labeled as 1,2, 3 and 4 in Figure 1(a). The points corresponding to −Tare labeled as 1’, 2’, 3’ and 4’ in the same picture. Sincethere exists a line separating x’s (elements of T ) from o’s(elements of −T ), the set T is sign definite. In contrast, ifthe elements of T are distributed according to Figure 1(b),the set will no longer be sign definite. Note that Definition 2is inspired by the fact that a real set T is sign definite withrespect to R if T and −T are separable via a point (on thehorizontal axis).

Definition 3: Given a graph G, a cycle space is the set ofall possible cycles in the graph. An arbitrary basis for thiscycle space is called a “cycle basis”.

Definition 4: In this work, a graph G is called weaklycyclic if every edge of the graph belongs to at most onecycle in G (i.e., the cycles of G are all edge-disjoint).

Definition 5: Consider a graph G, a subgraph Gs of thisgraph and a matrix W ∈ H|G|. Define WGs as a sub-matrix of W located in the intersection of those rows andcolumns of W whose indices belong to the vertex set of Gs.For instance, W(i, j) is obtained by intersecting rows i, jwith columns i, j of W, for every (i, j) ∈ G.

B. Problem Statement

Consider an undirected graph G with n vertices (nodes),where each edge (i, j) ∈ G has been assigned a nonzero edgeweight set c(1)ij , c

(2)ij , ..., c

(k)ij with k real/complex numbers

(note that the superscripts in the weights are not exponents).This graph is called a generalized weighted graph as everyedge is associated with a set of weights as opposed to a singleweight. Consider an unknown vector x =

[x1 · · · xn

]

belonging to Dn, where D is either R or C. For every i ∈ G,xi is a variable associated with node i of the graph G. Define:

y =|xi|2

∣∣ ∀i ∈ G,z =

Rec(t)ij xix

∗j

∣∣ ∀(i, j) ∈ G, t ∈ 1, ..., kNote that (i, j) ∈ G means that (i, j) is an edge of the graphand that i < j. The sets y and z can be regarded as twovectors, where• y collects the quadratic terms |xi|2’s (one term for each

vertex).• z collects the cross terms Rec(t)ij xix∗j’s (k terms for

each edge).Although the above formulation deals with Re

c(t)ij xix

∗j

whenever (i, j) ∈ G, it can handle terms of the formReαxjx∗i and Imαxix∗j for a complex weight α. Thiscan be carried out using the transformations:

Reαxjx∗i = Re(α∗)xix∗j,Imαxix∗j = Re(−αi)xix∗j

This part is concerned with the following optimization prob-lem:

minx∈Dn

f0(y, z)

subject to fj(y, z) ≤ 0, j = 1, 2, ...,m(1)

for given functions f0, ..., fm. The computational complexityof the above optimization problem depends in part on thestructure of the functions fj’s. Regardless of these functions,the optimization problem (1) is intrinsically hard to solve(NP-hard in the worst case) because y and z are bothnonlinear functions of x. The objective is to convexify thesecond-order nonlinearity embedded in y and z. To this end,notice that there exist two linear functions l1 : Cn×n →Rn and l2 : Cn×n → Rkτ such that y = l1 (xx∗) andz = l2 (xx∗), where τ denotes the number of edges in G.Motivated by the above observation, if xx∗ is replaced bya new matrix variable W, then y and z both become linearin W. This implies that the non-convexity induced by thequadratic terms Rec(t)ij xixj’s and |xi|’s all disappear if theoptimization problem (1) is reformulated in terms of W.However, the optimal solution W may not be decomposableas xx∗ unless some additional constraints are imposed onW. It is straightforward to verify that the optimizationproblem (1) is equivalent to

minW

f0(l1(W), l2(W)) (2a)

s.t. fj(l1(W), l2(W)) ≤ 0, j = 1, ...,m (2b)W 0, (2c)rankW = 1 (2d)

where there is an implicit constraint that W ∈ Sn if D =R and W ∈ Hn if D = C. To reduce the computationalcomplexity of the above problem, two actions can be taken:(i) removing the nonconvex constraint (2d), and (ii) relaxingthe convex, but computationally-expensive, constraint (2c) toa set of simpler constraints on certain low-order submatrices

of W. Based on this methodology, three relaxations will beproposed for the optimization problem (1) next.

SDP relaxation: This optimization problem is defined as

minW

f0(l1(W), l2(W)) (3a)

s.t. fj(l1(W), l2(W)) ≤ 0, j = 1, ...,m (3b)W 0 (3c)

Reduced SDP relaxation: Choose a set of cycles O1, ....,Opin the graph G such that they form a cycle basis. Let Ω denotethe set of all subgraphs O1, ....,Op as well as all edges ofG that do not belong to any cycle in the graph (i.e., bridgeedges). The reduced SDP relaxation is defined as

minW

f0(l1(W), l2(W)) (4a)

s.t. fj(l1(W), l2(W)) ≤ 0, j = 1, ...,m (4b)WGs 0, ∀Gs ∈ Ω (4c)

SOCP relaxation: This optimization problem is defined as

minW

f0(l1(W), l2(W)) (5a)

s.t. fj(l1(W), l2(W)) ≤ 0, j = 1, ...,m (5b)W(i, j) 0, ∀(i, j) ∈ G (5c)

The reason why the above optimization problem is called anSOCP problem is that the condition W(i, j) 0 can bereplaced by the linear and norm constraints

Wii,Wjj ≥ 0,

Wii + Wjj ≥∣∣∣∣ [ Wii Wjj

√2Wij

] ∣∣∣∣The above SDP, reduced SDP and SOCP relaxations

target the non-convexity caused by the nonlinear relationshipbetween x and (y, z). Note that these optimization problemsare convex relaxations only when the functions f0, ..., fm areconvex. If any of these functions is nonconvex, additionalrelaxations might be needed to convexify the SDP, reducedSDP or SOCP optimization problem. Define f opt, f opt

SDP, foptr-SDP

and f optSOCP as the optimal solutions of the optimization

problems (2), (3), (4) and (5), respectively. By comparingthe feasible sets of these optimization problems, it can beconcluded that

f optSOCP ≤ f

optr-SDP ≤ f

optSDP ≤ f

opt (6)

Given a particular optimization problem of the form (1), ifany of the above inequalities for f opt turns into an equality,the associated relaxation will be able to find the solutionof the original optimization problem. In this case, it is saidthat the relaxation is “tight” or “exact”. The objective ofthis part is to relate the exactness of the proposed relax-ations to the topology of the graph G and its weight setsc(1)ij , c

(2)ij , ..., c

(k)ij ’s.

It is noteworthy that the aforementioned problem formu-lation can be easily generalized in two directions:

• Allowance of weight sets with different cardinalities:The above problem formulation assumes that every edgeweight set has k elements. However, if the weight setshave different sizes, the trivial weight 0 can be added toeach set multiple times in such a way that all expandedsets reach the same cardinality.

• Inclusion of linear terms in x: The optimization prob-lem (1) is formulated in xx∗ with no linear term inx. This issue can be fixed by defining an expandedvector x as

[1 x∗

]∗. Then, the matrix xx∗ needs

to be replaced by a new matrix variable W under theconstraint W11 = 1.

C. Exactness of Conic Relaxations

Throughout this part, we assume that fj(y, z) is mono-tonic in every entry of z for j = 0, 1, ...,m (but possiblynonconvex in y and z). With no loss of generality, supposethat fj(y, z) is an increasing function with respect to allentries of z.

The objective of this part is to study the interrelationshipbetween f opt

SOCP, f optr-SDP, f opt

SDP and f opt. In particular, it is aimedto understand what properties the generalized weighted graphG should have to guarantee the exactness of some of theproposed relaxations. Some special cases of this problemhave been studied in [44]–[46]. In what follows, variousconditions will be provided to guarantee the exactness ofthe proposed SDP, reduced SDP and SOCP relaxation. Thereader may refer to [39] for more details.

The following statements hold in both real and complexcases D = R and D = C:

i) The SDP relaxation is exact (i.e., f optSDP = f opt) if and

only if it has a rank-1 solution Wopt.ii) The reduced SDP relaxation is exact (i.e., f opt

r-SDP = f opt)if and only if it has a solution Wopt such that

RankWoptGs = 1, ∀Gs ∈ Ω (7)

iii) The SOCP relaxation is exact (i.e., f optSOCP = f opt) if and

only if it has a solution Wopt such that

RankWopt(i, j) = 1, ∀(i, j) ∈ G

and that∑]Wopt

ij = 0, ∀r ∈ 1, 2, ..., p (8)

where the sum is taken over all directed edges (i, j) ofthe oriented cycle ~Or (note that ~Or denotes a directedcycle corresponding to Or). Moreover, the same resultholds even if the condition (8) is replaced by (7).

The above conditions reveal the role of the underlyinggraph of the optimization. By further simplifying theseconditions, it can be shown that the SOCP, reduced SDP andSDP relaxations are all tight in the real-valued case D = R,provided each weight set c(1)ij , c

(2)ij , ..., c

(k)ij is sign definite

with respect to R and∏(i,j)∈Or

σij = (−1)|Or|, ∀r ∈ 1, ..., p

where σij ∈ −1, 0, 1 shows the sign of the weight setassociated with the edge (i, j) ∈ G. This condition isnaturally satisfied in three special cases:• G is acyclic with arbitrary sign definite edge sets.• G is bipartite with positive weight sets.• G is arbitrary with negative weight sets.

If the SDP relaxation is not exact, it still has a low rank(rank-2) solution in two cases:• G is acyclic (but with potentially indefinite weight sets).• G is a weakly-cyclic bipartite graph with sign definite

edge sets.To study the complex-valued case D = C, assume that eachedge set c(1)ij , c

(2)ij , ..., c

(k)ij is sign definite with respect to

C. This assumption is trivially met if k ≤ 2 or the weight setcontains only real (or imaginary) numbers. It can be shownthat:• The SOCP, reduced SDP and SDP relaxations are all

tight if G is acyclic.• The SOCP, reduced SDP and SDP relaxations are tight

if each weight set contains only real or imaginarynumbers and∏

(i,j)∈ ~Or

σij = (−1)|Or|, ∀r ∈ 1, ..., p

where σij ∈ 0,±1,±i shows the sign of each weightset.

• The reduced SDP and SDP relaxations (but not neces-sarily the SOCP relaxation) are exact if G is bipartiteand weakly cyclic with positive or negative real weightsets.

• The reduced SDP and SDP relaxations (but not neces-sarily the SOCP relaxation) are exact if G is a weaklycyclic graph with imaginary weight sets and the signsσij = ±i.

Furthermore, if the graph G can be decomposed as a union ofedge-disjoint subgraphs in an acyclic way in such a way thateach subgraph has one of the above four structural properties,then the SDP relaxation is exact.

The above conditions will be examined on multiple exam-ples below.

D. Illustrative Examples

Example 1: The minimization of an unconstrained bivariatequartic polynomial can be carried out via an SDP relaxationobtained from the first-order sum-of-squares technique [47].In this example, we demonstrate how a computationallycheaper SOCP relaxation (in comparison to the foregoingSDP relaxation) can be used to solve the minimization of astructured bivariate quartic polynomial subject to an arbitrarynumber of structured bivariate quartic polynomials. To thisend, we first consider the unconstrained case, where the goalis to minimize the polynomial

f0(x1, x2) = x41 + ax22 + bx21x2 + cx1x2 (9)

with the real-valued variables x1 and x2, for arbitrary coef-ficients a, b, c ∈ R. In order to find the global minimum of

xx

-11x

2x

4x

3x

c b

(a)

1x

2x

3x

4x

5x

7x

6x

12c

13c

23c

14c

15c

45c

16c

17c

(b)

Fig. 2: Figures (a) and (b) show the weighted graph G forExamples 1 and 2, respectively.

this optimization problem, the standard convex optimizationtechnique cannot readily be used due to the non-convexityof f(x1, x2) in general. To address this issue, the aboveunconstrained minimization problem will be converted to aconstrained quadratic optimization problem. More precisely,the problem of minimizing f0(x1, x2) can be reformulatedin terms of x1, x2 and two auxiliary variables x3, x4 as:

minx∈R4

x23 + ax22 + bx3x2 + cx1x2 (10a)

s.t. x21 − x3x4 = 0, x24 − 1 = 0 (10b)

where x =[x1 x2 x3 x4

]∗. The above optimization

problem can be cast as follows:

minx∈R4,W∈S4

W33 + aW22 + bW23 + cW12 (11a)

s.t. W11 −W34 ≤ 0, W44 − 1 = 0 (11b)

and subject to the additional constraint W = xx∗. Note thatW11 −W34 ≤ 0 should have been W11 −W34 = 0, but thismodification does not change the solution. To eliminate thenon-convexity induced by the constraint W = xx∗, one canuse an SOCP relaxation obtained by replacing the constraintW = xx∗ with the convex constraints W(1, 2) 0,W(2, 3) 0 and W(3, 4) 0. To understand theexactness of this relaxation, the weighted graph G capturingthe structure of the optimization problem (10) should beconstructed. This graph is depicted in Figure 2(a). Since Gis acyclic, the SOCP relaxation is exact for all values ofa, b, c. Note that this does not imply that every solution Wof the SOCP relaxation has rank 1. However, there is a simplesystematic procedure for recovering a rank-1 solution froman arbitrary optimal solution of this relaxation.

Now, consider the constrained optimization case where aset of constraints

fj(x1, x2) = x41+ajx22+bjx

21x2+cjx1x2 ≤ dj j = 1, ...,m

is added to the optimization problem (9) for given coeffi-cients aj , bj , cj , dj . In this case, the graph G depicted inFigure 2(a) needs to be modified by replacing its edgesets b and c with b, b1, ..., bm and c, c1, ..., cm,

respectively. The SOCP and SDP relaxations correspondingto the new optimization problem are exact as long as the setsc, c1, ..., cm and b, b1, ..., bm are both sign definite.

Example 2: Consider the optimization problem

minx∈C7

x∗Mx s.t. |xi| = 1, i = 1, 2, ..., 7 (12)

where M is a given Hermitian matrix. Assume that theweighted graph G depicted in Figure 2(b) captures the struc-ture of this optimization problem, meaning that (i) Mij = 0for every pair (i, j) ∈ 1, 2, ...7 such that (i, j) 6∈ G,(j, i) 6∈ G and i 6= j, (ii) Mij is equal to the edge weight cijfor every (i, j) ∈ G. The SDP relaxation of this optimizationproblem is as follows:

minW∈H7

traceMW

s.t. W11 = · · · = W77 = 1,

W 0

Define O1 and O2 as the cycles induced by the vertexsets 1, 2, 3 and 1, 4, 5, respectively. Now, the reducedSDP and SOCP relaxations can be obtained by replacingthe constraint W 0 in the above optimization problemwith certain small-sized constraints based on O1 and O2, asmentioned before. The following statements hold:• The SDP, reduced SDP and SOCP relaxations are all

exact in the case where c12, c13, c14, c15, c23, c45 are realnumbers satisfying the inequalities c12c13c23 ≤ 0 andc14c15c45 ≤ 0.

• The SDP, reduced SDP and SOCP relaxations are allexact in the case where each of the sets c12, c13, c23and c14, c15, c45 has at least one zero element.

• The SDP and reduced SDP are exact in the case wherec12, c13, c14, c15, c23, c45 are imaginary numbers. Notethat the SOCP relaxation may not be tight. To illustratethis fact, assume that the weights of the graph G are allequal to +i and that the diagonal entries of the matrixM are zero. In this case, the SDP relaxation is known tobe tight, but the optimal objective values of the SOCPand SDP relaxations are equal to two different numbers-16 and -14.3923. Hence, the SOCP relaxation cannotbe exact.

The above results demonstrate how the combined effect ofthe graph topology and the edge weights makes variousrelaxations exact for the quadratic optimization problem (12).

Example 3: Consider the optimization problem

minx∈Cn

x∗Mx s.t. |xj | = 1, j = 1, 2, ...,m

(13)where M is a symmetric real-valued matrix. It has beenproven in [19] that this problem is NP-hard even in the casewhen M is restricted to be positive semidefinite. Consider thegraph G associated with the matrix M . The SDP and reducedSDP relaxations are exact for this optimization problem andtherefore this problem is polynomial-time solvable with anarbitrary accuracy, provided that G is bipartite and weaklycyclic. To understand how well the SDP relaxation works,

we pick G as a cycle with 4 vertices. Consider a randomlygenerated matrix M :

M =

0 −0.0961 0 −0.1245

−0.0961 0 −0.1370 00 −0.1370 0 0.7650

−0.1245 0 0.7650 0

After solving the SDP relaxation numerically, an optimalsolution Wopt is obtained as

Wopt =

1.0000 0.1767 −0.5516 0.65050.1767 1.0000 0.7235 −0.6327−0.5516 0.7235 1.0000 −0.99230.6505 −0.6327 −0.9923 1.0000

This matrix has rank-2 and thus it seems as if the SDPrelaxation is not exact. However, the fact is that this relax-ation has a hidden rank-1 solution. To recover that solution,one can write Wopt as the sum of two rank-1 matrices, i.e.,Wopt = (u1)(u1)∗ + (u2)(u2)∗ for two real vectors u1 andu1. It is straightforward to inspect that the complex-valuedrank-1 matrix (u1 + u2i)(u1 + u2i)∗ is another solution ofthe SDP relaxation. Thus, Wopt = u1 + u2i is an optimalsolution of the optimization problem (13).

Example 4: Consider the optimization problem

minx∈Cn

x∗M0x

s.t. x∗Mjx ≤ 0, j = 1, 2, ...,m

where M0, ....,Mm are symmetric real matrices, while xis an unknown complex vector. Similar to what was donein Example 1, a generalized weighted graph G can beconstructed for this optimization problem. Regardless ofthe edge weights, as long as the graph G is acyclic, theSDP, reduced SDP and SOCP relaxations are all tight. Asa result, this class of optimization problems is polynomial-time solvable with an arbitrary accuracy.

Example 5: As a generalization of linear programs, considerthe non-convex optimization problem

minx∈Rn

k∑i=1

a0iex∗M0ix +

l∑i=k+1

x∗M0ix + b∗0x

s.t.k∑i=1

ajiex∗Mjix +

l∑i=k+1

x∗Mjix + b∗jx ≤ 0

for j = 1, 2, ...,m, where aij’s are scalars, bj’s are n × 1vectors, and Mij’s are n×n symmetric matrices. This prob-lem involves linear terms, quadratic terms, and exponentialterms with quadratic exponents. Using the technique statedin Section II-B, the above optimization problem can bereformulated in terms of the rank-1 matrix xx∗ where x =[

1 x∗]∗

, from which an SDP relaxation can subsequentlybe obtained by replacing the matrix xx∗ with a new matrixvariable W under the constraint W11 = 1. By mappingthe structure of the optimization into a generalized weightedgraph and noticing that ex is an increasing function in x,it can be concluded that the SDP relaxation is exact if thefollowing conditions are all satisfied:

• aji is nonnegative for every j ∈ 0, ...,m and i ∈1, ..., k.

• bj is a non-positive vector for every j ∈ 0, ...,m.• Mji has only non-positive off-diagonal entries for everyj ∈ 0, ...,m and i ∈ 1, ..., l.

III. SPARSE QUADRATIC OPTIMIZATION

In the previous section, we explained that an optimizationwith two favorable structures may be solved through a conicrelaxation: (i) sparsity and (ii) sign-definite coefficients.Although Condition (ii) is satisfied for certain problems(e.g., power optimization problems due to the passivity of apower grid), but it may be restrictive in general. In contrast,“sparsity” is a universal feature in real-world problems. Theobjective of this part is to understand how sparsity affectsthe computational complexity of a problem. More details onthe results to be presented next can be found in [48].

A. Low-Rank Positive Semidefinite Matrix Completion

The low-rank positive semidefinite matrix completionproblem aims to design the unknown entries of a partiallyfilled matrix so that the resulting matrix becomes positivesemidefinite with a minimum rank. This fundamental prob-lem serves as a basis for studying the SDP relaxation forpolynomial optimization problems. To introduce the prob-lem, consider a simple graph G = (V, E) with n verticestogether with a known matrix W ∈ Sn+ (V and E denote thevertex set and edge set of the graph). The goal is to solvethe following optimization problem:

minW∈Sn+

rankW (14a)

s.t. Wij = Wij ∀(i, j) ∈ E (14b)

Wkk = Wkk ∀k ∈ V (14c)

Note that the matrix W inherits the values of its diagonaland off-diagonal entries corresponding to the edges of G fromthe given matrix W. Assume that the above optimization isfeasible. This problem is difficult to tackle due to its non-convex objective function. To reduce the complexity of theproblem, we will propose two convex relaxations based onthe graph notions of OS and treewidth.

Definition 6: Given a graph G = (V, E), let O = oksk=1

be a sequence of vertices of G with s elements. Denote as Gkthe subgraph induced by o1, . . . , ok for k = 1, ..., s. Let G′kbe the connected component of Gk containing ok. O is calledan OS-vertex sequence of G if, for every k ∈ 1, ..., s, thereexists a vertex wk ∈ V with the following three properties:

1) wk is a neighbor of ok, i.e., (ok, wk) ∈ E2) wk does not belong to the set o1, o2, ..., ok3) wk is not connected to any vertex in G′k other than ok

Denote the maximum cardinality among all OS-vertex se-quences of G as OS(G) [49] .

Figure 3 shows the construction of a maximal OS-vertexsequence of the Petersen graph. Dashed lines and bold lineshighlight nonadjacency and adjacency, respectively, to showhow wk at each step satisfies the conditions of Definition 6.

Fig. 3: A maximal OS-vertex sequence for the Petersen graph

Fig. 4: A maximal OS-vertex sequence for a tree

Figure 4 illustrates the procedure of finding a maximal OS-vertex sequence for a tree. The connected component of eachok in the subgraph induced by o1, . . . , ok is also shown.Notice that although w2 is connected to o1, it is a validchoice because o1 and o2 do not share the same connectedcomponent in G2. To develop a convex relaxation for thematrix completion problem (14), let Gc = (Vc, Ec) denotean arbitrary graph such that Vc = V and Ec ∩ E = φ.

Convex Relaxation I: This problem is defined as

minW∈Sn+

∑(i,j)∈Ec

tij Wij (15a)

s.t. Wij = Wij ∀(i, j) ∈ E (15b)

Wkk = Wkk ∀k ∈ V (15c)

where tij’s are arbitrary nonzero scalars.Assume that the above problem has a strictly feasible

(positive definite) point W (see [48] for a discussion onthe removal of this mild assumption). Then, every solutionof Convex Relaxation I, denoted as Wopt, satisfies theinequality

rankWopt ≤ n−minGs

OS(Gs ∪ Gc)

∣∣∣∣ Gs ⊆ G (16)

where

• The notation Gs ⊆ G means that Gs is a graph with nvertices whose edge set is a subset of the edge set ofG.

• Gs ∪ Gc denotes the edge-wise union of the graphs Gsand Gc.

Note that the inequality (16) holds for all possible nonzerovalues of the coefficients tij’s. Hence, the convex optimiza-tion (15) provides a suboptimal solution for the non-convexproblem (14) together with an upper bound on its optimalobjective value. Roughly speaking, a suitable choice of Gcmakes the upper bound n−minGs⊆G OS(Gs∪Gc) very smallfor a large class of spare graphs G’s. To elaborate on thisstatement, we render the notion of tree decomposition.

Definition 7: Given a graph G = (V, E), a tree T iscalled a tree decomposition of G if it satisfies the followingproperties:

1) Every node of T corresponds to and is identified by asubset of V . Alternatively, each node of T is regardedas a group of vertices of G.

2) Every vertex of G is a member of at least one node ofT .

3) Tk is a connected graph for k = 1, 2, ..., n, where Tkdenotes the subgraph of T induced by all nodes of Tcontaining the vertex k of G.

Fig. 5: A minimal tree decomposition for a ladder

4) The subtrees Ti and Tj have a node in common forevery (i, j) ∈ E .

The width of a tree decomposition is the cardinality ofits biggest node minus one (recall that each node of T isindeed a set containing a number of vertices of G). Thetreewidth of G is the minimum width over all possible treedecompositions of G and is denoted by tw(G).

Note that the treewidth of a tree is equal to 1. Figure 5shows a graph G with 6 vertices named a, b, c, d, e, f , to-gether with its minimal tree decomposition T . Every nodeof T is a set containing three members of V . The widthof this decomposition is therefore equal to 2. A large classof real-world graphs are believed to have small treewidthnumbers. As an example, we will verify the treewidth of Gfor two different applications later in this paper.

Given a tree decomposition T of the graph G with widtht, with no loss of generality assume that all nodes of Thave the same cardinality (see [48] for the general case). Wedesign a graph Gc according to the following procedure:• Step 1: Initialize T c as T and Gc as a graph with n

vertices and no edges.• Step 2: Identify a node of T c with degree 1 (i.e., a leaf).

Let V1 and V2 denote this node and its unique neighborin T c (note that V1, V2 ∈ V because each node of T cis indeed a collection of vertices of G)

• Step 3: Let p1, ..., pg and q1, ..., qg represent thesets V1 − V2 and V2 − V1, respectively. Add the edges(p1, q1),...,(pg, qg) to Gc and then remove node V1 fromT c.

• Step 4: Jump to Step 2 if T c is still nonempty.It can be shown that

n−minGs

OS(Gs ∪ Gc)

∣∣∣∣ Gs ⊆ G ≤ t+ 1 (17)

for this choice of Gc. Hence, the convex Optimization (15) isable to provide a suboptimal solution for Optimization (14)with the property that rankWopt ≤ t+1. In particular, if anoptimal tree decomposition is deployed for the constructionof Gc, then rankWopt ≤ tw(G) + 1 for all nonzerovalues of the coefficients tij’s. Note that the existence ofa solution for Optimization (14) of rank at most tw(G) + 1has already been proved in [50] for real-valued problems,but the technique stated above works for both real andcomplex problems. In addition, the above technique designsinfinitely many optimization problems, each of which returnssuch a solution. The importance of this result will becomeclear later in the paper. The problem of finding a tree

decomposition of minimum width is NP-complete in general[51]. Nevertheless, for a fixed integer t, the problem ofchecking the existence of a tree decomposition of width tand finding such a decomposition (if any) can be solvedin linear time [52], [53]. It is interesting to note that thetreewidth for the optimal decentralized control problem (tobe stated later in the paper) is equal to 2 due to its extremesparsity.

Assume that G is a large-scale graph with no clear sparsitypattern. In this case, it may be very difficult to find a goodtree decomposition or directly design a subgraph Gc mini-mizing the upper bound n −minGs⊆G OS(Gs ∪ Gc). Underthis circumstance, we introduce another convex relaxationfor Optimization (14).

Convex Relaxation II: This optimization problem is definedas

minW∈Hn

+

∑(i,j)∈E∪Ec

tij ImWij (18a)

s.t. ReWij = Wij ∀(i, j) ∈ E (18b)

Wkk = Wkk ∀k ∈ V (18c)

with nonzero coefficients tij’s, where the variable of theoptimization is a complex-valued matrix. Assume that theabove problem has a strictly feasible (positive definite) pointW. Let Wopt denote an arbitrary solution of this optimiza-tion. The matrix ReWopt turns out to be a suboptimalsolution of the matrix completion problem (14) satisfyingthe inequality

rankReWopt ≤ 2(n− OS (G ∪ Gc)

)(19)

At the cost of adding the factor 2, the bound provided in (19)is simpler than the one given in (16) due to obviating the needto take the minimum of OS(·) over a set of subgraphs Gs’s.The above bound is quite useful because it is a small numberfor a large class of sparse graphs, even in the case where Gcis considered as a trivial graph with no edges. This boundreduces to 4 for the optimal distributed control problem tobe studied later in the paper.

The bounds provided in (16) and (19) can both be im-proved (for non-chordal graphs) by supplanting OS(·) withmsr(·), where “msr” stands for the minimum semidefiniterank of a graph [54]–[56]. Indeed, msr(G) is equal to thesmallest rank of all positive semidefinite matrices with thesame support as the adjacency matrix of G.

B. Sparse Quadratic OptimizationConsider the standard non-convex quadratically-

constrained quadratic program (QCQP):

minx∈Rn−1

x∗A0x + 2b∗0x + c0 (20a)

s.t. x∗Akx + 2b∗kx + ck ≤ 0 for k = 1, . . . ,m(20b)

where Ak ∈ R(n−1)×(n−1), bk ∈ Rn−1 and ck ∈ R, fork = 0, . . . ,m. Define

Mk ,

[ck b∗kbk Ak

](21)

Each function fk has the linear representation fk(x) =traceMkW where

W , [1 x∗]∗[1 x∗] (22)

Conversely, an arbitrary matrix W ∈ Sn can be factorizedas (22) if it satisfies three properties: W11 = 1, W 0,and rankW = 1. Therefore, optimization (20) can bereformulated as follows:

minW∈Sn

traceM0W

s.t. traceMkW ≤ 0 for k = 1, . . . ,m

W11 = 1

W 0

rankW = 1

(23)

In the above reformulation of QCQP, the constraintrankW = 1 carries all the non-convexity. Neglecting thisconstraint yields an SDP relaxation [57], [58]. The existenceof a rank-1 solution for this SDP relaxation guaranteesthe equivalence between the original QCQP and its relaxedproblem. Let W denote an arbitrary solution of the SDPrelaxation of optimization (20). It is straightforward to verifythat W may become full rank and yet there would exist alow-rank solution. Indeed, it can naturally occur that the SDPrelaxation will have infinitely many solutions and thereforea solution with the lowest rank should be sought.

Low-Rank Solution: In an effort to find a low-rank SDPsolution, let G = (V, E) be a graph with n vertices suchthat (i, j) ∈ G if and only if the (i, j) entry of at leastone of the matrices M0,M1, ...,Mm is nonzero, for every1 ≤ i < j ≤ n. The graph G captures the sparsity of theoptimization (20). Observe that those off-diagonal entries ofW that correspond to non-existent edges of G play no directrole in the SDP relaxation. As a result, it can be inferred thatevery solution Wopt to the matrix completion problem (14)or its convex relaxations (15) and (18) is also a solution tothe SDP relaxation of the QCQP problem (20). Dependingon the choice of Gc in (15) and (18), different low-ranksolutions of the SDP relaxation can be generated for a sparsegraph G. In particular, there are infinitely many optimizationproblems with linear objectives, each of which generatesa solution Wopt of the SDP relaxation with rank at mosttwG+ 1, provided the optimal tree decomposition of G isknown. Without taking advantage of a tree decomposition,we can generate a solution with rank at most 2

(n−OS(G)

)in polynomial time (note that this solution can be foundefficiently, even though computing the theoretical upperbound on its rank would be an NP-hard problem).

Penalized SDP Relaxation: The strategy delineated aboveconsists of two steps: (i) finding an arbitrary (potentiallyhigh-rank) solution W of the SDP relaxation for QCQP,and (ii) turning the solution into a lower rank solutionWopt by solving a second convex optimization based onthe matrix completion approach. As will become clear, itis advantageous to integrate these two steps. This will be

carried out in the sequel. Consider the convex optimization

minW∈Sn+

traceM0W+ ε1traceW+ ε2∑

(i,j)∈Ectij Wij

s.t. traceMkW ≤ 0 for k = 1, . . . ,m

W11 = 1(24)

for a given graph Gc, a scalar ε1, and nonzero numbers ε2and tij’s. Notice that the objective of this optimization hastwo penalty terms: (i) a trace term motivated by the nuclearnorm technique for rank compensation, and (ii) a weightedsum of some off-diagonal entries of W motivated by thematrix completion approach described earlier. Assume thatSlater’s condition holds for Optimization (24) and its dual.As before, every solution Wopt of the above penalized SDPproblem satisfies the inequality

rankWopt ≤ n− minGs⊆G

OS(Gs ∪ Gc) (25)

where the right side of the inequality can be replaced byt + 1 if Gc is constructed from a tree decomposition ofG with width t such that all nodes are of the identicalsize. Note that the penalized SDP may become arbitrarilyclose to the SDP problem by making ε1 sufficiently smallor equal to zero. This means that an ε-approximation of alow-rank solution of the SDP relaxation of QCQP can beobtained through the penalized SDP problem. In other words,the proposed penalization eliminates high-rank solutions ofthe SDP relaxation. A similar penalization technique can bederived based on Optimization (18), leading to the upperbound 2

(n − OS (G ∪ Gc)

)on the rank of all solutions of

the corresponding penalized SDP.Consider a QCQP problem whose underlying sparsity

graph G has a relatively small treewidth. The above penalizedconvex relaxation generates only low-rank solutions for aninfinite choice of coefficients ε1, ε2 and tij’s. Our simula-tions on thousands of energy optimization and decentralizedcontrol problems suggest that it is possible to generate anear-global rank-1 solution by meticulously devising tij’sand tuning the regularization parameters ε1 and ε2 [31], [40],[59], [60].

IV. GENERAL POLYNOMIAL OPTIMIZATION

The preceding subsection provided some results on study-ing a sparse QCQP based on the graph notions of OS andtreewidth. A question arises as to whether the proposedapproach can be applied to a dense QCQP or a generalpolynomial optimization. To address this problem, considerthe optimization

minx∈Rn

f0(x)

s.t. fi(x) ≤ 0, i = 1, ...,m(26)

where f0, ..., fm are arbitrary polynomial functions. Assumethat the above problem is feasible. This class of optimizationincludes discrete optimization problems (xi = ±1 can be castas x2i = 1), and can approximate almost every continuousoptimization problem using a Taylor series expansion. The

above optimization problem can be converted to a QCQPvia introducing slack parameters and imposing additionalconstraints [2]. For instance, the polynomial x21 +x22x3 withthree variables can be expressed as x21 + x4x3 subject tothe additional constraint x22 = x4 with a new variable x4.This polynomial can also be cast as x21 + x2x4 subject tox4 = x2x3. More precisely, every polynomial optimizationcan be formulated as a QCQP by increasing the dimensionof the problem by a logarithmic factor. This will lead to aQCQP problem of the form (20), but perhaps with no sparsitystructure. In what follows, we will explain how the resultingdense QCQP can be sparsified. The details may be foundin [61].

Let G = (V, E) represent the sparsity graph (i.e., general-ized weighted graph) of the QCQP, which may be as sparseas a tree or as dense as a complete graph. Consider a vertexi of the graph G with degree at least 2 and reformulate theQCQP problem according to the following procedure.

Vertex Duplication Procedure: Perform the following ac-tions:• Replace the variable xi of QCQP with two new vari-

ables xi1 and xi2 .• Divide the neighboring nodes of vertex i in the graphG into two arbitrary sets, denoted as V1(i) and V2(i).

• For every j such that (i, j) ∈ E , replace all occurrencesof the term xixj in the objective and constraints of theoptimization with xi1xj if j ∈ V1(i) and with xi2xj ifj ∈ V2(i).

• Add the additional consistency constraint xi1 = xi2 tothe optimization.

Note that the idea of duplicating a parameter has beenextensively utilized in distributed computation. The aboveprocedure modifies the graph G as follows:• Vertex i is replaced by two new vertices i1 and i2.• Vertices i1 and i2 are connected to the neighboring

subsets V1(i) and V2(i), respectively.• Vertices i1 and i2 are both connected to vertex 1 to

account for the consistency constraint (note that xi1 =xi2 is indeed equal to x1xi1 = x1xi2 as x1 plays therole of number 1).

As can be seen, the above procedure aims to sparsify thegraph and reduce its treewidth. In particular, if the aboveprocedure is repeated a sufficient number of times (on theorder of the number of edges of G), then all edges (i, j) ofG with i, j ≥ 2 will become isolated and hence the resultinggraph will have treewidth t = 2 (because the removal ofvertex 1 in the new graph eliminates all cycles of the resultinggraph). This implies that every polynomial optimization canbe converted to an equivalent sparse QCQP whose SDPrelaxation has a solution Wopt with rank at most t+ 1 = 3(see Subsection III-B). A question arises as to whether thereexists a relaxation for which rankWopt ≤ 2. To addressthis question, consider the following procedure for an edge(i, j) ∈ G.

Edge Elimination Procedure: Perform the following ac-

tions:• Add two auxiliary variables z1 and z2.• Impose the additional constraints:

z1 =xi + xj

2, z2 =

xi − xj2

(27)

• Replace every instance of the product xixj in the QCQPproblem with z21 − z22 .

The above procedure eliminates the edge (i, j) and itsrepetition makes the resulting graph have treewidth t = 1.Hence, the SDP relaxation corresponding to the obtainedsparse QCQP has a matrix solution with rank 1 or 2. Thisresult has two implications:

i) The NP-hardness of various subclasses of polynomialoptimization, e.g., combinatorial optimization, is onlyrelated to the existence of a not rank-1 but low-rankSDP solution, where the upper bound on the rank isconstant and does not depend on the size of the originaloptimization.

ii) By approximating the low-rank solution of the SDPrelaxation with a rank-1 matrix, an approximate solutionof the original problem may be obtained whose close-ness to the global solution can also be upper bounded.

These results offer a new insight into the computationalcomplexity of polynomial optimization (Property (i)) andenable to seek a near-global solution (Property (ii)).

There are two important parameters associated with anSDP relaxation: (i) optimal cost serving as a lower boundon the optimal value of the QCQP, and (ii) minimum rankof the SDP solution. It can be shown that the proposedsparsification technique leads to a non-unique hierarchy ofSDP relaxations, which does not improve the lower boundbut reduces the rank. In general, since there are many waysto sparsify the graph (by determining which vertex of G tochoose and how to partition its edges), it is imperative toperform the sparsification in such a way that the optimalvalue of SDP relaxation is not decreased noticeably and yetthe rank is improved significantly.

Assume that we have designed an SDP relaxation possess-ing a low-rank matrix solution Wopt. Now, three strategiescould be taken to find a near-global (sub-optimal) solutionof the original QCQP:• Since Wopt has only a few undesirable (nonzero)

eigenvalues, it may be converted to an approximatesolution via a local search algorithm. Based on theeigenvalue decomposition, it is straightforward to designan iterative algorithm with the property that the rankof the solution does not increase at any iteration ofthe algorithm. This leads to a sequence of low-rankmatrices, which tends to converge to a rank-1 solution.The obtained solution may ultimately need to be ap-proximated by a rank-1 matrix if it is not ultimatelyrank-1.

• The unwanted nonzero eigenvalues of Wopt may beeliminated by means of a penalization technique suchas the one mentioned earlier.

• Another technique is to directly approximate Wopt witha rank-1 matrix by solving a convex optimization.

V. CASE STUDY: OPTIMAL DISTRIBUTED CONTROL

Consider the problem of designing an optimal distributedcontroller for a multi-channel deterministic or stochasticsystem, where the optimality is measured with respect toa linear-quadratic, H2, or H∞ performance index. It hasbeen long known that this problem is computationally hardand, in particular NP-hard in the worst case [62]–[64].Great effort has been devoted to investigating this highlycomplex problem for special types of systems, includingspatially distributed systems [65]–[69], dynamically decou-pled systems [70], [71], weakly coupled systems [72], andstrongly connected systems [73]. Another special case thathas received considerable attention is the design of an opti-mal static distributed controller [74], [75]. Early approachesfor the optimal decentralized control problem were basedon parameterization techniques [76], [77], which were thenevolved into matrix optimization methods [78], [79].

Due to the recent advances in the area of convex optimiza-tion, the focus of the existing research efforts has shiftedfrom deriving a closed-form solution for the above controlsynthesis problem to finding a convex formulation of theproblem that can be efficiently solved numerically [80]–[82]. This has been carried out in the seminal work [83] byderiving a sufficient condition named quadratic invariance,which has been generalized in [84] by deploying the conceptof partially order sets. These conditions have been furtherinvestigated in several other papers [85]–[87]. A differentapproach is taken in the recent papers [88], [89], where ithas been shown that the distributed control problem can becast as a convex optimization for positive systems.

A. Time-Domain Formulation

Consider the linear discrete-time systemx[τ + 1] = Ax[τ ] +Bu[τ ]

y[τ ] = Cx[τ ], ∀τ ∈ Z+ (28)

where x[τ ] ∈ Rn, u[τ ] ∈ Rm and y[τ ] ∈ Rr denotethe state, input and output of the system, respectively. Thesystem matrices A,B,C and the initial state x[0] are allknown. The goal is to design a fixed-order dynamic controllerwith a pre-specified structure to minimize the cost function∑pτ=0(x[τ ]∗Q x[τ ] + u[τ ]∗Ru[τ ]) for an arbitrary terminal

time p and positive definite matrices Q and R. To solve thisoptimal distributed control (ODC) problem, we denote theunknown controller as

z[τ + 1] = Acz[τ ] +Bcy[τ ]u[τ ] = Ccz[τ ] +Dcy[τ ]

, ∀τ ∈ Z+ (29)

where z[τ ] ∈ Rnc represents the state of the controller, nc de-notes its known degree, and the quadruple (Ac, Bc, Cc, Dc)needs to be designed. Since the controller is requiredto have a pre-determined distributed structure, the 4-tuple(Ac, Bc, Cc, Dc) must belong to some given polytope K.This polytope enforces certain entries of Ac, Bc, Cc, and

yr[1]

yr[2]

yr[p]

y2[1]

y2[2]

y2[p]

K11 K22 Krr

y1[1]

y1[2]

y1[p]

um[1]

um[2]

um[p]

u2[1]

u2[2]

u2[p]

u1[1]

u1[2]

u1[p]

xn[1]

xn[2]

xn[p]

x2[1]

x2[2]

x2[p]

x1[1]

x1[2]

x1[p]

Output

Input

State

Controller

Fig. 6: The graph showing the sparse nonlinearity of ODCfor the time-domain formulation (node 1 is not shown in thegraph due to its connection to almost all nodes of the graph).

Dc to be zero. ODC is a nonlinear optimization becausethe dynamics of the controller has some non-convex termssuch as Acz[τ ] and Bcy[τ ]. In order to solve the above ODCproblem, define:

v :=[

1 h∗ x[0]∗ · · · x[p]∗ y[0]∗ · · · y[p]∗

u[0]∗ · · · u[p]∗ z[0]∗ · · · z[p]∗]∗

where h denotes a vector consisting of all free (nonzero)entries of the matrices Ac, Bc, Cc, and Dc. The ODCproblem can be cast as a quadratic optimization with respectto the vector v and as a linear optimization in the matrix vv∗.Hence, an SDP relaxation of this problem can be derived byreplacing vv∗ with a new matrix variable W.

B. Lyapunov-Domain Formulation

The previous formulation of the ODC problem was re-stricted to deterministic systems. Consider now the stochasticsystem

x[τ + 1] = Ax[τ ] +Bu[τ ] + Ed[τ ]y[τ ] = Cx[τ ] + Fv[τ ]

, τ ∈ Z+

(30)where d[τ ] and v[t] are random variables representing thesystem’s disturbance and measurement noise, which areassumed to be zero-mean white-noise random processes. Theobjective is to design a distributed controller with a pre-specified structure for the above system to minimize the costfunction

limτ→+∞

E (x[τ ]∗Qx[τ ] + u[τ ]∗Ru[τ ]) (31)

where E· shows the expectation operator. Since we can nolonger formulate ODC as a QCQP with respect to the time-domain signals (due to unknown disturbances), we resort toa bilinear matrix formulation in terms of Lyapunov matrices.

To explain the idea, consider the special case of designing astatic controller. Define two covariance matrices as below:

Σd = EEd[0]d[0]∗E∗, Σv = EFv[0]v[0]∗F ∗ (32)

The problem of designing an optimal static structured con-troller u[t] = Ky[t] for the system (30) to minimize (31) canbe formulated as the minimization of

tracePΣd +MΣv +K∗RKΣv (33)

subject to the constraintsG G (AG+BL)∗ L∗

G Q−1 0 0AG+BL 0 G 0

L 0 0 R−1

0, (34a)

[P II G

] 0, (34b)[

M (BK)∗

BK G

] 0, (34c)

L = KCG (34d)

where K is a matrix variable with forced zeros in certainentries, P is an unknown Lyapunov matrix, and G ∈ Sn,L ∈ Rn×r and M ∈ Sr are auxiliary matrices. The aboveproblem is non-convex due to the nonlinear term L = KCG.To convexify the problem, define v as a vector consisting ofnumber 1, h, and all entries of the matrix G. Then, everyentry of KCG can be expressed as a quadratic function of v.Hence, the above optimization has a natural SDP relaxationwith a variable W playing the role of vv∗.

C. Existence of Low-Rank Solution

Two quadratic formulations of the ODC problem havebeen proposed for deterministic and stochastic systems. Thegraphs capturing the sparsity of these problems are bothextremely sparse with small treewidth numbers. For example,in the special case of designing a fully decentralized staticcontroller (diagonal K), the graph for the time-domainformulation reduces to a bunch of stars after removing itsnode 1. This fact is illustrated in Figure 6. As a result,the treewidth of this graph is equal to 2, which impliesthat the SDP relaxation of the ODC problem has a low-rank solution with rank at most 3 under this circumstance.The same conclusion holds for the Lyapunov-domain for-mulation. Extensive simulations have been conducted in [40],[59], [60], where it has been verified on random and physicalsystems that the rank-3 solution can be well approximatedby a rank-1 matrix. For instance, several hundred randomsystems were generated in those papers for which near-optimal decentralized controllers with a global optimalitydegree of at least 99% were designed. The same observationwas made for the distributed control of power grids. Thereader may refer to [59], [60] to see how the numericalcomplexity of the proposed SDP relaxations may also bereduced significantly, leading to solving computationally-cheap SDP relaxations.

VI. CASE STUDY: POWER OPTIMIZATION PROBLEMS

In the past five decades, many optimization techniqueshave been studied for the non-convex optimal power flow(OPF) problem, including: linear programming, NewtonRaphson, quadratic programming, nonlinear programming,Lagrange relaxation, interior point methods, artificial in-telligence, artificial neural network, fuzzy logic, geneticalgorithm, evolutionary programming, and particle swarmoptimization [90]–[102]. Most of these methods are built onthe Karush-Kuhn-Tucker (KKT) necessary conditions, whichcan only find a local solution (as opposed to a globallyoptimal solution). Some recent efforts have been focusedon convex optimization [103]–[105]. The practical difficultyof treating the non-linearity of OPF has resulted in mostoptimization formulations resorting to approximations suchas linearization. The problem of discovering new approachesfor avoiding linearization has received a special attention byFederal Energy Regulatory Commission in the past few years[106]–[109].

A. Problem Formulation

Consider an n-bus power network described by a graphG = (V, E), where each vertex belonging to V = 1, . . . , nrepresents a node (bus) of the network and each edgebelonging to E represents a transmission line. Let yij denotethe admittance of the line (i, j) ∈ E . Define V ∈ Cn asthe voltage phasor vector where its component Vk representsthe complex voltage at node k ∈ V . Assume that eachnode of the network is connected to a known load as wellas a generator with an unknown production level. OPFis a resource allocation problem, which aims to optimizethe production levels of the generators. To formulate theproblem, let Pk and Qk denote the net active and reactivepowers injected at node k ∈ G (net power is equal togeneration minus load). OPF can be expressed as:

minV,P,Q∈Cn

∑k∈V

fk(Pk) (35a)

s.t. V mink ≤ |Vk| ≤ V max

k , k ∈ V (35b)

Pmink ≤ Pk ≤ Pmax

k , k ∈ V (35c)

Qmink ≤ Qk ≤ Qmax

k , k ∈ V (35d)ReVi(V ∗i − V ∗j )y∗ij ≤ Pmax

ij , (i, j) ∈ E (35e)

Pk +Qk√−1 =

∑i∈N (k)

Vk(V ∗k − V ∗i )y∗ki, k ∈ V (35f)

where V mink , V max

k , Pmink , Pmax

k , Qmink , Qmax

k and Pmaxij are

network limits, N (k) denotes the set of neighbors of vertexk, and fk(Sk) is a convex function accounting for the powergeneration cost at node k.

B. Existence of Low-Rank Solution

Problem (35) is quadratic in the vector V. Therefore, anSDP relaxation of OPF can be derived by reformulating theproblem in terms of VV∗ and then replacing VV∗with anew variable W. It is known that (see [27]–[30], [32], [110]–[115]):

1

2

3

5

4

7

8

9

6

11

14

13 12

10

6,12,13

1,2,5

2,4,5

2,3,4

4,5,9

6,9,13

4,7,9 7,8

5,6,9

9,13,14

6,9,11 9,10,11

Fig. 7: The IEEE 14-bus test case and its minimal treedecomposition

• The SDP relaxation is exact for IEEE benchmark sys-tems with 14, 30, 57, 118 and 300 buses, several Polishsystems, and many randomly generated power networks.This technique is the first method proposed since theintroduction of the OPF problem in 1962 that is ableto find a provably global solution for certain OPFproblems.

• Under some practical assumptions, the SDP relaxationis exact for distribution networks (acyclic graphs). How-ever, it may not be exact for transmission networks(cyclic graphs) unless there is a sufficient number oftransformers in the network.

• There are OPF problems defined over transmissionnetworks for which the SDP exact is not exact but apenalized SDP relaxation works.

• The sign definite condition stated before holds for powernetworks and this is one of the main reasons behind thesuccess of the SDP relaxation for OPF.

Since the SDP relaxation is not always exact for OPF overtransmission networks, it is highly desirable to seek a near-global solution. We have calculated the treewidth of G forseveral power systems and reported our findings in [115].It can be seen that the treewidth of a Polish network with3375 nodes is at most 28. Figure 7 shows a minimal treedecomposition associated with the IEEE 14 case. As long asthe treewidth is small, the SDP relaxation of OPF will havea low-rank solution for transmission networks, which maybe leveraged to design a near-global solution. This idea hasbeen tested on 7000 instances of OPF in [32], [115].

VII. CONCLUSIONS

This tutorial paper aims to study an arbitrary non-convexpolynomial optimization problem through semidefinite pro-gramming (SDP) relaxations combined with graph-theoreticalgorithms. Three problems are investigated in detail. First,a method is proposed to study how the underlying struc-ture of an optimization problem reduces the computationalcomplexity of the problem. For this purpose, the structureof the optimization is mapped into a weighted graph andit is shown that the SDP relaxation is exact if the graphpossesses certain properties. Second, it is shown that the SDPrelaxation of a sparse optimization problem has a solutionwhose rank can be characterized in terms of the sparsity levelof the problem. Third, it is explained that every polynomialoptimization problem admits a sparse representation whose

SDP relaxation has a rank 1 or 2 matrix solution. Twoengineering applications of these results are also discussed.

REFERENCES

[1] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge,2004.

[2] Y. Nesterov, A. S. Nemirovskii, and Y. Ye, Interior-point polynomialalgorithms in convex programming. SIAM, 1994.

[3] D. P. Bertsekas, “Nonlinear programming,” 1999.[4] A. Ben-Tal and A. S. Nemirovski, Lectures on modern convex

optimization: analysis, algorithms, and engineering applications.SIAM, 2001.

[5] K. G. Murty and S. N. Kabadi, “Some NP-complete problems inquadratic and nonlinear programming,” Mathematical programming,vol. 39, no. 2, pp. 117–129, 1987.

[6] E. D. Klerk, “The complexity of optimizing over a simplex, hy-percube or sphere: a short survey,” Central European Journal ofOperations Research, vol. 16, pp. 111–125, 2008.

[7] G. L. Nemhauser and L. A. Wolsey, Integer and combinatorialoptimization. Wiley New York, 1988.

[8] C. H. Papadimitriou and K. Steiglitz, Combinatorial optimization:algorithms and complexity. Courier Dover Publications, 1998.

[9] S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan, “Linearmatrix inequalities in system and control theory,” Studies in AppliedMathematics, SIAM, 1994.

[10] M. X. Goemans and D. P. Williamson, “Improved approxima-tion algorithms for maximum cut and satisfiability problems usingsemidefinite programming,” Journal of the ACM (JACM), vol. 42,no. 6, pp. 1115–1145, 1995.

[11] J. Briet, F. Vallentin et al., “Grothendieck inequalities for semidefiniteprograms with rank constraint,” arXiv preprint arXiv:1011.1754,2010.

[12] N. Alon, K. Makarychev, Y. Makarychev, and A. Naor, “Quadraticforms on graphs,” Inventiones mathematicae, vol. 163, no. 3, pp.499–522, 2006.

[13] M. Laurent and A. Varvitsiotis, “Computing the grothendieck con-stant of some graph classes,” Operations Research Letters, vol. 39,no. 6, pp. 452–456, 2011.

[14] M. X. Goemans and D. P. Williamson, “Approximation algorithmsfor max-3-cut and other problems via complex semidefinite program-ming,” Journal of Computer and System Sciences, vol. 68, pp. 422–470, 2004.

[15] Y. Nesterov, “Semidefinite relaxation and nonconvex quadratic opti-mization,” Optimization Methods and Software, vol. 9, pp. 141–160,1998.

[16] Y. Ye, “Approximating quadratic programming with bound andquadratic constraints,” Mathematical Programming, vol. 84, pp. 219–226, 1999.

[17] Y. Ye, “Approximating global quadratic optimization with convexquadratic constraints,” Journal of Global Optimization, vol. 15, pp.1–17, 1999.

[18] S. Zhang, “Quadratic maximization and semidefinite relaxation,”Mathematical Programming A, vol. 87, pp. 453–465, 2000.

[19] S. Zhang and Y. Huang, “Complex quadratic optimization andsemidefinite programming,” SIAM Journal on Optimization, vol. 87,pp. 871–890, 2006.

[20] Z. Luo, N. Sidiropoulos, P. Tseng, and S. Zhang, “Approximationbounds for quadratic optimization with homogeneous quadratic con-straints,” SIAM Journal on Optimization, vol. 18, pp. 1–28, 2007.

[21] S. He, Z. Luo, J. Nie, and S. Zhang, “Semidefinite relaxation boundsfor indefinite homogeneous quadratic optimization,” SIAM Journalon Optimization, vol. 19, pp. 503–523, 2008.

[22] S. He, Z. Li, and S. Zhang, “Approximation algorithms for homo-geneous polynomial optimization with quadratic constraints,” Math-ematical Programming, vol. 125, pp. 353–383, 2010.

[23] J. B. Lasserre, “An explicit exact SDP relaxation for nonlinear 0-1programs,” in Integer Programming and Combinatorial Optimization.Springer, 2001, pp. 293–303.

[24] S. Kim and M. Kojima, “Exact solutions of some nonconvexquadratic optimization problems via SDP and SOCP relaxations,”Computational Optimization and Applications, vol. 26, no. 2, pp.143–154, 2003.

[25] S. Sojoudi and J. Lavaei, “On the exactness of semidefinite relaxationfor nonlinear optimization over graphs: Part I,” IEEE Conference onDecision and Control, 2013.

[26] S. Sojoudi and J. Lavaei, “On the exactness of semidefinite relaxationfor nonlinear optimization over graphs: Part II,” IEEE Conference onDecision and Control, 2013.

[27] J. Lavaei and S. H. Low, “Zero duality gap in optimal power flowproblem,” IEEE Transactions on Power Systems, vol. 27, no. 1, pp.92–107, 2012.

[28] J. Lavaei, “Zero duality gap for classical OPF problem convexifiesfundamental nonlinear power problems,” American Control Confer-ence, 2011.

[29] S. Sojoudi and J. Lavaei, “Physics of power networks makes hardoptimization problems easy to solve,” IEEE Power & Energy SocietyGeneral Meeting, 2012.

[30] J. Lavaei, B. Zhang, and D. Tse, “Geometry of power flows in treenetworks,” IEEE Power & Energy Society General Meeting, 2012.

[31] S. Sojoudi, R. Madani, and J. Lavaei, “Low-rank solution of convexrelaxation for optimal power flow problem,” IEEE SmartGridComm,2013.

[32] R. Madani, S. Sojoudi, and J. Lavaei, “Convex relaxation for optimalpower flow problem: Mesh networks,” To appear in IEEE Transac-tions on Power Systems, 2014.

[33] M. Fazel, H. Hindi, and S. Boyd, “Log-det heuristic for matrixrank minimization with applications to hankel and euclidean distancematrices,” American Control Conference, vol. 3, pp. 2156–2162,2003.

[34] B. Recht, M. Fazel, and P. A. Parrilo, “Guaranteed minimum ranksolutions to linear matrix equations via nuclear norm minimization,”SIAM Review, vol. 52, pp. 471–501, 2010.

[35] A. Barvinok, “Problems of distance geometry and convex propertiesof quadartic maps,” Discrete and Computational Geometry, vol. 12,pp. 189–202, 1995.

[36] G. Pataki, “On the rank of extreme matrices in semidefinite pro-grams and the multiplicity of optimal eigenvalues,” Mathematics ofOperations Research, vol. 23, pp. 339–358, 1998.

[37] A. Barvinok, “A remark on the rank of positive semidefinite matricessubject to affine constraints,” Discrete & Computational Geometry,vol. 25, no. 1, pp. 23–31, 2001.

[38] W. Ai, Y. Huang, and S. Zhang, “On the low rank solutions for linearmatrix inequalities,” Mathematics of Operations Research, vol. 33,no. 4, pp. 965–975, 2008.

[39] S. Sojoudi and J. Lavaei, “Exactness of semidefinite relaxations fornonlinear optimization problems with underlying graph structure,” toappear in SIAM Journal on Optimization, 2014.

[40] J. Lavaei, “Optimal decentralized control problem as a rank-constrained optimization,” Allerton, 2013.

[41] J. F. Sturm and S. Zhang, “On cones of nonnegative quadraticfunctions,” Mathematics of Operations Research, vol. 28, pp. 246–267, 2003.

[42] Y. Huang and S. Zhang, “Complex matrix decomposition andquadratic programming,” Mathematics of Operations Research,vol. 32, pp. 758–768, 2007.

[43] W. Ai, Y. Huang, and S. Zhang, “On the low rank solutions for linearmatrix inequalities,” Mathematics of Operations Research, vol. 33,pp. 965–975, 2008.

[44] S. Kim and M. Kojima, “Exact solutions of some nonconvexquadratic optimization problems via SDP and SOCP relaxations,”Computational Optimization and Applications, vol. 26, p. 143154,2003.

[45] S. Bose and K. M. C. D. F. Gayme, S. H. Low, “Quadraticallyconstrained quadratic programs on acyclic graphs with applicationto power flow,” arXiv:1203.5599v1, 2012.

[46] M. Padberg, “The boolean quadric polytope: some characteristics,facets and relatives,” Mathematical programming, vol. 45, no. 1-3,pp. 139–172, 1989.

[47] P. A. Parrilo and B. Sturmfels, “Minimizing polynomial functions,”Algorithmic and quantitative real algebraic geometry, DIMACS Se-ries in Discrete Mathematics and Theoretical Computer Science,vol. 60, pp. 83–99, 2003.

[48] R. Madani, G. Fazelnia, S. Sojoudi, and J. Lavaei, “Low-ranksolutions of matrix inequalities with applications to polynomial opti-mization and matrix completion problems,” Conference on Decisionand Control, 2014.

[49] P. Hackney, B. Harris, M. Lay, L. H. Mitchell, S. K. Narayan, andA. Pascoe, “Linearly independent vertices and minimum semidefiniterank,” Linear Algebra and its Applications, vol. 431, no. 8, pp. 1105–1115, 2009.

[50] M. Laurent and A. Varvitsiotis, “A new graph parameter related tobounded rank positive semidefinite matrix completions,” Mathemat-ical Programming, pp. 1–35, 2012.

[51] S. Arnborg, D. G. Corneil, and A. Proskurowski, “Complexity offinding embeddings in ak-tree,” SIAM Journal on Algebraic DiscreteMethods, vol. 8, no. 2, pp. 277–284, 1987.

[52] J. Matousek and R. Thomas, “Algorithms finding tree-decompositionsof graphs,” Journal of Algorithms, vol. 12, no. 1, pp. 1–22, 1991.

[53] H. L. Bodlaender, “A linear-time algorithm for finding tree-decompositions of small treewidth,” SIAM Journal on computing,vol. 25, no. 6, pp. 1305–1317, 1996.

[54] S. M. Fallat and L. Hogben, “The minimum rank of symmetricmatrices described by a graph: a survey,” Linear Algebra and itsApplications, vol. 426, no. 2, pp. 558–582, 2007.

[55] W. Barrett, H. van der Holst, and R. Loewy, “Graphs whose minimalrank is two,” Electronic Journal of Linear Algebra, vol. 11, no. 258-280, p. 687, 2004.

[56] J. Sinkovic and H. van der Holst, “The minimum semidefiniterank of the complement of partial k-trees,” Linear Algebra and itsApplications, vol. 434, no. 6, pp. 1468–1474, 2011.

[57] L. Vandenberghe and S. Boyd, “Semidefinite programming,” SIAMreview, vol. 38, no. 1, pp. 49–95, 1996.

[58] K. M. Anstreicher, “On convex relaxations for quadratically con-strained quadratic programming,” Mathematical programming, vol.136, no. 2, pp. 233–251, 2012.

[59] G. Fazelnia, R. Madani, and J. Lavaei, “Convex relaxation for optimaldistributed control problem,” Conference on Decision and Control,2014.

[60] A. Kalbat, R. Madani, G. Fazelnia, and J. Lavaei, “Efficient convexrelaxation for stochastic optimal distributed control problem,” Aller-ton, 2014.

[61] R. Madani, G. Fazelnia, and J. Lavaei, “Rank-2 matrix solutionfor semidefinite relaxations of arbitrary polynomial optimizationproblems,” 2014. [Online]. Available: http://www.ee.columbia.edu/∼lavaei/CDC Polynomial 2014.pdf

[62] H. Witsenhausen, “A counterexample in stochastic optimum control,”SIAM Journal on Control, vol. 6, no. 1, pp. 131–147, 1968.

[63] C. Papadimitriou and J. Tsitsiklis, “Intractable problems in controltheory,” SIAM Journal on Control and Optimization, vol. 24, no. 4,pp. 639–654, 1986.

[64] V. D. Blondel and J. N. Tsitsiklis, “A survey of computationalcomplexity results in systems and control,” Automatica, vol. 36, no. 9,pp. 1249 – 1274, 2000.

[65] R. D’Andrea and G. Dullerud, “Distributed control design forspatially interconnected systems,” IEEE Transactions on AutomaticControl, vol. 48, no. 9, pp. 1478–1495, 2003.

[66] B. Bamieh, F. Paganini, and M. A. Dahleh, “Distributed controlof spatially invariant systems,” IEEE Transactions on AutomaticControl, vol. 47, no. 7, pp. 1091–1107, 2002.

[67] C. Langbort, R. Chandra, and R. D’Andrea, “Distributed controldesign for systems interconnected over an arbitrary graph,” IEEETransactions on Automatic Control, vol. 49, no. 9, pp. 1502–1519,2004.

[68] N. Motee and A. Jadbabaie, “Optimal control of spatially distributedsystems,” IEEE Transactions on Automatic Control, vol. 53, no. 7,pp. 1616–1629, 2008.

[69] G. Dullerud and R. D’Andrea, “Distributed control of heterogeneoussystems,” IEEE Transactions on Automatic Control, vol. 49, no. 12,pp. 2113–2128, 2004.

[70] T. Keviczky, F. Borrelli, and G. J. Balas, “Decentralized recedinghorizon control for large scale dynamically decoupled systems,”Automatica, vol. 42, no. 12, pp. 2105–2115, 2006.

[71] F. Borrelli and T. Keviczky, “Distributed LQR design for identicaldynamically decoupled systems,” IEEE Transactions on AutomaticControl, vol. 53, no. 8, pp. 1901–1912, 2008.

[72] D. D. Siljak, “Decentralized control and computations: status andprospects,” Annual Reviews in Control, vol. 20, pp. 131–141, 1996.

[73] J. Lavaei, “Decentralized implementation of centralized controllersfor interconnected systems,” IEEE Transactions on Automatic Con-trol, vol. 57, no. 7, pp. 1860–1865, 2012.

[74] M. Fardad, F. Lin, and M. R. Jovanovic, “On the optimal design ofstructured feedback gains for interconnected systems,” in 48th IEEEConference on Decision and Control, 2009, pp. 978–983.

[75] F. Lin, M. Fardad, and M. R. Jovanovic, “Augmented lagrangianapproach to design of structured optimal state feedback gains,” IEEETransactions on Automatic Control, vol. 56, no. 12, pp. 2923–2929,2011.

[76] J. Geromel, J. Bernussou, and P. Peres, “Decentralized controlthrough parameter space optimization,” Automatica, vol. 30, no. 10,pp. 1565 – 1578, 1994.

[77] R. A. Date and J. H. Chow, “A parametrization approach to optimalH2 and H∞ decentralized control problems,” Automatica, vol. 29,no. 2, pp. 457 – 463, 1993.

[78] G. Scorletti and G. Duc, “An LMI approach to dencentralized H∞control,” International Journal of Control, vol. 74, no. 3, pp. 211–224, 2001.

[79] G. Zhai, M. Ikeda, and Y. Fujisaki, “Decentralized H∞ controllerdesign: a matrix inequality approach using a homotopy method,”Automatica, vol. 37, no. 4, pp. 565 – 572, 2001.

[80] G. A. de Castro and F. Paganini, “Convex synthesis of localizedcontrollers for spatially invariant systems,” Automatica, vol. 38, no. 3,pp. 445 – 456, 2002.

[81] B. Bamieh and P. G. Voulgaris, “A convex characterization ofdistributed control problems in spatially invariant systems with com-munication constraints,” Systems & Control Letters, vol. 54, no. 6,pp. 575 – 583, 2005.

[82] X. Qi, M. Salapaka, P. Voulgaris, and M. Khammash, “Structuredoptimal and robust control with multiple criteria: a convex solution,”IEEE Transactions on Automatic Control, vol. 49, no. 10, pp. 1623–1640, 2004.

[83] M. Rotkowitz and S. Lall, “A characterization of convex problemsin decentralized control,” IEEE Transactions on Automatic Control,vol. 51, no. 2, pp. 274–286, 2006.

[84] P. Shah and P. A. Parrilo, “H2-optimal decentralized control overposets: a state-space solution for state-feedback,” http://arxiv.org/abs/1111.1498, 2011.

[85] L. Lessard and S. Lall, “Optimal controller synthesis for the decen-tralized two-player problem with output feedback,” American ControlConference, 2012.

[86] A. Lamperski and J. C. Doyle, “Output feedback H2 model matchingfor decentralized systems with delays,” American Control Confer-ence, 2013.

[87] M. Rotkowitz and N. Martins, “On the nearest quadratically invariantinformation constraint,” IEEE Transactions on Automatic Control,vol. 57, no. 5, pp. 1314–1319, 2012.

[88] T. Tanaka and C. Langbort, “The bounded real lemma for internallypositive systems and H-infinity structured static state feedback,” IEEETransactions on Automatic Control, vol. 56, no. 9, pp. 2218–2223,2011.

[89] A. Rantzer, “Distributed control of positive systems,” http://arxiv.org/abs/1203.0047, 2012.

[90] J. Carpentier, “Contribution to the economic dispatch problem,”Bulletin Society Francaise Electriciens, 1962.

[91] R. Baldick, Applied Optimization: Formulation and Algorithms forEngineering Systems. Cambridge, 2006.

[92] M. Huneault and F. Galiana, “A survey of the optimal power flowliterature,” IEEE Transactions on Power Systems, vol. 6, no. 2, pp.762–770, 1991.

[93] R. A. Jabr, A. H. Coonick, and B. J. Cory, “A primal-dual interiorpoint method for optimal power flow dispatching,” IEEE Transactionson Power Systems, vol. 17, no. 3, pp. 654–662, 2002.

[94] G. L. Torres and V. H. Quintana, “Optimal power flow by a nonlinearcomplementarity method,” IEEE Transactions on Power Systems,vol. 15, no. 3, pp. 1028–1033, 2000.

[95] H. Wang, C. E. Murillo-Sanchez, R. D. Zimmerman, and R. J.Thomas, “On computational issues of market-based optimal powerflow,” IEEE Transactions on Power Systems, vol. 22, no. 3, pp. 1185–1193, 2007.

[96] J. A. Momoh, M. E. El-Hawary, and R. Adapa, “A review of selectedoptimal power flow literature to 1993. Part I: Nonlinear and quadraticprogramming approaches,” IEEE Transactions on Power Systems,1999.

[97] J. A. Momoh, M. E. El-Hawary, and R. Adapa, “A review ofselected optimal power flow literature to 1993. Part II: Newton,

linear programming and interior point methods,” IEEE Transactionson Power Systems, 1999.

[98] H. Wei, H. Sasaki, J. Kubokawa, and R. Yokoyama, “An interiorpoint nonlinear programming for optimal power flow problems witha novel data structure,” IEEE Transactions on Power Systems, vol. 13,no. 3, pp. 870–877, 1998.

[99] W. M. Lin, C. H. Huang, and T. S. Zhan, “A hybrid current-poweroptimal power flow technique,” IEEE Transactions on Power Systems,vol. 23, no. 1, pp. 177–185, 2008.

[100] Q. Y. Jiang, H. D. Chiang, C. X. Guo, and Y. J. Cao, “Power-currenthybrid rectangular formulation for interior-point optimal power flow,”IET Generation, Transmission & Distribution, vol. 3, no. 8, pp. 748–756, 2009.

[101] K. S. Pandya and S. K. Joshi, “A survey of optimal power flowmethods,” Journal of Theoretical and Applied Information Technol-ogy, 2008.

[102] H. D. Chiang and M. E. Baran, “On the existence and uniquenessof load flow solution for radial distribution power networks,” IEEETransactions on Circuits and Systems, vol. 37, no. 3, pp. 410–416,1990.

[103] R. A. Jabr, “Radial distribution load flow using conic programming,”IEEE Transactions on Power Systems, vol. 21, no. 3, pp. 1458–1459,2006.

[104] K. F. X. Bai, H. Wei and Y. Wang, “Semidefinite programmingfor optimal power flow problems,” International Journal of ElectricPower & Energy Systems, vol. 30, no. 6-7, pp. 383–392, 2008.

[105] R. A. Jabr, “Optimal power flow using an extended conic quadraticformulation,” IEEE Transactions on Power Systems, vol. 23, no. 3,pp. 1000–1008, 2008.

[106] A. Schecter and R. P. O’Neill, “Exploration of theACOPF feasible region for the standard IEEE test setoptimal power flow,” 2013, available on FERC website athttp://www.ferc.gov/industries/electric/indus-act/market-planning/opf-papers/acopf-6-test-problem-properties.pdf.

[107] P. A. Lipka, R. P. O’Neill, and S. Oren, “Developing line currentmagnitude constraints for IEEE test problems,” 2013, availableon FERC website at http://www.ferc.gov/industries/electric/indus-act/market-planning/opf-papers/acopf-7-line-constraints.pdf.

[108] M. Pirnia, R. P. O’Neill, P. A. Lipka, and C. Campaigne,“A computational study of linear approximations to theconvex constraints in the iterative linear IV-ACOPFformulation,” 2013, available on FERC website at http://www.ferc.gov/industries/electric/indus-act/market-planning/opf-papers/acopf-8-preprocessed-constraints-iliv-acopf.pdf.

[109] C. Campaigne, P. A. Lipka, M. Pirnia, R. P. O’Neill, and S. Oren,“Testing step-size limits for solving the linearized current volt-age AC optimal power flow,” 2013, available on FERC websiteat http://www.ferc.gov/industries/electric/indus-act/market-planning/opf-papers/acopf-9-stepsizelimits-iliv-acopf.pdf.

[110] J. Lavaei and S. H. Low, “Convexification of optimal power flowproblem,” 48th Annual Allerton Conference, 2010.

[111] J. Lavaei, A. Rantzer, and S. H. Low, “Power flow optimization usingpositive programming,” 18th IFAC World Congress, 2011.

[112] S. Sojoudi and S. H. Low, “Optimal charging of plug-in hybridelectric vehicles in smart grids,” IEEE Power and Energy SocietyGeneral Meeting, 2011.

[113] M. Kraning, E. Chu, J. Lavaei, and S. Boyd, “Dynamic networkenergy management via proximal message passing,” Foundations andTrends in Optimization, vol. 1, no. 2, pp. 1–54, 2013.

[114] S. Sojoudi and J. Lavaei, “Convexification of optimal power flowproblem by means of phase shifters,” IEEE SmartGridComm, 2013.

[115] R. Madani, M. Ashraphijuo, and J. Lavaei, “Promises of conicrelaxation for contingency-constrained optimal power flow problem,”Allerton, 2014.