On Local Minima of Cubic...

46
1 CRM-DIMACS Workshop on Mixed-Integer Nonlinear Programming October 2019, Montreal On Local Minima of Cubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS, MAE, CSML Joint work with Jeffrey Zhang Princeton, ORFE

Transcript of On Local Minima of Cubic...

Page 1: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

1

CRM-DIMACS Workshop on Mixed-Integer Nonlinear Programming

October 2019, Montreal

On Local Minima ofCubic Polynomials

Amir Ali AhmadiPrinceton, ORFE

Affiliated member of PACM, COS, MAE, CSML

Joint work with

Jeffrey ZhangPrinceton, ORFE

Page 2: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Deciding local minimality

2

Consider the optimization problem

Given a point ๐‘ฅ, decide if it is a local minimum.

min๐‘ฅโˆˆโ„๐‘›

๐‘“(๐‘ฅ)

๐‘ฅ โˆˆ ฮฉ

Why local minima?

- Global minima are often intractable- Recent interest in local minima, particularly in machine

learning applications- Existing notions that local minima are โ€œeasier to findโ€ or are

sufficient for applications- Formal understanding of local minima is desirable

Page 3: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Local minima

3

A point ๐‘ฅ is a local minimum of

if there exists a ball of radius ๐œ– > 0 such that ๐‘ ๐‘ฅ โ‰ค ๐‘(๐‘ฅ) for all ๐‘ฅ โˆˆ ๐ต๐œ– ๐‘ฅ โˆฉ ฮฉ.

๐‘ฅ is a strict local minimum if ๐‘ ๐‘ฅ < ๐‘(๐‘ฅ) for all ๐‘ฅ โˆˆ ๐ต๐œ– ๐‘ฅ โˆฉ ฮฉ\ ๐‘ฅ.

min๐‘ฅโˆˆโ„๐‘›

๐‘(๐‘ฅ)

๐‘ž๐‘– ๐‘ฅ โ‰ฅ 0, ๐‘– = 1, โ€ฆ , ๐‘š

Our focus: polynomial optimization problems

๐‘“ is a polynomial, ฮฉ is defined by polynomial inequalities.

min๐‘ฅโˆˆโ„๐‘›

๐‘“(๐‘ฅ)

๐‘ฅ โˆˆ ฮฉ

Page 4: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Unconstrained quadratic optimization Linear Programming

4

Known tractable cases

Check coefficients of characteristic polynomialCheck if ๐‘ฅ is optimal. If it is, add ๐‘๐‘‡๐‘ฅ = ๐‘๐‘‡ ๐‘ฅ as a constraint, and solve sequence of LPs

min๐‘ฅโˆˆโ„๐‘›

1

2๐‘ฅ๐‘‡๐‘„๐‘ฅ + ๐‘๐‘‡๐‘ฅ

๐‘ฅ is a local minimum if and only if๐‘„ ๐‘ฅ + ๐‘ = 0

๐‘„ โ‰ฝ 0

๐‘ฅ is a strict local minimum if and only if

๐‘„ ๐‘ฅ + ๐‘ = 0๐‘„ โ‰ป 0

min๐‘ฅโˆˆโ„๐‘›

๐‘๐‘‡๐‘ฅ

๐ด๐‘ฅ = ๐‘๐‘ฅ โ‰ฅ 0

๐‘ฅ is a local minimum if and only if it is optimal.

๐‘ฅ is a strict local minimum if and only it is the unique optimal solution.

Compute ๐‘› leading principal minors๐ด ๐‘ฅ = ๐‘, ๐‘ฅ โ‰ฅ 0, and ๐‘๐‘‡ ๐‘ฅ is attainable in the dual

Page 5: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Unconstrained quartic optimization Quadratic programming

๐‘ is a quartic polynomial

Known intractable cases

5

A matrix ๐‘€ is copositive if and only if 0 is a local minimum of

๐‘ฅ12

โ€ฆ๐‘ฅ๐‘›

2

๐‘‡

๐‘€๐‘ฅ1

2

โ€ฆ๐‘ฅ๐‘›

2

๐‘‡

min๐‘ฅโˆˆโ„๐‘›

1

2๐‘ฅ๐‘‡๐‘„๐‘ฅ + ๐‘๐‘‡๐‘ฅ

๐ด๐‘ฅ โ‰ฅ ๐‘

min๐‘ฅโˆˆโ„๐‘›

๐‘(๐‘ฅ)

min๐‘ฅโˆˆโ„๐‘›

๐‘ฅ๐‘‡๐‘€๐‘ฅ

๐‘ฅ โ‰ฅ 0

A matrix ๐‘€ is copositive if ๐‘ฅ๐‘‡๐‘€๐‘ฅ โ‰ฅ 0, โˆ€๐‘ฅ โ‰ฅ 0

or of

Strict local minimality also NP-hard.

Page 6: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Unconstrained quartic optimization Quadratic programming

๐‘ is a quartic polynomial

Summary of prior literature

6

min๐‘ฅโˆˆโ„๐‘›

1

2๐‘ฅ๐‘‡๐‘„๐‘ฅ + ๐‘๐‘‡๐‘ฅ

๐ด๐‘ฅ โ‰ฅ ๐‘

min๐‘ฅโˆˆโ„๐‘›

๐‘(๐‘ฅ)

Open cases?

Unconstrained cubic minimization

Unconstrained quadratic optimization Linear Programming

min๐‘ฅโˆˆโ„๐‘›

1

2๐‘ฅ๐‘‡๐‘„๐‘ฅ + ๐‘๐‘‡๐‘ฅ

min๐‘ฅโˆˆโ„๐‘›

๐‘๐‘‡๐‘ฅ

๐ด๐‘ฅ = ๐‘๐‘ฅ โ‰ฅ 0

Poly-time (both for local min and strict local min)

NP-hard (both for local min and strict local min)

Om

ar Kh

ayyam (1

04

8-1

13

1)

Page 7: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Outline

โ€ข Part I: Testing local minimality of a given pointfor a cubic polynomial

7

โ€ข Part 2: Finding a local minimum of a cubic polynomial

Page 8: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Classical optimality conditions

8

First Order Necessary Condition (FONC) Second Order Necessary Condition (SONC)

Second Order Sufficient Condition (SOSC):

is a local minimum โ‡’ no descent directions at

Unlike quadratics, not sufficient for cubic polynomials

๐‘ ๐‘ฅ1, ๐‘ฅ2 = ๐‘ฅ22 โˆ’ ๐‘ฅ1

2๐‘ฅ2 --+

+

๐’™ ๐’™

๐‘ฅ is a local minimum โ‡’ โˆ‡๐‘ ๐‘ฅ = 0 ๐‘ฅ is a local minimum โ‡’ โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0

FONC + โˆ‡2๐‘ ๐‘ฅ โ‰ป 0 โ‡’ ๐‘ฅ is a (strict) local minimum

A direction ๐‘‘ is a descent direction for ๐‘ at ๐‘ฅ if for some ๐›ผโˆ— > 0,๐‘ ๐‘ฅ + ๐›ผ๐‘‘ < ๐‘( ๐‘ฅ) for all ๐›ผ โˆˆ (0, ๐›ผโˆ—)

Page 9: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Necessary and sufficient condition for local minima

9

Theorem (Third Order Condition, TOC)

Let p be a cubic polynomial and suppose satisfies FONC and SONC. Then is a local minimum of ๐‘ if and only if

๐‘‘ โˆˆ ๐‘(๐›ป2๐‘( )) โ‡’ ๐›ป๐‘3 ๐‘‘ = 0

Moreover, this condition can be checked in polynomial time.

๐‘(๐›ป2๐‘( ๐‘ฅ)) is the null space of Hessian at ๐‘ฅ

๐‘3 is the cubic component of ๐‘

๐‘ฅ

๐‘ฅ

๐‘ฅ

Page 10: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Example: origin a local minimum

10

๐‘ ๐‘ฅ1, ๐‘ฅ2 = ๐‘ฅ22 + ๐‘ฅ1๐‘ฅ2

2

๐›ป๐‘ 0,0 =00

๐›ป2๐‘ 0,0 =0 00 2

๐›ป๐‘3(๐‘ฅ1, ๐‘ฅ2) =2๐‘ฅ2

2

2๐‘ฅ1๐‘ฅ2

๐‘‘ โˆˆ ๐‘(๐›ป2๐‘ ๐‘ฅ ) โ‡’ ๐›ป๐‘3 ๐‘‘ = 0?

๐›ป๐‘3(๐›ผ, 0) =00-

-

+

+

Page 11: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Example: origin not a local minimum

11

๐‘ ๐‘ฅ1, ๐‘ฅ2 = ๐‘ฅ22 โˆ’ ๐‘ฅ1

2๐‘ฅ2

๐›ป๐‘ 0,0 =00

๐›ป2๐‘ 0,0 =0 00 2

๐›ป๐‘3(๐‘ฅ1, ๐‘ฅ2) =โˆ’2๐‘ฅ1๐‘ฅ2

โˆ’๐‘ฅ12

๐‘‘ โˆˆ ๐‘(๐›ป2๐‘ ๐‘ฅ ) โ‡’ ๐›ป๐‘3 ๐‘‘ = 0?

๐›ป๐‘3(๐›ผ, 0) =0

โˆ’๐›ผ2

+

+

--

Page 12: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

A more natural condition?

12

๐‘‘ โˆˆ ๐‘(๐›ป2๐‘ ๐‘ฅ ) = 0 โ‡’ ๐‘3 ๐‘‘ = 0 Necessary for ๐ถ3 functions (where ๐‘3

would be the cubic component of the Taylor expansion). โ€œThird Order Necessary Conditionโ€ (TONC)

๐‘ ๐‘ฅ1, ๐‘ฅ2 = ๐‘ฅ22 โˆ’ ๐‘ฅ1

2๐‘ฅ2

Not sufficient for local optimality, even for cubics

Guarantees no descent directions for cubic polynomials

Does not guarantee no parabolas of descent

Page 13: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Is TOC necessary for general functions?

13

๐‘ ๐‘ฅ1, ๐‘ฅ2 = ๐‘ฅ14 + ๐‘ฅ1

2 + ๐‘ฅ22

๐‘‘ โˆˆ ๐‘ ๐›ป2๐‘ ๐‘ฅ โ‡’ ๐›ป๐‘3 ๐‘‘ = 0?

๐›ป๐‘3 =2๐‘ฅ2

2

4๐‘ฅ1๐‘ฅ2

๐›ป๐‘3(0,1) =20

๐›ป๐‘ 0,0 =00

๐›ป2๐‘ 0,0 =0 00 2

Easy to see not sufficient for higher degree polynomials (e.g., ๐‘ ๐‘ฅ = ๐‘ฅ5),but is it necessary? No!

Page 14: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

A characterization of local minima for cubics

14

Theorem (Third Order Condition, TOC)

Let p be a cubic polynomial and suppose satisfies FONC and SONC. Then is a local minimum of ๐‘ if and only if

๐‘‘ โˆˆ ๐‘(๐›ป2๐‘( )) โ‡’ ๐›ป๐‘3 ๐‘‘ = 0

Moreover, this condition can be checked in polynomial time.

๐‘ฅ

๐‘ฅ

๐‘ฅ

Page 15: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Proof of characterization of local minima (1/3)

15

๐‘ ๐‘ฅ + ๐œ†๐‘ฃ = ๐‘ ๐‘ฅ + ๐œ†๐›ป๐‘ ๐‘ฅ ๐‘‡๐‘ฃ +1

2๐œ†2๐‘ฃ๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ฃ +

Taylor expansion of cubic polynomials

๐œ†3๐‘3(๐‘ฃ)๐‘œ(๐œ†3)

๐‘ ๐‘ฅ + ๐›ผ๐‘‘ + ๐›ฝ๐‘ง = ๐‘ ๐‘ฅ + ๐›ป๐‘ ๐‘ฅ ๐‘‡ ๐›ผ๐‘‘ + ๐›ฝ๐‘ง

+1

2๐›ผ๐‘‘ + ๐›ฝ๐‘ง ๐‘‡๐›ป2๐‘ ๐‘ฅ ๐›ผ๐‘‘ + ๐›ฝ๐‘ง

+๐‘3 ๐›ผ๐‘‘ + ๐›ฝ๐‘ง

Suppose ๐‘ฅ satisfies FONC, SONC. We show ๐‘ฅ is a local min iff TOC.For any unit vectors ๐‘‘ in the null space of ๐›ป2๐‘( ๐‘ฅ) and ๐‘ง in the range of ๐›ป2๐‘ ๐‘ฅ ,

0

๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง

๐‘3 ๐›ผ๐‘‘ + ๐›ฝ๐‘ง = ๐‘3 ๐›ผ๐‘‘ + ๐›ฝ๐›ป๐‘3 ๐›ผ๐‘‘ ๐‘‡๐‘ง +1

2๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘3 ๐›ผ๐‘‘ ๐‘ง + ๐›ฝ3๐‘3(๐‘ง)

0

๐‘ ๐‘ฅ + ๐›ผ๐‘‘ + ๐›ฝ๐‘ง โˆ’ ๐‘( ๐‘ฅ) =

1

2๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง + ๐›ผ2๐›ฝ๐›ป๐‘3 ๐‘‘ ๐‘‡๐‘ง +

1

2๐›ผ๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ3๐‘3(๐‘ง)

Page 16: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Proof of characterization (2/3) [Sufficiency]

16

๐‘ ๐‘ฅ + ๐›ผ๐‘‘ + ๐›ฝ๐‘ง โˆ’ ๐‘( ๐‘ฅ) =

1

2๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง + ๐›ผ2๐›ฝ๐›ป๐‘3 ๐‘‘ ๐‘‡๐‘ง +

1

2๐›ผ๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ3๐‘3(๐‘ง)

๐›ป๐‘3 ๐‘‘ = 0 โˆ€ ๐‘‘ โˆˆ ๐‘(๐›ป2๐‘ ๐‘ฅ ) โ‡’ local minimum

1

2๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง +

1

2๐›ผ๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ3๐‘3(๐‘ง)

1

2๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง +

1

2๐›ผ๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ3๐‘3(๐‘ง)

๐›ฝ2(1

2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง +

1

2๐›ผ๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ๐‘3 ๐‘ง )

โ‰ฅ smallest nonzero eigenvalue

Upper bounded in abs. value

Therefore, โˆƒ๐›ผโˆ—, ๐›ฝโˆ— such that if ๐›ผ < ๐›ผโˆ—, ๐›ฝ < ๐›ฝโˆ—,this expression is nonnegative

Page 17: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Proof of characterization (3/3) [Necessity]

17

Pick a sequence ๐›ฝ๐‘– โ†’ 0, ๐›ผ๐‘– โˆ ๐›ฝ๐‘–

1

3

1

2๐›ฝ๐‘–

2 ๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง + ๐›ฝ๐‘–5/3

๐›ป๐‘3 ๐‘‘ ๐‘‡ ๐‘ง +1

2๐›ฝ๐‘–

7/3 ๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ๐‘–

3๐‘3( ๐‘ง)

๐‘ ๐‘ฅ + ๐›ผ๐‘‘ + ๐›ฝ๐‘ง โˆ’ ๐‘( ๐‘ฅ) =

1

2๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง + ๐›ผ2๐›ฝ๐›ป๐‘3 ๐‘‘ ๐‘‡๐‘ง +

1

2๐›ผ๐›ฝ2๐‘ง๐‘‡๐›ป2๐‘3 ๐‘‘ ๐‘ง + ๐›ฝ3๐‘3(๐‘ง)

local minimum โ‡’ ๐›ป๐‘3 ๐‘‘ = 0 โˆ€๐‘‘ โˆˆ ๐‘ ๐›ป2๐‘ ๐‘ฅ

Otherwise, for the sake of contradiction pick ๐‘‘ โˆˆ ๐‘ ๐›ป2๐‘ ๐‘ฅ

such that ๐›ป๐‘3( ๐‘‘) โ‰  0 and pick ๐‘ง = โˆ’๐›ป๐‘3( ๐‘‘)

1

2๐›ฝ2 ๐‘ง๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ง + ๐›ผ2๐›ฝ๐›ป๐‘3

๐‘‘๐‘‡

๐‘ง +1

2๐›ผ๐›ฝ2 ๐‘ง๐‘‡๐›ป2๐‘3

๐‘‘ ๐‘ง + ๐›ฝ3๐‘3( ๐‘ง)

< 0

Page 18: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Characterization of strict local minima

18

Proposition

is a strict local minimum of a cubic polynomial ๐‘ if and only if

๐‘ ๐‘ฅ + ๐›ผ๐‘‘ = ๐‘ ๐‘ฅ + ๐›ผ๐›ป๐‘ ๐‘ฅ ๐‘‡๐‘‘ +1

2๐›ผ2๐‘‘๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘‘ + ๐›ผ3๐‘3 ๐‘‘ .

Proof. Only need to show ๐‘ฅ strict local min โ‡’ โˆ‡2๐‘ ๐‘ฅ โ‰ป 0.

Otherwise for the sake of contradiction pick ๐‘‘ โˆˆ ๐‘ ๐›ป2๐‘ ๐‘ฅ .

๐‘ฅ

๐›ป๐‘ ๐‘ฅ = 0 (FONC)

๐›ป2๐‘ ๐‘ฅ โ‰ป 0 (SOSC)

(Note: SOSC is not necessary in general: ๐‘ ๐‘ฅ = ๐‘ฅ4.)

Observation: If a cubic has a strict local min, then that is the unique local min.

Proof.

Page 19: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Checking local minimality in polynomial time

19

โ€ข Input: ๐‘, ๐‘ฅ

โ€ข Compute gradient and Hessian of ๐‘ at ๐‘ฅ

โ€ข Check FONC and SONC

โ€ข Compute gradient of ๐‘3, evaluated on the null space of ๐›ป2๐‘ ๐‘ฅ

๐›ป๐‘3 ๐‘ฅ =

๐œ•๐‘3

๐œ•๐‘ฅ1( ๐‘ฅ)

โ€ฆ

๐œ•๐‘3

๐œ•๐‘ฅ๐‘›( ๐‘ฅ)

โ‡’

๐œ•๐‘3

๐œ•๐‘ฅ1(๐›ผ1๐‘ฃ1 + ๐›ผ2๐‘ฃ2 + โ‹ฏ + ๐›ผ๐‘˜๐‘ฃ๐‘˜)

โ€ฆ

๐œ•๐‘3

๐œ•๐‘ฅ๐‘›(๐›ผ1๐‘ฃ1 + ๐›ผ2๐‘ฃ2 + โ‹ฏ + ๐›ผ๐‘˜๐‘ฃ๐‘˜)

๐‘”1(๐›ผ1, โ€ฆ , ๐‘Ž๐‘˜)

โ€ฆ

๐‘”๐‘› ๐›ผ1, โ€ฆ , ๐‘Ž๐‘˜

โ€ข All coefficients of all ๐‘”๐‘– must be zero

โ€ข Compute a basis {๐‘ฃ1, ๐‘ฃ2, โ€ฆ , ๐‘ฃ๐‘˜} for null space of ๐›ป2๐‘ ๐‘ฅ (solving linear systems)

For strict local minima, check FONC and SOSC (leading ๐‘› principal minors must be positive)

Page 20: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Outline

โ€ข Part I: Testing local minimality of a given pointfor a cubic polynomial

20

โ€ข Part 2: Finding a local minimum of a cubic polynomial

Page 21: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Finding local minima

21

Given a cubic polynomial ๐‘, can we efficiently find a local minimum of ๐‘?

Unfortunatelyโ€ฆ

Theorem

Deciding if a cubic polynomial has a critical point is strongly NP-hard.

Reduction from MAXCUT

Given a graph ๐บ = (๐‘‰, ๐ธ), partition the vertices into two sets such that as many edges as possible are between vertices in opposite sets

Letโ€™s start with a โ€œsimplerโ€ question. Can we efficiently find a critical point of ๐‘?

Page 22: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

MAXCUT (decision version)

22

Is there a cut of size ๐‘˜?

Quadratic satisfiability

1 โˆ’ ๐‘ฅ๐‘–2 = 0, ๐‘– = 1, โ€ฆ , ๐‘›

1

4

๐‘–,๐‘— โˆˆ๐ธ

(1 โˆ’ ๐‘ฅ๐‘–๐‘ฅ๐‘—) = ๐‘˜

Critical points of a cubic polynomial

๐‘ ๐‘ฅ1, โ€ฆ , ๐‘ฅ๐‘›, ๐‘ฆ0, ๐‘ฆ1, โ€ฆ , ๐‘ฆ๐‘› = ๐‘ฆ๐‘œ

1

4

๐‘–,๐‘— โˆˆ๐ธ

1 โˆ’ ๐‘ฅ๐‘–๐‘ฅ๐‘— โˆ’ ๐‘˜ +

๐‘–=1

๐‘›

๐‘ฆ๐‘–(1 โˆ’ ๐‘ฅ๐‘–2)

Page 23: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Critical points of a cubic polynomial

23

๐›ป๐‘ ๐‘ฅ, ๐‘ฆ =

๐‘‘๐‘

๐‘‘๐‘ฅ๐‘–

๐‘‘๐‘

๐‘‘๐‘ฆ0

๐‘‘๐‘

๐‘‘๐‘ฆ๐‘–

=

โˆ’๐‘ฆ0

4

๐‘–,๐‘— โˆˆ๐ธ

๐‘ฅ๐‘— โˆ’ 2๐‘ฅ๐‘–๐‘ฆ๐‘–

1

4

๐‘–,๐‘— โˆˆ๐ธ

1 โˆ’ ๐‘ฅ๐‘–๐‘ฅ๐‘— โˆ’ ๐‘˜

1 โˆ’ ๐‘ฅ๐‘–2

๐‘ ๐‘ฅ, ๐‘ฆ = ๐‘ฆ๐‘œ

1

4

๐‘–,๐‘— โˆˆ๐ธ

1 โˆ’ ๐‘ฅ๐‘–๐‘ฅ๐‘— โˆ’ ๐‘˜ +

๐‘–=1

๐‘›

๐‘ฆ๐‘–(1 โˆ’ ๐‘ฅ๐‘–2)

Any cut of size ๐‘˜ โ‡’ critical point (๐‘ฅ = cut, ๐‘ฆ = 0)

Any critical point โ‡’ cut of size ๐‘˜ (๐‘ฅ โ‡’ cut)

But this doesnโ€™t necessarily mean finding local minima is NP-hard.First some geometryโ€ฆ

Page 24: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

24

Some geometric properties of local minima

Theorem

The local minima of any cubic polynomial ๐‘ form a convex set.

Lemma

If is a local minimum of a cubic polynomial ๐‘ and ๐‘‘ โˆˆ ๐‘(โˆ‡2๐‘ ), then for any ๐›ผ,

๐›ป๐‘ +๐›ผ๐‘‘ = 0

Proof (of theorem).Let ๐‘ฅ and ๐‘ฆ be local minima. Note ๐‘ is constant on the line between ๐‘ฅ and ๐‘ฆ. Consider ๐‘ง = ๐‘ฅ + ๐›ผ(๐‘ฆ โˆ’ ๐‘ฅ)

FONC: ๐‘ฆ โˆ’ ๐‘ฅ โˆˆ ๐‘ ๐›ป2๐‘ ๐‘ฅ + Lemma

SONC: Convex combination of PSD matrices is PSD

TOC: ๐‘(๐›ป2๐‘((1 โˆ’ ๐›ผ)๐‘ฅ + ๐›ผ๐‘ฆ))= ๐‘( 1 โˆ’ ๐›ผ ๐›ป2๐‘ ๐‘ฅ + ๐›ผ๐›ป2๐‘(๐‘ฆ))

= ๐‘ ๐›ป2๐‘ ๐‘ฅ โˆฉ ๐‘ ๐›ป2๐‘ ๐‘ฆ โŠ† ๐‘ โˆ‡2๐‘ ๐‘ฅ

๐‘ฅ

๐‘ฅ

๐‘ฅ

Page 25: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Convexity of the set of local minima

25

Critical points

Local minima

๐‘ ๐‘ฅ1, ๐‘ฅ2 = ๐‘ฅ13 + 3๐‘ฅ1

2๐‘ฅ2 + 3๐‘ฅ1๐‘ฅ22 + ๐‘ฅ2

3 โˆ’ ๐‘ฅ1 โˆ’ ๐‘ฅ2

๐‘ convex

Page 26: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Set of local minima not necessarily polyhedral

26

๐‘ ๐‘ฅ1, ๐‘ฅ2, ๐‘ฅ3, ๐‘ฅ4 =1

2๐‘ฅ1

2๐‘ฅ32 + 2๐‘ฅ1๐‘ฅ3๐‘ฅ4 +

1

2๐‘ฅ1๐‘ฅ4

2 โˆ’1

2๐‘ฅ2๐‘ฅ3

2

+๐‘ฅ2๐‘ฅ3๐‘ฅ4 + 2๐‘ฅ2๐‘ฅ42 + ๐‘ฅ3

2 + ๐‘ฅ42

๐‘ฅ3 = 0, ๐‘ฅ4 = 0 โˆฉ2 + ๐‘ฅ1 โˆ’ ๐‘ฅ2 2๐‘ฅ1 + ๐‘ฅ2

2๐‘ฅ1 + ๐‘ฅ2 2 + 2๐‘ฅ1 + 4๐‘ฅ2โ‰ป 0

Page 27: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Convexity region

27

Definition (Convexity region)

The convexity region of a polynomial ๐‘ is the set๐‘ฅ โˆˆ โ„๐‘› โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0}

โ€ข The convexity region of a cubic polynomial is a spectrahedron (we call it a โ€œCH-spectrahedronโ€)

โ€ข Any spectrahedron ๐‘ฅ โˆˆ โ„๐‘› ๐‘–=1๐‘› ๐ด๐‘–๐‘ฅ๐‘– + ๐‘„ โ‰ฝ 0} with ๐ด๐‘– in โ„๐‘šร—๐‘š

is the shadow of a CH-spectrahedron in dimension ๐‘› + ๐‘š.

โ€ข Not every spectrahedron is a CH-spectrahedron:

Page 28: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

28

A โ€œconvexโ€ optimization problem

Theorem

If a cubic polynomial ๐‘ has a local minimum, the solution set of the following optimization problem is the closure of its local minima.

In particular, the optimal value of this โ€œconvexโ€ problem gives the value of ๐‘ at any local minimum.

min๐‘ฅ

๐‘(๐‘ฅ)

โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0

Note: the value of ๐‘ at local minima must be the same.

Very rough intuition:

Page 29: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

29

A โ€œconvexโ€ optimization problem proof (1/2)

local minimum

โ€œbetter pointโ€

Local minima are optimal to

min๐‘ฅ

๐‘(๐‘ฅ)

โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0

Page 30: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

30

local minimum

optimal point

FONC: โœ”SONC: โœ”

TOC: ๐‘ โˆ‡2๐‘ ๐‘ง โŠ† ๐‘(โˆ‡2๐‘ ๐‘ฅ )

A โ€œconvexโ€ optimization problem proof (2/2)

๐‘ฅ๐‘ฆ ๐‘ง

Solutions tomin

๐‘ฅ๐‘(๐‘ฅ)

โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0

are in closure of local minima

๐‘ ๐‘ฅ = ๐‘(๐‘ฆ)

Page 31: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

31

Sum of squares polynomials

Page 32: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Sum of squares polynomials

32

โ€ข A polynomial ๐‘ is a sum of squares (sos) if it can be written as

๐‘(๐‘ฅ) = ๐‘ž๐‘–2(๐‘ฅ)

โ€ข Any sos polynomial is nonnegative

โ€ข Imposing that a polynomial is sos is a semidefinite constraint

โ€ข A matrix of polynomials ๐‘€(๐‘ฅ) is an sos-matrix if the polynomial ๐‘ฆ๐‘‡๐‘€(๐‘ฅ)๐‘ฆ is sos, or equivalently if ๐‘€ ๐‘ฅ = ๐‘… ๐‘ฅ ๐‘… ๐‘ฅ ๐‘‡

Sum of squares relaxations

min๐‘ฅ

๐‘“(๐‘ฅ) = max๐›พ

๐›พ

๐‘“ ๐‘ฅ โˆ’ ๐›พ is a nonnegative polynomial

sos

Find lower bounds on the optimal value of a polynomial optimization problem

โ‰ฅ

Page 33: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

33

Sum of squares relaxations for constrained problems

min๐‘ฅโˆˆโ„๐‘›

๐‘(๐‘ฅ)

๐‘ž๐‘– ๐‘ฅ โ‰ฅ 0, ๐‘– = 1, โ€ฆ , ๐‘š

max๐›พ,๐œŽ๐‘– ๐‘ ๐‘œ๐‘ 

๐›พ

๐‘ ๐‘ฅ โˆ’ ๐›พ = ๐œŽ0(๐‘ฅ) +

๐‘–=1

๐‘š

๐‘ž๐‘–(๐‘ฅ)๐œŽ๐‘–(๐‘ฅ)

โ‰ค

Lasserre hierarchy:

For ๐œŽ๐‘– of fixed degree, this is an SDP of size polynomial in data

As deg ๐œŽ๐‘– โ†’ โˆž, the optimal value of the sos program will converge to the true optimal value (under a mild assumption)

Putinarโ€™s Psatz

Page 34: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

34

Theorem

If ๐‘ has a local minimum, the first level of this sos relaxation (i.e., when deg ๐œŽ = deg ๐‘† = 2) is tight.

Sos relaxation

min๐‘ฅ

๐‘(๐‘ฅ)

โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0

max๐œŽ(๐‘ฅ),๐‘†(๐‘ฅ)

๐›พ

๐‘(๐‘ฅ) โˆ’ ๐›พ = ๐œŽ(๐‘ฅ) + ๐‘‡๐‘Ÿ(๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ )

๐œŽ is sos๐‘† is an sos-matrix

โ‰ค

Proof.Produce an algebraic identity that attains the best possible value.For any local minimum ๐‘ฅ,

๐‘ ๐‘ฅ โˆ’ ๐‘โˆ— =1

3๐‘ฅ โˆ’ ๐‘ฅ ๐‘‡๐›ป2๐‘ ๐‘ฅ ๐‘ฅ โˆ’ ๐‘ฅ + ๐‘‡๐‘Ÿ(๐›ป2๐‘ ๐‘ฅ

1

6๐‘ฅ โˆ’ ๐‘ฅ ๐‘ฅ โˆ’ ๐‘ฅ ๐‘‡ )

Value at local min ๐œŽ ๐‘ฅ

sos

๐‘†(๐‘ฅ)sos-matrix

Page 35: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

How to extract a local min itself?

35

๐œŽ(๐‘ฅ) + ๐‘‡๐‘Ÿ(๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ )

Idea: Find the zeros of

Solve:min

๐‘ฅ0

๐‘‡๐‘Ÿ ๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0

๐œŽ ๐‘ฅ = 0

๐›ป2๐‘ ๐‘ฅ โ‰ฝ 0

Nonlinear constraintsโ€ฆ

or are they?

Page 36: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Recovering a local minimum

36

๐‘‡๐‘Ÿ ๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0? (cubic equation)

๐›ป2๐‘ ๐‘ฅ โ‰ฝ 0 is a linear matrix inequality โœ”

๐œŽ is an sos quadratic, so the solutions to ๐œŽ ๐‘ฅ = 0 can be found by solving a system of linear equations

โœ”

Observation:

Since ๐‘† is a quadratic sos matrix, ๐‘†(๐‘ฅ) = ๐‘… ๐‘ฅ ๐‘… ๐‘ฅ ๐‘‡, where ๐‘…(๐‘ฅ) is affine

More geometryโ€ฆ

min๐‘ฅ

0

๐‘‡๐‘Ÿ ๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0

๐œŽ ๐‘ฅ = 0

๐›ป2๐‘ ๐‘ฅ โ‰ฝ 0

๐‘‡๐‘Ÿ โˆ‡2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0 โ‡” โˆ‡2๐‘ ๐‘ฅ ๐‘… ๐‘ฅ = 0 (quadratic equation)

๐‘…๐‘– ๐‘ฅ โˆˆ ๐‘ โˆ‡2๐‘ ๐‘ฅ , โˆ€๐‘–

Page 37: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

37

Relative Interior

Definition (Relative Interior)

The relative interior of a nonempty convex set ๐‘† is the set๐‘ฅ โˆˆ ๐‘† โˆ€๐‘ฆ โˆˆ ๐‘†, โˆƒ๐›ผ > 1, ๐‘ฆ + ๐›ผ ๐‘ฆ โˆ’ ๐‘ฅ โˆˆ ๐‘†}

Page 38: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

More geometry

38

Lemma

Lemma

๐‘ฅ

๐‘ฅ

Convex combination of PSD matrices: ๐‘ โˆ‡2๐‘ ๐‘ฅ โŠ† ๐‘ โˆ‡2๐‘ ๐‘ฅ

Let ๐‘ฅ be a local minimum of a cubic polynomial ๐‘. Then for any ๐‘ฅ โˆˆ โ„๐‘›

and ๐‘‘ โˆˆ ๐‘(โˆ‡2๐‘ ๐‘ฅ ), ๐‘‘๐‘‡โˆ‡2๐‘ ๐‘ฅ ๐‘‘ = 0.

Let ๐‘ฅ be a local minimum of a cubic polynomial ๐‘. Then for any ๐‘ฅ in the

relative interior of the convexity region of ๐‘, ๐‘ ๐›ป2๐‘( ๐‘ฅ) = ๐‘ ๐›ป2๐‘ ๐‘ฅ .

Proof (of second lemma).

First Lemma + โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0: ๐‘ โˆ‡2๐‘ ๐‘ฅ โŠ† ๐‘ โˆ‡2๐‘ ๐‘ฅ

Page 39: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Rewriting the cubic equation

39

โ€ข Find any point ๐‘ฅ in the relative interior of the convexity region

โ€ข Find a basis {๐‘ฃ1, ๐‘ฃ2, โ€ฆ , ๐‘ฃ๐‘˜} for ๐‘(๐›ป2๐‘ ๐‘ฅ )

โ€ข Decompose ๐‘† ๐‘ฅ = ๐‘… ๐‘ฅ ๐‘… ๐‘ฅ ๐‘‡

โ€ข Impose ๐‘…๐‘– ๐‘ฅ โˆˆ ๐‘ ๐›ป2๐‘ ๐‘ฅ โˆ€๐‘– as ๐‘…๐‘– ๐‘ฅ = ๐‘—=1๐‘˜ ๐›ผ๐‘—๐‘ฃ๐‘— โˆ€๐‘–

(linear constraint!)

โœ”

What does this buy us?

Goal: Impose ๐‘‡๐‘Ÿ ๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0

For any ๐‘ฅ such that โˆ‡2๐‘ ๐‘ฅ โ‰ฝ 0, this is equivalent to

imposing ๐‘…๐‘– ๐‘ฅ โˆˆ ๐‘ โˆ‡2๐‘ ๐‘ฅ โˆ€๐‘– โ‡” ๐‘‡๐‘Ÿ ๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0

Page 40: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

An SDP!

40

min๐‘ฅ

0

๐‘‡๐‘Ÿ ๐›ป2๐‘ ๐‘ฅ ๐‘† ๐‘ฅ = 0

๐œŽ ๐‘ฅ = 0

๐›ป2๐‘ ๐‘ฅ โ‰ฝ 0

Theorem

The relative interior of the feasible set of this SDP is the set of local minima of ๐‘.

Rewritable as an SDP!

Two steps require a point in the relative interior of a set

How can we get a point in the relative interior of a set?

Page 41: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Finding a point in the relative interior

41

Definition (Relative Interior)

The relative interior of a nonempty convex set ๐‘† is the set๐‘ฅ โˆˆ ๐‘† โˆ€๐‘ฆ โˆˆ ๐‘†, โˆƒ๐›ผ > 0, ๐‘ฅ + ๐›ผ ๐‘ฆ โˆ’ ๐‘ฅ โˆˆ ๐‘†}

Page 42: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Algorithm for finding a local minimum

โ€ข Find an sos-certified lower bound for value at any local minimum

โ€ข Find any point in the relative interior of the convexity region

โ€ข Find a basis for the null space of the Hessian of any local minimum

โ€ข Find relative interior solution of equivalent SDP

42

Page 43: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Overall result

43

Theorem

Deciding if a cubic polynomial ๐‘ has a local minimum, and finding one if it does, can be done in polynomially many calls to an SDP blackbox, Choleskly decompositions, and linear system solves of polynomial size.

SDP Blackbox Optimal value

Can be used to recover solutions

Page 44: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Why the blackbox assumption?

44

Local minima can be irrational:

๐‘ ๐‘ฅ = ๐‘ฅ3 โˆ’ 6๐‘ฅ

๐‘ฅ = 2 is the unique local minimum

Even if there are rational local minima, they can all have size exponential in the input:

๐ด ๐‘ฅ =

๐‘ฅ1 22 1

๐‘ฅ2 ๐‘ฅ1

๐‘ฅ1 1

โ€ฆ

โ€ฆ โ‹ฑ โ€ฆ

โ€ฆ๐‘ฅ๐‘› ๐‘ฅ๐‘›โˆ’1

๐‘ฅ๐‘›โˆ’1 1

๐‘ ๐‘ฅ = ๐‘ฆ๐‘‡๐ด ๐‘ฅ ๐‘ฆ, where Local minima:๐‘ฆ = 0 โˆฉ {๐ด ๐‘ฅ โ‰ป 0}

๐‘ฅ1 > 4, ๐‘ฅ2 > 16, โ€ฆ , ๐‘ฅ๐‘› > 22๐‘›

Page 45: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Summary

โ€ข Given a cubic polynomial ๐‘ and a point ๐‘ฅ, checking whether ๐‘ฅ is a local minimum of ๐‘can be done in polynomial time in the Turing model

โ€ข It is strongly NP-hard to test if a cubic polynomial has a critical point

โ€ข Given a cubic polynomial ๐‘, we can test if there is a local minimum by solving polynomially many SDPs of polynomial size

45

Page 46: On Local Minima of Cubic Polynomialsaaa/Public/Presentations/Local_minima_cubics_Montreal_MINLP19.pdfCubic Polynomials Amir Ali Ahmadi Princeton, ORFE Affiliated member of PACM, COS,

Thank you!

46

Want to know more? aaa.princeton.eduprinceton.edu/~jeffz