Parameterized Model Order Reduction via Quasi-Convex Optimization

Kin Cheong Sou with Luca Daniel and Alexandre Megretski

Systems on Chip or PackageInterconnect & Substrate

Courtesy of Harris semiconductor

RF Inductors

MEMresonators

210/22/2010

DSP

Digital

LNA

LO

Analog RF

ADC

ADC

Mixed SignalI

Q

From 3D Geometry to Circuit Model

Fig. thanks to Coventor

•Need accurate mathematical models of components•Describe components using Maxwell equations, Navier-

Stokes equations, etc

310/22/2010

From 3D Geometry to Circuit Model

dt

dEH

dt

dHE

4 2 2

4 2 20

( )w

elec a

u u uEI S F p p dy

x x t

3 ( )((1 6 ) ) 12

puK u p p

t

dt

dEH

dt

dHE

inBvvGdt

dvvC )()(

Z(f)Z(f) Z(f)Z(f) Z(f)Z(f) Z(f)Z(f)

•Model generated by available field solver•Field solver models usually high order

410/22/2010

RF Inductor Model Reduction

•Spiral radio frequency (RF) inductor•Impedance•State space model has 1576 states•Reduced model has 8 states

1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2

x 1010

0

0.5

1

1.5

2

2.5

3x 10

4

1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2

x 1010

-1.5

-1

-0.5

0

0.5

1

1.5x 10

-7

R L

f f

x full 1576 states- reduced 8 states

x full 1576 states- reduced 8 states

2 Z f R f j f L f

510/22/2010

•Parameter dependent RF inductor•Two design parameters:

- Wire width w- Wire separation d

d

w

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 1010

0

2000

4000

6000

8000

10000

12000

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 1010

-5

-4

-3

-2

-1

0

1

2

3

4

5x 10

-8

f f

R L

d = 1umd = 3umd = 5um

d = 1umd = 3umd = 5um

610/22/2010

RF Inductor Parameter Dependency

Parameterized Reduced Modeling

Parameterizedreduced model

Gr(d,w)

•One reduced model with explicit dependency on parameters

•Fast generation of reduced model for all parameter values

710/22/2010

d

w

DSPLNA

LO

ADC

ADC

I

Q

,d w

reducedmodel

Parameterized Reduced Model Example

•Parameter dependent complex system

•Parameterized reduced order model

•Coefficients depend explicitly on d•Low order, inexpensive to simulate

2, ( , )r

sG s G s

s

dd d

d

99

99

0.5 2,

0.499

s sG s

d dd

ds s d

810/22/2010

Continuous/Discrete-time Setups

10/22/2010 9

Continuous-time Discrete-time

left-half plane & imaginary axis unit disk & unit circle

& G s G j & jG z G e

1

1

zs

z

s

zs

•Parameterized moment matching methods- References:

• [Grimme et al. AML 99]• [Daniel et al. TCAD 04]• [Pileggi et al. ICCAD 05]• [Bai et al. ICCAD 07]

- Reduced model order increases with number of parameters rapidly- Require knowledge of state space model

•Rational transfer function fitting methods- Does not require state space model- Reduced model order does not increase with number of parameters- More expensive than moment matching in general

1010/22/2010

Parameterized Model Reduction Methods

Moment Matching Method

10/22/2010 11

1G z C zI A B D

1

r V U VG z C z A B DUI

Projection withUV = I

Full model Reduced model

( ) ( )k kn n

k r k

d dG z G z

dz dz

with the moment matching properties

10-1

100

101

102

103

104

105

10-10

10-8

10-6

10-4

10-2

100

102

Ma

gn

itud

e (

ab

s)

Bode Diagram

Frequency (rad/sec)

8th order full 4th order MM

moments matched

user specified

Rational Transfer Function Fitting

10/22/2010 12

r

p zG z

q z

input output

•Idea from system ID – reduced model matching I/O data

? ?p z q z

•Data in time domain or frequency domain•Data from state space model or experiment measurement

Explanations in Two Steps

10/22/2010 13

•Will present a rational transfer function fitting method

•First describe basic non-parameterized reduction

•Then extend basic method to parameterized setup

Non-Parameterized Model Order Reduction

Non-parameterized Problem Statement

•Given transfer function G(z)

•Find parameterized reduced model of order r

0

0

rr

r rr

p z p z pG z

q z q z q

p zG z

q z

subject to stableq z

,minimize

p q

1510/22/2010

•Reduced model found as the solution

dec. vars.

roots inside unit circle

H norm error

•Can obtain state space realization from p(z) and q(z)

Difficulty with H Norm Reduction

10/22/2010 16

p zG z

q z

j j j jG e q e p e q e

•Difficulty #2: abs. value on the “wrong” side

iff

•Difficulty #1: stability constraint not convex if r >2

331 5q z z 33

2 5q z z

31 2 27

2 2 25

q qz z but

branchingsolutions

convex combo. of stable poly.not necessarily stable

Idea From Optimal Hankel Reduction

10/22/2010 17

minrG

rG G

stablerGs.t.

order rG r

,min

rG FrG FG

stable, anti-stabler FGs.t.

order rG r

minQ

G Q

( )Q H rs.t.

,

,1

Obtain s.t.r Han

n

r Han ii r

G

G G G

anti-stablerelaxation

redefinedec. var.Solve AAK problem

efficiently (Glover)

suboptimalsolution

Anti-Stable Relaxation in Rational Fit

10/22/2010 18

•Similar to Hankel reduction, add anti-stable term

1

1

f zp zG z

q z q z

subject to stable, degq z f r

,minimize

p q

added DOF

•In Hankel MR, entire anti-stable term is decision variable•Here, only numerator f is decision variable

flip polesof q(z)

Combine Stable and Anti-stable Terms

10/22/2010 19

•Combine stable and anti-stable terms in reduced model

1

1

f zp z b z jc z

q z a zq z

10 1

10 1

111

r rr

r rr

r rrj

a z a a z z a z z

b z b b z z b z z

c z c z z c z z

•New decision variables are trigonometric polynomials

0 1

0 1

1

cos

cos

sin

j

j

j

a e a a

b e b b

c e c

Stability and Positivity

10/22/2010 20

•Can show

stableq z 0, ja e

-1.5 -1 -0.5 0 0.5 1 1.5-1.5

-1

-0.5

0

0.5

1

1.5

•Overcome Difficulty #1, trigonometric positivity convex constraint 1 2e.g. 1 cos cos 2 0a a

a1

a 2

•Overcome Difficulty #2, the trouble making abs. value is gone!

j

j

j

j

j jG e a e b e j

a e

c e

a e

Quasi-Convex Relaxation

10/22/2010 21

•Quasi-convex relaxation

b z jc zG z

a z

subject to 0, for 1a z z

, ,minimize

a b c

•Original optimal H norm model reduction problem

p zG z

q z

subject to stableq z

,minimize

p q

Quasi-convex problem,easy to solve

From Relaxation Back to H Reduction

10/22/2010 22

•Obtain (a,b,c) by solving quasi-convex relaxation

•Spectral factorize a to obtain stable denominator q*

* * 1z K za q q z for some K

•With q* found, search for numerator p* by solving

*

* arg minp

p zG z

zp

q

convex problem

Quality of Suboptimal Reduced Model

10/22/2010 23

•Minimizing upper bound of Hankel norm error

1

1

H

f zb z jc z p z p zG z G z G z

a z q z q zq z

•H norm error upper bound

*

* , ,1 min

a b c

p z b z jc zG z r G z

q z a z

Back to Big Picture – Model Reduction

10/22/2010 24

minrG

rG G

stablerGs.t.

order rG r

, ,mina b c

b jcG

a

0, 1a z z s.t.

optimal a(z), b(z), c(z) suboptimal p(z), q(z)

discussed

discussed

discussed

How to solve it?

Quasi-Convex Optimization

Quasi-Convex Optimization

10/22/2010 26

J(x)

x

All sub-level setsare convex sets

•Quasi-convex function is “almost convex”

Local (also global) minima Local (but not global) minima

Function not necessarily convex

•[Outer loop] Bisection search for objective value•[Inner loop] Convex feasibility problem (e.g. LP, SDP)

•Convex problem algorithms: 1) interior-point method 2) cutting plane method

Cutting Plane Method

optimal point

covering set

iterate 1

iterate 2

2710/22/2010

•Optimization problem data described by oracle

•What is the oracle in our model reduction problem?

Oraclecall oracle

retu

rn c

ut

kept

removed

call oracle

return cut

keptremoved

Model Reduction Oracles

10/22/2010 28

Oracle #1 (objective value):

b z jc zG z

a z

j j j j jG e a e b e c e a e

Oracle #2 (positive denominator):

Discretize frequency finite number of linear inequalities, “easy”

for any fixed

0, ja e

•Given candidate a(z), b(z), c(z), check two conditions

Cannot discretize frequency!

Positivity Check

10/22/2010 29

0 0.5 1 1.5 2 2.5 3 3.5-2

-1

0

1

2

3

4

5

6

t

stationary pointsr = 8 case

ja e

•Check only finite number of stationary points

•Much harder to check in the parameterized case

Back to Big Picture – Model Reduction

10/22/2010 30

minrG

rG G

stablerGs.t.

order rG r

, ,mina b c

b jcG

a

0, 1a z z s.t.

optimal a(z), b(z), c(z) suboptimal p(z), q(z)

discussed

discussed

discussed

Solved withcutting planemethod

Parameterized Model Order Reduction

Problem Statement

•Given parameter dependent transfer function G(z,d)

•Find parameterized reduced model of order r

0

0

,,

,

rr

r rr

p z p zd d dd

d

pG z

q z q z qd d

max , ,d

rd dG z G z

subject to ,q z d

,minimize

p q

stable for all d

3210/22/2010

•Reduced model found as the solution design parameter

Parameterized Reduced Model Example

•Parameter dependent complex system

•Parameterized reduced order model

•Coefficients depend explicitly on d•Low order, inexpensive to simulate

2, ( , )r

zG z G z

z

dd d

d

99

99

0.5 2,

0.499

z zG z

d dd

dz z d

3310/22/2010

10 1

10 1

111

,

,

,

r rr

r rr

r rrj

d d d d

d d

a z a a z z a z z

b z b b z zd db z z

c z c z z c z zd d d

1

0 1

, 2 sin 2 sin

, 2 cos 2 cos

jr

jr

d d d

d d d

c e c c r

a de a a a r

jz e

3410/22/2010

Parameterized Decision Variables

•Decision variables = parameterized trig. poly.

•When evaluated on unit circle, i.e.

Parameterized Quasi-Convex Relaxation

10/22/2010 35

, ,,

,

b z jd dd

c zG z

a dz

subject to , 0, for 1 and a z d z d

, ,minimize

a b c

•Parameterized quasi-convex relaxation

•Solution technique similar to non-parameterized case•Some extension requires more care, e.g.

check , 0, for 1 and a z zd d

Parameterized positivity check is hard!

Parameterized Positivity Check

10/22/2010 36

0 1 2, cos cos 2ja e d a d a d a d denominator

1 2

poly of

cos

ia d d

d d d

a simple parameterdependency

denominator = multivariate trigonometric polynomial

cos 3 cos 5cos 2 cos e.g.

•Positivity check of multivariable trig. poly. is hard•Another variant is multivariable ordinary polynomial our focus

…

Positivity Check of Multivariate Polynomials

Checking Polynomial Positivity – Special Cases

10/22/2010 38

•Univariate case simple, check the roots of derivative

4 3 23 2 4 0 ?x x x Is it true for all x,

•Multivariate quadratic form is easy but important

1 1

1 2 3 1 2 1 3 2 2

3

2 2 2

3

2 3 2

2 3 6 4 3 1 0 0 ?

2 0 3

Tx x

x x x x x x x x x

x x

polynomial nonnegative matrix positive semidefinite

Checking Polynomial Positivity – General Case

10/22/2010 39

•Positivity check of general multivariate polynomial is hard

2 2 31 24

2 24

1 12 5 2 0 ?x x x x x x Question: [from Parrilo & Lall]

2 21 11 12 13 1

4 4 2 2 3 2 21 2 1 2 1 2 2 12 22 23 2

1 2 13 23 33 1 2

2 5 2

Tx q q q x

x x x x x x x q q q x

x x q q q x x

= Q (Gram matrix)Monomials of relevant degrees

•What if we still write out “quadratic form”?

Checking Polynomial Positivity – General Case

10/22/2010 40

•To find Q, equate coefficients of all monomials

132 2q31 2 :x x2 21 2 :x x 12 331 2q q

230 2q31 2 :x x

112 q41 :x

225 q42 :x

•Gram matrix Q is typically not unique. If we can find Q ≥ 0

2 21 1

4 4 2 2 3 2 21 2 1 2 1 2 2 2

1 2 1 2

2 5 2 0

Tx x

x x x x x x x Q x

x x x x

Generally, linear constraintson Q, i.e. L(Q) = 0

Semidefinite Program/LMI Optimization

10/22/2010 41

0minimize

subject to 0

0

Q

T

L Q

L Q

Q Q

linear objective

linear constraints

pos. def. matrix variable

•Standard form:

•Efficiently solvable in theory and practice•Polynomial-time algorithm available•Efficient free solvers: SeDuMi, SDPT3, etc.

•Lots of applications•KYP lemma, Lyapunov function search, filter design,

circuit sizing, MAX-CUT, robust optimization …

Read Boyd and Vandenberghe’s SIAM review

Positivity Check is Sufficient Only

10/22/2010 42

2 2 21 2 3 1 2 1 3

1 1

2 2

3 3

2 3 6 4

2 3 2

3 1 0

2 0 3

T

x x x x x x x

x x

x x

x x

4 4 2 2 31 2 1 2 1 2

2 21 12 22 2

1 2 1 2

2 5 2T

x x x x x x

x x

x Q x

x x x x

spans R3 does not span R3

Quadratic case General case

•Requiring Q ≥ 0 sufficient but not necessary!2 4 4 2 2 21 2 1 2 1 21 3x x x x x x Positive? Can you find Q ≥ 0?

Sum of Squares (SOS)

10/22/2010 43

•Finding Q ≥ 0 equivalent to sum of squares decomposition•In our example, we can find

2 3 1 2 2 0 0

3 5 0 3 3 1 1

1 0 5 1 1 3

1

2 23

1

T T

Q

2 24 4 2 2 3 2 2 21 2 1 2 1 2 1 2 1 2 2 1 2

1 12 5 2 2 3 3

2 2x x x x x x x x x x x x x

sum of squares positive semidefinite Q nonnegativity

Wrap Up

4 Turn RF Inductor PMOR

d

w

1 1.5 2 2.5 3 3.5 4 4.5 51

1.5

2

2.5

3

3.5

4

4.5

5

W ( m)

d (

m)

0 1 2 3 4 5 6 7 8 9 10

x 109

0

5

10

15

f (Hz)

Q

x full model- QCO PROM

4510/22/2010

•4 turn RF inductor with substrate•Circle: training data•Triangle: test data

Summary (1)

10/22/2010 46

•Motivation for model reduction in design automation•PDE high order ODE reduced ODE•Parameterized reduced modeling facilitates design

•Model reduction based on rational transfer function fitting

•H problem difficult, resort to anti-stable relaxation

•Relaxation easy to solve, closely related to H problem

•Quasi-convex optimization•Efficient algorithms exist (e.g. cutting plane method)•Cutting plane method in model reduction setting

Summary (2)

10/22/2010 47

•Parameterized model reduction•Reduced rational transfer function, coefficients are

function of design parameters•Easily extended from non-parameterized case, except

positivity check is difficult

•Positivity check of multivariate polynomials•Univariate case easy, quadratic case easy•General case requires semidefinite programs, only

sufficient•Related to sum of squares optimization

Some References (1)

10/22/2010 48

•Parameterized reduced modeling•Moment matching: Eric Grimme’s PhD thesis•Parameterized moment matching:

L. Daniel, O. Siong, C. L., K. Lee, and J. White, “A multiparameter moment matching model reduction approach for generating geometrically parameterized interconnect performance models,” IEEE Trans. on Computer-Aided Design of Integrated Circuits and Systems, vol. 23, no. 5, pp. 678–693.

•Parameterized rational fitting:Kin Cheong Sou; Megretski, A.; Daniel, L.; , "A Quasi-Convex Optimization Approach to Parameterized Model Order Reduction," Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on , vol.27, no.3, pp.456-469, March 2008

•MIMO rational fitting/interpolation:A. Sootla, G. Kotsalis, A. Rantzer, “Multivariable Optimization-Based Model Reduction”, IEEE Transactions on Automatic Control, 54:10, pp. 2477-2480, October 2009

Lefteriu, S. and Antoulas, A. C. 2010. A new approach to modeling multiport systems from frequency-domain data. Trans. Comp.-Aided Des. Integ. Cir. Sys. 29, 1 (Jan. 2010), 14-27

Some References (2)

10/22/2010 49

•Convex/quasi-convex optimization•Convex optimization:

S. Boyd and L. Vandenberghe, “Convex Optimization”, Cambridge University Press, 2004.

•Ellipsoid Cutting plane method:Bland, Robert G., Goldfarb, Donald, Todd, Michael J. Feature Article--The Ellipsoid Method: A SurveyOPERATIONS RESEARCH 1981 29: 1039-1091

•Multivariate polynomials and sum of squares•Ordinary polynomial case: Pablo Parrilo’s PhD thesis•Trigonometric polynomial case:

B. Dumitrescu, “Positive Trigonometric Polynomials and Signal Processing Applications”, Springer, 2007