Solutions for Nonlinear Equations

Lecture 8

Alessandra Nardi

Thanks to Prof. Newton, Prof. Sangiovanni, Prof. White, Jaime Peraire, Deepak Ramaswamy, Michal Rewienski, and Karen Veroy

Outline

• Nonlinear problems• Iterative Methods• Newton’s Method

– Derivation of Newton– Quadratic Convergence– Examples– Convergence Testing

• Multidimensonal Newton Method– Basic Algorithm – Quadratic convergence– Application to circuits

Need to Solve

( 1) 0d

t

VV

d sI I e

Nonlinear Problems - Example

0

1

IrI1 Id

0)1(1

0

11

1

1

IeIeR

III

tV

e

s

dr

11)( Ieg

Nonlinear Equations

• Given g(V)=I

• It can be expressed as: f(V)=g(V)-I

Solve g(V)=I equivalent to solve f(V)=0

Hard to find analytical solution for f(x)=0

Solve iteratively

Nonlinear Equations – Iterative Methods

• Start from an initial value x0

• Generate a sequence of iterate xn-1, xn, xn+1 which hopefully converges to the solution x*

• Iterates are generated according to an iteration function F: xn+1=F(xn)

Ask• When does it converge to correct solution ?• What is the convergence rate ?

Newton-Raphson (NR) MethodConsists of linearizing the system.

Want to solve f(x)=0 Replace f(x) with its linearized version and solve.

Note: at each step need to evaluate f and f’

functionIterationxfxdx

dfxx

xxxdx

dfxfxf

iesTaylor Serxxxdx

dfxfxf

kkkk

kkkkk

)()(

))(()()(

))(()()(

11

11

***

Newton-Raphson Method – Graphical View

Newton-Raphson Method – Algorithm

Define iteration

Do k = 0 to ….

until convergence

• How about convergence?

• An iteration {x(k)} is said to converge with order q if there exists a vector norm such that for each k N:

qkk xxxx ˆˆ1

)()(1

1 kkkk xfxdx

dfxx

2* * * 2

2

*

0 ( ) ( ) ( )( ) ( )( )

some [ , ]

k k k k

k

df d ff x f x x x x x x x

dx dx

x x x

Mean Value theoremtruncates Taylor series

But10 ( ) ( )( )k k k kdf

f x x x xdx

by Newtondefinition

Newton-Raphson Method – Convergence

Subtracting

21 * 1 * 2

2 ( ) [ ( )] ( )( )k k kdf d f

x x x x x xdx d x

Convergence is quadratic

21 * * 2

2( )( ) ( )( )k k kdf d fx x x x x x

dx d x

Dividing through

21

2

21 * *

Let [ ( )] ( )

then

k k

k k k

df d fx x K

dx d x

x x K x x

Newton-Raphson Method – Convergence

Newton-Raphson Method – Convergence

Local Convergence Theorem

If

Then Newton’s method converges given a sufficiently close initial guess (and

convergence is quadratic)

boundedisK

boundeddx

fdb

roay from zebounded awdx

dfa

)

)

2

2

21

2 21 * * *

* 2

2

1 *

*

( ) 2

( ) 1 0,

2 ( ) 1

2 ( ) 2 ( )

1( ) (

( 1

)2

)

k k

k k k k

k k k k k

k kk

dfx x

dx

x x x x

x x x x x x x

f x x fin

x

d x x

or x x x xx

Convergence is quadratic

Newton-Raphson Method – ConvergenceExample 1

1 2

1 *

* *1

2 *

( ) 2

2 ( 0) ( 0)

10 0

( ) 0,

for 02

1( ) ( )

2

0

k k

k k k

k k k

k k

dfx x

dx

x x x

x x x x

or x x x

f x

x

x x

Convergence is linear

Note : not bounded

away from zero

1df

dx

Newton-Raphson Method – ConvergenceExample 2

Newton-Raphson Method – ConvergenceExample 1, 2

0 Initial Guess,= 0x k

Repeat { 1

k

k k kf x

x x f xx

} Until ?

1 1? ?k k kf x thresholdx x threshold

1k k

Newton-Raphson Method – Convergence

Need a "delta-x" check to avoid false convergence

X

f(x)

1kx kx *x

1 a

kff x

1 1 a r

k k kx xx x x

Newton-Raphson Method – ConvergenceConvergence Checks

Also need an " " check to avoid false convergencef x

X

f(x)

1kx kx

*x

1 a

kff x

1 1 a r

k k kx xx x x

Newton-Raphson Method – ConvergenceConvergence Checks

demo2

Newton-Raphson Method – Convergence

X

f(x)

Convergence Depends on a Good Initial Guess

0x

1x2x 0x

1x

Newton-Raphson Method – ConvergenceLocal Convergence

Convergence Depends on a Good Initial Guess

Newton-Raphson Method – ConvergenceLocal Convergence

Nodal Analysis

1 2At Node 1: 0i i

1bv

+

-

1i2i 2

bv

3i+ -

3bv

+

-

NonlinearResistors

i g v

1v2v

1 1 2 0g v g v v

3 2At Node 2: 0i i

3 1 2 0g v g v v

Two coupled nonlinear equations in two unknowns

Nonlinear Problems – Multidimensional Example

* *Problem: Find such tha 0t x F x * N N N and : x F

Multidimensional Newton Method

functionIterationxFxJxx

atrixJacobian M

x

xF

x

xF

x

xF

x

xF

xJ

iesTaylor SerxxxJxFxF

kkkk

N

NN

N

)()(

)()(

)()(

)(

))(()()(

11

1

1

1

1

***

Multidimensional Newton MethodComputational Aspects

)()()( :solve Instead

sparse).not is(it )( computenot Do

)()(:

1

1

11

kkkk

k

kkkk

xFxxxJ

xJ

xFxJxxIteration

Each iteration requires:

1. Evaluation of F(xk)

2. Computation of J(xk)

3. Solution of a linear system of algebraic equations whose coefficient matrix is J(xk) and whose RHS is -F(xk)

0 Initial Guess,= 0x k

Repeat {

1 1Solve for k k k k kFJ x x x F x x

} Until 11 , small en ug o hkk kx x f x

1k k

Compute ,k kFF x J x

Multidimensional Newton MethodAlgorithm

If

1) Inverse is boundedkFa J x

) Derivative is Lipschitz ContF Fb J x J y x y

Then Newton’s method converges given a sufficiently close initial guess (and

convergence is quadratic)

Multidimensional Newton Method Convergence

Local Convergence Theorem

Application of NR to Circuit EquationsCompanion Network

• Applying NR to the system of equations we find that at iteration k+1:– all the coefficients of KCL, KVL and of BCE of

the linear elements remain unchanged with respect to iteration k

– Nonlinear elements are represented by a linearization of BCE around iteration k

This system of equations can be interpreted as the STA of a linear circuit (companion network) whose elements are specified by the linearized BCE.

Application of NR to Circuit EquationsCompanion Network

• General procedure: the NR method applied to a nonlinear circuit whose eqns are formulated in the STA form produces at each iteration the STA eqns of a linear resistive circuit obtained by linearizing the BCE of the nonlinear elements and leaving all the other BCE unmodified

• After the linear circuit is produced, there is no need to stick to STA, but other methods (such as MNA) may be used to assemble the circuit eqns

Note: G0 and Id depend on the iteration count k G0=G0(k) and Id=Id(k)

Application of NR to Circuit EquationsCompanion Network – MNA templates

Application of NR to Circuit EquationsCompanion Network – MNA templates

Modeling a MOSFET(MOS Level 1, linear regime)

d

Modeling a MOSFET(MOS Level 1, linear regime)

DC Analysis Flow DiagramFor each state variable in the system

Implications• Device model equations must be continuous with continuous

derivatives and derivative calculation must be accurate derivative of function (not all models do this - Poor diode models and breakdown models don’t - be sure models are decent - beware of user-supplied models)

• Watch out for floating nodes (If a node becomes disconnected, then J(x) is singular)

• Give good initial guess for x(0)

• Most model computations produce errors in function values and derivatives. Want to have convergence criteria || x(k+1) - x(k) || < such that > than model errors.

Summary

• Nonlinear problems

• Iterative Methods

• Newton’s Method– Derivation of Newton

– Quadratic Convergence

– Examples

– Convergence Testing

• Multidimensonal Newton Method– Basic Algorithm

– Quadratic convergence

– Application to circuits