CHBE 305

Stiffness and Multistep Methods

Stiffness! Stiff systems have rapidly changing components together with other

slowly changing ones.

�

dydt

= −1000y + 3000 − 2000e− t

y(0) = 0

The analytical solution is :y = 3− 0.998e−1000t − 2.002e−t

Stiffness! Stiff systems have rapidly changing components together with other

slowly changing ones.

�

dydt

= −1000y + 3000 − 2000e− t

y(0) = 0

The analytical solution is :y = 3− 0.998e−1000t − 2.002e−t

Stiffness

�

Homogeneous equation :

dydt

= −ay

y(0) = y0

The analytical solution is : y = y0e−at

Euler's method :

yi+1 = yi + dyidt

h ⇒ yi+1 = yi − ayih ⇒ yi+1 = yi 1− ah( )

For stability, we must have : yi+1

yi< 1 ⇒ 1- ah < 1 ⇒ h < 2

aFor a = 1000 : h < 0.002 Adaptive methods are

not the solution

-0.5

0

0.5

1

1.5

2

2.5

0.00 0.05 0.10 0.15

y

x

a=100

Solution with Explicit Euler’s Method

Step # h=0.05 h=0.025 h=0.0050 2 2.000 2.0001 -8 -3.000 1.0002 32 4.500 0.5003 -128 -6.750 0.2504 512 10.125 0.1255 -2048 -15.188 0.0636 8192 22.781 0.0317 -32768 -34.172 0.0168 131072 51.258 0.0089 -524288 -76.887 0.004

10 2097152 115.330 0.002

dydt

= −100y t( ) y 0( ) = 2

Stiffness

�

Homogeneous equation :

dydt

= −ay

y(0) = y0

The analytical solution is : y = y0e−at

Euler's method :

yi+1 = yi + dyidt

h ⇒ yi+1 = yi − ayih ⇒ yi+1 = yi 1− ah( )

For stability, we must have : yi+1

yi< 1 ⇒ 1- ah < 1 ⇒ h < 2

aFor a = 1000 : h < 0.002 Adaptive methods are

not the solution

Implicit Euler Method

�

Implicit form of Euler's method :

yi+1 = yi + dyi+1

dth ⇒ yi+1 = yi − ayi+1h ⇒ 1+ ah( )yi+1 = yi

or yi+1 = yi1+ ah( )

Regardless of the size of the step,

yi+1

yi< 1 ⇒ y i → 0 as i→∞

Implicit vs. Explicit Euler

0

0.5

1

1.5

2

0.00 0.05 0.10 0.15 0.20

y

x

Implicit Euler - h = 0.05

Explicit Eulerh = 0.005

a = 100

Solution of “Stiff” ODE

�

dydt

= −1000y + 3000 − 2000e− t

y(0) = 0

Explicit Euler :

yi+1 = yi + −1000yi + 3000 − 2000e−ti( )h

Implicit Euler :

yi+1 = yi + −1000yi+1 + 3000 − 2000e−ti+1( )h ⇒

yi+1 = yi + +3000h − 2000e− ti+1

1+ 1000h

Solution of “Stiff” ODE

�

dydt

= −1000y + 3000 − 2000e− t

y(0) = 0

Explicit Euler :

yi+1 = yi + −1000yi + 3000 − 2000e−ti( )h

Implicit Euler :

yi+1 = yi + −1000yi+1 + 3000 − 2000e−ti+1( )h ⇒

yi+1 = yi + 3000h − 2000e−ti+1

1+ 1000h

Stiff Systems of ODEs

�

dy1

dt= −5y1 + 3y2

dy2

dt= 100y1 − 301y2

with IC : y1 0( ) = 52.29

y2 0( ) = 83.82

The analytical solution is : y1 = 52.96e−3.9899t − 0.67e−302.0101t

y2 = 517.83e−3.9899t + 65.99e−302.0101t

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5f(t

)

t

e-3.9899 t

e-302.1010t

Stiff Systems of ODEs

�

dy1

dt= −5y1 + 3y2

dy2

dt= 100y1 − 301y2

with IC : y1 0( ) = 52.29

y2 0( ) = 83.82

The analytical solution is : y1 = 52.96e−3.9899t − 0.67e−302.0101t

y2 = 517.83e−3.9899t + 65.99e−302.0101t

0

0.2

0.4

0.6

0.8

1

0 0.01 0.02 0.03 0.04 0.05

f(t)

t

e-3.9899 t

e-302.1010t

Implicit Euler

�

y1,i+1 = y1,i + −5y1,i+1 + 3y2,i+1( )hy2,i+1 = y2,i + 100y1,i+1 − 302y2,i+1( )h

Collecting terms we obtain :1+5h( )y1,i+1 − 3hy2,i+1 = yi,1−1000hy1,i+1 + 1+ 301h( )y2,i+1 = y2,i

For nonlinear ODEs, the system becomes nolinear and the solution ofthe ODE even more difficult.

To compute y1,i+1 and y2,i+1

we must solve this 2 × 2system of linear equations

Methods for Stiff Systems

! Implicit Euler is unconditionally stable , but only first-order accurate.

! Higher order implicit methods can be developed, but the stability limits for such methods are too stringent.

! Gear’s methods are preferred because they have have much larger stability limits. They are based on backward difference formulas.

Dynamical Systems

Higher order ODEs appear very often in engineering applications.

You will see many examples when you take the process control class!

Higher Order ODEs and Eigenvalues

Linear 2nd Order Ordinary Differential Equation:

a2d 2ydt 2 + a1

dydt

+ a0y = F t( )

Equivalent System of 1st Order ODEs:dydt

= z

dzdt

=F t( ) − a1z − a0y

a2

Higher Order ODEs and Eigenvalues

Linear 2nd Order Ordinary Differential Equation:

a2d 2ydt 2 + a1

dydt

+ a0y = F t( )

Equivalent System of 1st Order ODEs:dydt

= z

dzdt

=F t( ) − a1z − a0y

a2

Higher Order ODEs and Eigenvalues

Homogeneous ODE:

a2d 2ydt 2 + a1

dydt

+ a0y = 0

General solution: y = ert

a2r2ert + a1re

rt + a0ert = 0 ⇒

a2r2 + a1r + a0 = 0 Characteristic equation

Characteristic valuesor eigenvalues

Higher Order ODEs and Eigenvalues

Eigenvalues:

r1,r2 =−a1 ± a1

2 − 4a0a2

a0

Discriminant positive: y = c1er1t + c2e

r2 t

Discriminant zero: y = c1 + c2t( )ert

Discriminant negative: y = c1eλ+µi( )t + c2e

λ−µi( )t

But eµit = cosµt + i sinµt

⎫⎬⎪

⎭⎪⇒

⇒ y = c1eλt cosµt + c2e

λt sinµt

Dynamical Systems

Stiffness! Stiff systems have rapidly changing components together with other

slowly changing ones.

�

dydt

= −1000y + 3000 − 2000e− t

y(0) = 0

The analytical solution is :y = 3− 0.998e−1000t − 2.002e−t

Systems of ODEs:Transient Operation of a CSTR

�

A k1⎯ → ⎯ B First order reaction Constant volumeDimensionless Balances :dudt

= 1− u( ) − u ⋅Da ⋅ exp γ 1− 1v( )[ ]

Le dvdt

= 1− v( ) + β ⋅ u ⋅Da ⋅ exp γ 1− 1v( )[ ]

Initial conditions : u = u0, v = v0 at t = 0

Dimensionless Numbers:Le : includes heat capacity of systemDa: includes rate of reaction

Transient CSTR

β=0.15, γ=30, Da=0.115, Le=1080, u(0)=0.7, v(0)=1.0

Transient CSTR

β=0.15, γ=30, Da=0.115, Le=1080, u(0)=0.7, v(0)=1.0

These are not the steady statesolutions!

Transient CSTR

β=0.15, γ=30, Da=0.115, Le=1080, u(0)=0.7, v(0)=1.0

Transient CSTR

β=0.15, γ=30, Da=0.115, Le=1080, u(0)=0.7, v(0)=1.0

MATLAB integrators for stiff systems

! ode15s: Variable order solver based on the numerical differentiation formulas. It is a multistep method that optionally uses the backward differentiation formulas also known as Gear's method. It is used for stiff problems of low to medium accuracy. Recommendation: Try ode15s when ode45 fails, or is very inefficient, and you suspect that the problem is stiff.

! ode23s: Based on a modified Rosenbrock formula of order 2. Because it is a one-step solver, it may be more efficient than ode15s at crude tolerances. It can solve some kinds of stiff problems for which ode15s is not effective.

MATLAB integrators for stiff systems

! ode23t: An implementation of the trapezoidal rule using a "free" interpolant. Use this solver if the problem is only moderately stiff and you need a solution without numerical damping.

! ode23tb: An implementation of TR-BDF2, an implicit Runge-Kutta formula with a first stage that is a trapezoidal rule step and a second stage that is a backward differentiation formula of order two. By construction, the same iteration matrix is used in evaluating both stages. Like ode23s, this solver may be more efficient than ode15s at crude tolerances

Integrating stiff ODEs

d 2ydt 2 − µ 1− y2( ) dydt + y = 0

Initial conditions: y 0( ) = 1, dydt

0( ) = 1

This 2nd-order ODE reduces to a system of two 1st-order ODEs:

dy1

dt= y2

dy2

dt= µ 1− y1

22( )y2 − y1

Problem becomes stifferwith increasing values of µ

MATLAB ODE Solvers

0

1

10

100

1,000

ode45 ode15s

0.72

608.80

0.59

6.20

0.450.29

μ=10μ=100μ=1000

Multistep Methods

Use information from previous points

Multistep Methods

Use information from previous points

Review of Heun’s Method

�

Predictor : yi+10 = yi + f xi,yi( )h

Corrector : yi+1 = yi +f xi,yi( ) + f xi+1,yi+1

0( )2

h

Predictor is O h2( ) and corrector is O h3( ).

Can we improve the predictor?

Non-Self-Starting Heun Method

�

Predictor : yi+10 = yi−1 + f xi,yi( )2h

Corrector : yi+1 = yi +f xi,yi( ) + f xi+1,yi+1

0( )2

h

Predictor is now O h3( ) (albeit with a larger h).

BUT, we need to know the dependent variable a previous point yi-1.

Non-Self-Starting Heun Method

Predictor : yi+10 = yi−1

m + f xi , yim( )2h

Corrector: yi+1j = yi

m +f xi , yi

m( ) + f xi+1, yi+1j−1( )

2h

( for j = 1,2,...m)

Corrector is applied iteratively to refine solution.

Note that yim and yi

m are final results of the corrector

iterations at the previous time steps!

Stopping criterion : εa =yi+1

j − yi+1j−1

yi+1j 100% < εS

Approach

Most multi-step methods:

! Use an open integration method (like the midpoint method) to make an initial estimate. This predictor requires a previous data point.

! Then a closed integration formula (i.e. trapezoidal rule) is iteratively applied to improve the solution.

! To improve the accuracy of multistep methods we should use higher-order predictors and correctors.

Open formulas (Adams-Bashforth)

Second-order open Adams (Adams-Bashforth) formula:

yi+1 = yi + h32f ti , yi( )− 1

2f ti−1, yi−1( )⎛

⎝⎜⎞⎠⎟

Higher nth-order Adams-Bashforth formulas:

yi+1 = yi + h βk f ti=k , yi−k( )k=0

n−1

∑ +O hn+1( )

These formulas are used as predictors

Closed Formulas (Adams-Moulton)

Second-order closed formula (Adams-Moulton):

yi+1 = yi + h12f ti+1, yi+1( ) + 1

2f ti , yi( )⎛

⎝⎜⎞⎠⎟

Higher nth-order Adams-Moulton formulas:

yi+1 = yi + h βk f ti+1−k , yi+1−k( )k=0

n−1

∑ +O hn+1( )

These formulas are used as correctors

Milne’s Method

�

Most common multistep method :

Predictor : yi+10 = yi−3

m + 4h3

2 f im − f i−1

m + 2 f i−2m( )

Corrector : yi+1j = yi−1

m + h3f i−1m + 4 f i

m + f i+1j−1( )

(Simpson's 1 3 rule)

Modifiers : EP = 2829

yim − yi

0( ) EC = − 1

29yi+1m − yi+1

0( )

Gives highly accurate results, but may perform poorly in some cases

Fourth-Order Adams Method

�

Another popular multistep method :

Predictor : yi+10 = yi

m + h 5524

f im − 59

24f i−1m + 37

24f i−2m − 9

24f i−3m⎛

⎝ ⎜

⎞ ⎠ ⎟

(Fourth - order Adams - Bashforth)

Corrector : yi+1j = yi

m + h 924

f i+1j−1 + 19

24f im − 5

24f i−1m + 1

24f i−2m⎛

⎝ ⎜

⎞ ⎠ ⎟

(Fourth - order Adams - Moulton)

Modifiers : EP = 251270

yim − yi

0( ) EC = − 19

270yi+1m − yi+1

0( )More function evaluations than Milne’s method

MATLAB ode113 integrator

! This is a multistep method.

! Implements a variable-order Adams-Bashforth-Moulton algorithm.

! Recommended for stringent error tolerances or computationally intensive ODE functions.