Solving partial differential equationswith exponential integrators

Jitse Niesen

in collaboration withWill Wright (La Trobe University, Melbourne)

Applied maths seminar, Leeds, 21 January 2008

Outline

I The problemI Solving PDEs via semi-discretizationI Stiffness and its effects

I Exponential integratorsI Exponential Euler methodI Construction of higher-order methodsI A predictor-corrector methodI Example using Fourier transform

I Computing the matrix exponentialI Pade with scaling and squaringI Krylov methodsI Example using finite differences

I Conclusions

Solving ordinary differential equations

Consider the ordinary differential equation

u′ = f (u), u(t0) = u0.

The derivative can be approximated by

u′(t) ≈ u(t + h)− u(t)

h(h small).

This leads to the Euler method:

un+1 = un + hf (un).

The solution of the ODE at time t = nh is approximated by un.

Solving partial differential equations

The Allen–Cahn equation:∂u

∂t=∂2u

∂x2+ u − u3.

x0∆x

h

x1 x2 · · · xM−1 xM xM+1t0

t1

t2

∂2

∂x2u(x , t) ≈ 1

∆x2

(u(x −∆x , t)− 2u(x , t) + u(x + ∆x , t)

)

Solving partial differential equations II

This approximation converts the PDE into a system of ODEs:u′1u′2...

u′M−1

u′M

=1

∆x2

−2 11 −2 1

. . .. . .

. . .

1 −2 11 −2

u1

u2...

uM−1

uM

+

u1 − u3

1

u2 − u32

...uM−1 − u3

M−1

uM − u3M

This ODE can be solved by the Euler method.

However, the ODE is stiff: Df has very negative eigenvalues.The most negative one is ≈ −4/∆x2.

Compare σ( ∂2

∂x2 ) = (−∞, 0].

Stiffness and its effects

Euler method: un+1 = un + hf (un).

If you solve a stiff ODE with the Euler method then you have topick a sufficiently small time step h, otherwise the numerical resultbecomes unstable.In the example, h < 1

2∆x2; this is the CFL condition.

Backward Euler method: un+1 = un + hf (un+1).

The backward Euler method does not suffer from stiffness, butrequires solving a system of nonlinear equations. This system canget very large, especially in multiple dimensions.

Better methods than (backward) Euler exist, but the basic problemremains the same:

How to solve a large system of stiff ODEs?

Outline

I The problemI Solving PDEs via semi-discretizationI Stiffness and its effects

I Exponential integratorsI Exponential Euler methodI Construction of higher-order methodsI A predictor-corrector methodI Example using Fourier transform

I Computing the matrix exponentialI Pade with scaling and squaringI Krylov methodsI Example using finite differences

I Conclusions

Exponential Euler method

To solve: u′ = Lu + N(u)

Variation of constants

u(t) = etLu(0) +

∫ t

0e(t−τ)LN(u(τ)) dτ

≈ etLu(0) +

∫ t

0e(t−τ)LN(u(0)) dτ

= etLu(0) +etL − 1

tLN(u(0)).

This leads to the exponential Euler method

un+1 = ehLun +ehL − 1

hLN(un).

This method is not affected by stiffness in L. (Certaine 1960)

How to achieve higher order

The nonautonomous differential equation

u′(t) = Lu(t) + v0 +1

1!tv1 +

1

2!t2v2 + · · ·

has exact solution

u(t) = etLu(0) + hϕ1(tL)v0 + h2ϕ2(tL)v1 + h3ϕ3(tL)v2 + · · ·

The ϕ functions are defined by

ϕ1(x) =ex − 1

x

ϕ2(x) =ex − 1− x

x2

ϕ3(x) =ex − 1− x − 1

2x2

x3

A predictor-corrector method

We are using predictor-corrector methods like

I Predictor is exponential Adams–Bashforth:

u∗n+1 = ehLun + hϕ1(hL)N(un) + h2ϕ2(hL)(N(un)−N(un−1)

)I Corrector based on exponential Adams–Moulton:

un+1 = u∗n+1 + h2ϕ2(hL)(N(u∗n+1)− 2N(un) + N(un−1)

)I A posteriori error estimate, used to change step size h

This method has order 2, i.e., error = O(h2).We have constructed methods up to order 6.

Other exponential methods also exist (without corrector step,based on Runge–Kutta, using exact Jacobian).

Evaluating ϕ functions for scalars

Using the definition, e.g., ϕ2(x) = (ex − 1− x)/x2,leads to loss of precision for small x due to subtraction of twonearly equal numbers.

Therefore, use Taylor or Pade approximations:

ϕ2(x) ≈ c0 + c1x + c2x2 + c3x

3

d0 + d1x + d2x2 + d3x3when |x | small.

Also combine ϕ functions using ϕk(z) = zϕk+1(z) + 1k! .

For example, ehLv0 + ϕ1(hL)v1 + ϕ2(hL)v2 = ϕ2(hL)w0 + w1.

But what to do for matrices? Sometimes, L can be diagonalized.If L approximates Laplacian ∇2 with periodic boundary conditions,

L = F−1DF =⇒ ϕ(L) = F−1ϕ(D)F .

Gray–Scott equation

ut = du∇2u − uv2 + f (1− u), x , y ∈ [0, 1]

vt = dv∇2v + uv2 − (f + k)v

with periodic boundary conditions.

Outline

I The problemI Solving PDEs via semi-discretizationI Stiffness and its effects

I Exponential integratorsI Exponential Euler methodI Construction of higher-order methodsI A predictor-corrector methodI Example using Fourier transform

I Computing the matrix exponentialI Pade with scaling and squaringI Krylov methodsI Example using finite differences

I Conclusions

Computing the matrix exponential

ϕ functions for matrices are like the matrix exponential eA.

“Least dubious” way uses Pade approximation (valid when ‖A‖small) with scaling and squaring: (Moler & Van Loan 1978)

1. Scaling: Find least k such that ‖B‖ < 5.4 with B = 2−kA.

2. Pade: eB ≈ c0 + c1B + c2B2 + · · ·+ c13B

13

d0 + d1B + d2B2 + · · ·+ d13B13.

3. Squaring: Compute eA by squaring eB k times. [e2B = (eB)2]

Constants 5.4 and 13 are optimal. (Higham 2005)

Use same method for ϕ functions with modified squaring step:

ϕk(2B) =1

2k

(ϕ0(B)ϕk(B) +

k∑j=1

1

(k − j)!ϕj(B)

).

Krylov method

Pade approximation with squaring and scaling is too costly forlarge matrices (N ' 103).

Idea of Krylov-subspace methods is to project the action of A onthe Krylov subspace with dimension m� N:

span{v ,Av ,A2, . . . ,Am−1v}.

The Lanczos (if A symmetric) or Arnoldi algorithm produces anorthogonal N-by-m matrix Vm and an m-by-m matrix Hm such that

A ≈ VmHmV Tm =⇒ ϕ(A) ≈ Vmϕ(Hm)V T

m .

Hm is small so we can compute ϕ(Hm) via Pade.(Friesner, Tuckerman & Dornblaser 1989)

Adaptivity in the Krylov process

Recall: The nonautonomous differential equation

u′(t) = Lu(t) + v0 +1

1!tv1 +

1

2!t2v2 + · · ·

has exact solution

u(t) = etLu(0) + hϕ1(tL)v0 + h2ϕ2(tL)v1 + h3ϕ3(tL)v2 + · · ·

This can be used to split the computation of ϕ(A) in substeps.

We have two parameters to play with:

I The dimension m of the Krylov subspace.

I The size of the substeps.

Adapt them to achieve the required precision with the least effort,using an error estimate due to Saad (1992) and Sidje (1998).

Overview of the algorithm

To solve: u′ = Lu + N(u) over time interval [0,T ].

1. Split [0,T ] in smaller intervals.

2. Three stages per step: predictor, corrector, error estimate.

3. Every stage requires computation of a linear combination of ϕfunctions, which we reduces to one ϕ function.

4. Split the computation of the ϕ function in substeps.

5. Reduce the size of the matrix by going to the Krylov subspace.

6. Use Pade with scaling and squaring to compute theϕ function of the smaller matrix.

We use adaptivity at 1, 4 and 5.

Example: Allen–Cahn in two dimensions

The equation we considered at the beginning:

ut = α∇2u + u − u3, x , y ∈ [0, 1], t ∈ [0, 1]

Neumann boundary conditions, α = 0.1, 100× 100 grid.

What is the effect of adapting the Krylov subspace dimension m?

m = {15, 15, 15} m = {32.4, 9.7, 3.1} m = {32, 10, 3}(adaptive)

steps 34 35 34substeps 601 114 1371

L · v 9015 1872 10871eA 679 490 2101

time 39.41 12.98 58.94error 1.5 · 10−4 1.5 · 10−4 1.6 · 10−4

Example: Allen–Cahn in two dimensions II

100

101

102

10−12

10−10

10−8

10−6

10−4

10−2

100

102

cpu time

glob

al e

rror

2D Allen−Cahn α = 0.1

odeexp5oderos4ode15sradau5exp4

Implicit: ode15s (Matlab), radau5 (Hairer & Wanner)Exponential: odeexp5 (us), oderos4 (Caliari & Ostermann),

exp4 (Hochbruck, Lubich & Selhofer)

Discussion

I All methods except ours are given the exact Jacobian.

I Exponential methods require rhs = Lu + N(u) with stiffnessconcentrated in L.

I Performance depends on stiffness:I Explicit methods best on non-stiff problems.I Exponential integrators promising on mildly stiff problems, but

not enough data to be sure.I Implicit methods better on very stiff problems ?

(Krylov approximation converges slowly)

I Alternative methods for evaluating the ϕ function:I Leja point interpolation (Caliari & Ostermann)I Using contour integrals (Schmelzer & Trefethen)I RD-rational approximations (Moret & Novati 2004)I Rational Krylov (Grimm & Hochbruck)

Implicit-explicit (IMEX) method

Our comparison did not include IMEX methods.

Recall backward Euler method: un+1 = un + hf (un+1).Use the splitting f (u) = Lu + N(u) to get

un+1 = un + h(Lun+1 + N(un+1)

).

This is a nonlinear equation. However, if stiffness is in linear part,then why not use the linear equation

un+1 = un + h(Lun+1 + N(un)

).

Can be solved with Krylov techniques.

What is faster: ehLv or (I − hL)−1v?

Theory suggests ehLv . However, in many situations preconditioninghelps a lot for (I − hL)−1v .

Conclusion

Many exponential integrators have been constructed.Some can compete with state-of-the-art implicit methods.

However, some important question are still open:

I How do exponential integrators compare to implicit and IMEXmethods based on subspace techniques?

I How do exponential integrators fare on real-world problems?

HELP!

Conclusion

Many exponential integrators have been constructed.Some can compete with state-of-the-art implicit methods.

However, some important question are still open:

I How do exponential integrators compare to implicit and IMEXmethods based on subspace techniques?

I How do exponential integrators fare on real-world problems?

HELP!