Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite...

61
Stochastic Finite Elements and Fast Iterative Solvers David Silvester & Catherine Powell School of Mathematics University of Manchester http://www.maths.manchester.ac.uk/djs/ Silvester 2007 – p. 1/54

Transcript of Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite...

Page 1: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Stochastic Finite Elements andFast Iterative Solvers

David Silvester & Catherine PowellSchool of Mathematics

University of Manchester

http://www.maths.manchester.ac.uk/djs/

Silvester 2007 – p. 1/54

Page 2: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Joint work with

• Jonathan Boyle (EPSRC EP/C000528/1)• Oliver Ernst & Elisabeth Ullmann (TU Bergakademie

Freiberg, ARC Project 1279)

Silvester 2007 – p. 2/54

Page 3: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Outline

A Toy Problem (deterministic)

• P-IFISS Toolbox• HSL_MI20 Algebraic Multigrid

A Practical Problem (uncertain/stochastic)

• Monte Carlo Method• Stochastic Finite Element Method

• Dundee 1981–1987

Silvester 2007 – p. 3/54

Page 4: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

A Toy Problem

Given boundary “pressure head” data g, and an isotropicpermeability matrix A = µI2, such that

0 < µ∗ ≤ µ(~x) ≤ µ∗ <∞ ∀~x ∈ D ⊂ R2 :

we want to compute p such that

−∇ · A∇p = 0 in D,

p = g on ΓD,

A∇p · ~n = 0 on ΓN .

Silvester 2007 – p. 4/54

Page 5: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

A Toy Problem

Given boundary “pressure head” data g, and an isotropicpermeability matrix A = µI2, such that

0 < µ∗ ≤ µ(~x) ≤ µ∗ <∞ ∀~x ∈ D ⊂ R2 :

we want to compute p such that

−∇ · A∇p = 0 in D,

p = g on ΓD,

A∇p · ~n = 0 on ΓN .

Or alternatively, to compute the pair (~u, p) such that

A−1~u+ ∇p = 0 in D,

∇ · ~u = 0 in D,

p = g on ΓD; ~u · ~n = 0 on ΓN .Silvester 2007 – p. 4/54

Page 6: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Flow Problem I

Silvester 2007 – p. 5/54

Page 7: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Flow Problem III

Silvester 2007 – p. 6/54

Page 8: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Flow Problem IV

Streamlines of interpolated RT0 flux solution

Silvester 2007 – p. 7/54

Page 9: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

PIFISSPotential (Incompressible) Flow & Iterative

Solution Software Guide ∗

David J. SilvesterSchool of Mathematics, University of Manchester

[email protected].

Catherine E. PowellSchool of Mathematics, University of Manchester

[email protected].

Version 1.0, released 7 February 2007

Contents

1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 A Model Steady-State Diffusion Problem . . . . . . . . . . . . 34 Potential Flow Problems (Groundwater flow) . . . . . . . . . . 75 Directory structure . . . . . . . . . . . . . . . . . . . . . . . . 126 Help facility and function glossary . . . . . . . . . . . . . . . 137 Appendix: List of Test Problems . . . . . . . . . . . . . . . . . 15

∗This software was developed with support from the UK Engineering and Physical Sci-ences Research Council and the British Council under the British–German Academic Re-search Collaboration (ARC) scheme.

Silvester 2007 – p. 8/54

Page 10: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Mixed formulation

Define

Flux space X := ~v ∈ H(div;D) : ~v · ~n = 0 on ΓN )

Pressure space M := L2(D)

We want to find (~u, p) ∈ X ×M such that

(A−1~u, ~v) − (p,∇ · ~v) =< g, ~v · ~n > ∀~v ∈ X,

−(∇ · ~u, q) = 0 ∀q ∈M ;

with (·, ·) denoting the L2(D) inner product and < ·, · >denoting the L2(ΓD) inner product.

Silvester 2007 – p. 9/54

Page 11: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

RT(0) mixed approximation

Silvester 2007 – p. 10/54

Page 12: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Discrete Formulation

Introducing the basis sets

Xh = span~φini=1, Flux basis functions;

Mh = spanψjmj=1; Pressure basis functions.

gives the finite element problem(

M BT

B 0

)(

u

p

)

=

(

g

0

)

,

in terms of the associated discrete matrices

Mij = (µ−1~φi, ~φj), local mass

Bij = −(∇ · ~φj , ψi), divergence

Silvester 2007 – p. 11/54

Page 13: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Fast Solver

We solve the (symmetric–) system Lx = f

(

M BT

B 0

)(

u

p

)

=

(

g

0

)

with M ∈ Rn×n, and B ∈ R

m×n, using MINRES with theblock diagonal preconditioning

P−1 =

(

M−1∗

0

0 Q−1∗

)

.

Silvester 2007 – p. 12/54

Page 14: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Given that the blocks M∗ (mass matrix diagonal) and Q∗

(via amg) satisfy

γ2 ≤ uTMu

uTM∗u≤ Γ2 ∀u ∈ R

n,

θ2 ≤ pTBM−1∗BTp

pTQ∗p≤ Θ2 ∀p ∈ R

m,

then the eigenvalues of the preconditioned problem,(

M BT

B 0

)(

u

p

)

= λ

(

M∗ 0

0 Q∗

)(

u

p

)

,

lie in the union of intervals that are bounded away from zeroand ±∞, independently of h and µ.

Silvester 2007 – p. 13/54

Page 15: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Algebraic Multigrid Ingredients (HSL_MI20)

Fine Grid : Ωh = 1, 2, . . . , NSplitting Routine : Ωh → C ∪ F

Coarse Grid : ΩH = C ⊂ Ωh

Smoother : Symmetric Gauss-Seidel

Interpolation Operator : IhH

Restriction Operator : IHh = (Ih

H)T

Coarse-grid Operator : AH = IHh A

hIhH

Convergence theory available for M-matrices.

BM−1∗BT is symmetric, positive definite and diagonally

dominant with positive diagonal entries and negativeoff-diagonal entries.

Silvester 2007 – p. 14/54

Page 16: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Sample eigenvalue bounds: AMG V-cycle

θ2 ≤ pTBM−1∗BTp

pTQ∗p≤ Θ2 ∀p ∈ R

m

1. µ = 1

2. µ =(

1 + 103(

x2 + y2))−1

3. µ = 10−3 in Ω∗, µ = 1 in Ω\Ω∗

1. 2. 3.

θ2 0.954 0.950 0.954Θ2 1 1 1

Silvester 2007 – p. 15/54

Page 17: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Sample MINRES iteration counts

h 116

132

164

1128

1. Ideal 26 26 26 26AMG 26 (0.18) 26 (0.48) 26 (1.90) 26 (9.06)

2. Ideal 26 26 26 26AMG 26 (0.20) 26 (0.65) 26 (2.57) 26 (11.57)

3. Ideal 25 25 25 25AMG 25 (0.20) 26 (0.60) 27 (2.72) 27 (10.64)

Silvester 2007 – p. 16/54

Page 18: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

For details, check out

Catherine Powell & David SilvesterOptimal Preconditioning for Raviart-Thomas MixedFormulation of Second-Order Elliptic ProblemsSIAM J. Matrix Anal. Appl., 25, 2004.

David Silvester & Catherine PowellPIFISS Potential (Incompressible) Flow & IterativeSolution Software guide,MIMS Eprint 2007.14, February 2007.

Silvester 2007 – p. 17/54

Page 19: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

A Toy Problem (deterministic)

• P-IFISS Toolbox• HSL_MI20 Algebraic Multigrid

A Practical Problem (uncertain/stochastic)

• Monte Carlo Method• Stochastic Finite Element Method

• Dundee 1981–1987

Silvester 2007 – p. 18/54

Page 20: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Waste Isolation Pilot Plant (WIPP) Site

• Operational nuclear waste repository in ChihuahuanDesert in New Mexico.

• Extensive measurements, research and testing prior tostart of operation; hence a lot of data available.

• Domain: rectangular horizontal 2D section throughmost transmissive rock layer (Culebra Dolomite)

• Stochastic data obtained from measurements ofpermeability and pressure head at 41 boreholes[LaVenue et al. (1990)] using geostatistical techniques[Cliffe et al. (2000)].

• Boundary data for pressure head on the domainboundary is obtained through a calibration process[LaVenue et al. (1990)].

Silvester 2007 – p. 19/54

Page 21: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

0 0.5 1 1.5 2

x 104

0

0.5

1

1.5

2

2.5

3

x 104

mE

mN

WIPP Site: Bore Locations

H1H2H3

H4

H5H6

H7

H9

H10

H11

H12

H14

H15H16

H17

H18

DOE1

DOE2

P14

P15

P17

P18

WIPP12WIPP13

WIPP18WIPP19WIPP21WIPP22

WIPP25

WIPP26

WIPP27

WIPP28

WIPP30

ERDA9

CB1

ENGLE

USGS1

D268

AEC7

0 0.5 1 1.5 2

x 104

0

0.5

1

1.5

2

2.5

3

x 104 WIPP Site: Sandia Grid

mE

mN

Silvester 2007 – p. 20/54

Page 22: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Modelling Uncertainty

We model permeability A = µI2 as a random field withstochastic transmissivity µ(~x) ∼ T (~x, ω). More precisely,

• (Ω,B, P ) is a complete probability space;

• T : D × Ω → R is a family of random variables indexedby ~x ∈ D: or a “random field” (RF);

• ∀~x ∈ D: T (~x, ·) is a random variable (RV);

• ∀ω ∈ Ω: T (·, ω) is a realization of the transmissivity field.

Silvester 2007 – p. 21/54

Page 23: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Basic Definitions

〈ξ〉 :=

Ω

ξ(ω) dP (ω) expected valueof RV ξ : Ω → R

T (~x) := 〈T (~x, ·)〉 meanof RF T at ~x ∈ D

covT (~x,~y) :=〈(T (~x, ·) − T (~x))(T (~y, ·) − T (~y))〉 covarianceof RF T at ~x, ~y ∈ D

varT (~x) := covT (~x, ~x) varianceof RF T at ~x ∈ D

σT (~x) :=√

varT (~x) standard deviationof RF T at ~x ∈ D.

Silvester 2007 – p. 22/54

Page 24: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Randomization

We restrict random variables to those with finite varianceL2

P (Ω): the space of square integrable random variableswith respect to P :

ξ : Ω → R, 〈ξ2〉 :=

Ω

ξ(ω)2 dP (ω) <∞.

We look for a weak solution (~u, p) in the tensor productspace (X ×M) ⊗ L2

P (Ω).

Silvester 2007 – p. 23/54

Page 25: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Randomization

We restrict random variables to those with finite varianceL2

P (Ω): the space of square integrable random variableswith respect to P :

ξ : Ω → R, 〈ξ2〉 :=

Ω

ξ(ω)2 dP (ω) <∞.

We look for a weak solution (~u, p) in the tensor productspace (X ×M) ⊗ L2

P (Ω). The Stochastic Mixed Formulationis:

Find (~u, p) ∈ (X ×M) ⊗ L2P (Ω) such that

〈(T−1~u, ~v)〉 − 〈(p,∇ · ~v)〉 = 〈< g, ~v · ~n >〉 ∀~v ∈ X ⊗ L2P (Ω),

−〈(q,∇ · ~u)〉 = 0 ∀q ∈M ⊗ L2P (Ω).

Silvester 2007 – p. 23/54

Page 26: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

The associated Stochastic Primal Formulation is to findp ∈ Xg ⊗ L2

P (Ω) such that

〈(T ∇p,∇q)〉 = 0 ∀q ∈ X ⊗ L2P (Ω).

where Xg := ~v ∈ H1(D) : p = g on ΓD and X := X0.

Silvester 2007 – p. 24/54

Page 27: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

The associated Stochastic Primal Formulation is to findp ∈ Xg ⊗ L2

P (Ω) such that

〈(T ∇p,∇q)〉 = 0 ∀q ∈ X ⊗ L2P (Ω).

where Xg := ~v ∈ H1(D) : p = g on ΓD and X := X0.

If this problem is to be well posed we require that

0 < µ∗ ≤ T (~x, ω) ≤ µ∗ <∞ a.e. in D × Ω.

which rules out Gaussian random fields ...

Silvester 2007 – p. 24/54

Page 28: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

... unless the mean value (T (~x), assumed constant) is largecompared to the variance and we cheat!

To apply Monte Carlo we generate NR realizations ofGaussian RF T−1 with mean 1, variance 0.01, and solve adeterministic flow problem for each realization:

NR max varp max varuxmax varuy

100 2.0668e-4 0.0113 3.5023e-41000 2.3266e-4 0.0084 2.5289e-45000 2.3108e-4 0.0087 2.7190e-4

50000 2.3467e-4 0.0084 2.7288e-4∞ 2.29· · ·e-4

Such a naive approach will not work for “larger” variancehowever ...

Silvester 2007 – p. 25/54

Page 29: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

In practice a lognormal distribution is assumed T = exp(Z)where Z is a homogeneous, isotropic, random field withzero mean which is associated with the covariance function

C(~x, ~y) = σ2 exp (−‖~x− ~y‖λ

).

For the WIPP site we have “known” values of σ and λ.Silvester 2007 – p. 26/54

Page 30: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Stochastic Finite Element Discretization

Primal problem: find p ∈ Xg ⊗ L2P (Ω) such that

〈(T ∇p,∇q)〉 = 0 ∀q ∈ X ⊗ L2P (Ω).

Step 1: Define deterministic discretization.

We choose a suitable finite dimensional subspace

Xh = spanφ1(~x), φ2(~x), . . . , φnx(~x) ⊂ X.

This can be done independently of the stochasticdiscretization.

Silvester 2007 – p. 27/54

Page 31: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Step 2: Define finite dimensional stochasticity.

We choose M independent random variablesξm : Ω → R, m = 1, 2, . . . ,M, which sufficiently capture thestochastic variability of the problem.Consequence: the RF occurring in stochastic PDE satisfies

T (~x, ω) ∼ T (~x, ξ1(ω), . . . , ξM (ω)) =: T (~x, ξ(ω)),

with independent RVξmMm=1. Set further

Γm := ξm(Ω) range space of RV ξmMm=1,

ρm : Γm → [0,∞) density function of ξm,

Γ := Γ1 × · · · × ΓM joint range of vector ξ = (ξ1, . . . , ξM ),

ρ(ξ) := ρ1(ξ1) · . . . · ρM (ξM ) joint probability density of ξ1, . . . , ξM .Silvester 2007 – p. 28/54

Page 32: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Replacing L2P (Ω) with L2

ρ(Γ), the variational formulationbecomes

Find p ∈ Xg ⊗ L2ρ(Γ) such that

〈a(p, q)〉 = 0 ∀q ∈ X ⊗ L2ρ(Γ)

where

〈a(p, q)〉 =

Γ

ρ(ξ)

D

T (~x, ξ)∇p(~x, ξ) · ∇q(~x, ξ) d~x dξ,

We have thus transformed the stochastic problem into adeterministic problem with an M -dimensional parameter ξ.

Silvester 2007 – p. 29/54

Page 33: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Step 3: Define discretization of W := L2ρ(Γ)

Given a finite dimensional subspace (common choices aregiven below)

W h = spanψ1(ξ), ψ2(ξ), . . . , ψnξ(ξ) ⊂W = L2

ρ(Γ),

the tensor product space X ⊗W is approximated by

Xh⊗W h =

v ∈ L2(D×Γ) : v ∈ spanφ(~x)ψ(ξ) : φ ∈ Xh, ψ ∈W h

.

The trial function ph ∈ Xh ⊗W h is thus of the form

ph(~x, ξ) =∑

i,j

pi,j φi(~x)ψj(ξ)

with a set of nx · nξ coefficients pi,j, nx = dimXh,nξ = dimW h.

Silvester 2007 – p. 30/54

Page 34: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Construction of W h

We discretize each variable of W = L2ρ1

(Γ1) ⊗ · · · ⊗ L2ρM

(ΓM )

independently and form tensor product space

W h = span

ψα(ξ) =M∏

j=1

ψαj(ξj) : ψαj

∈W hj ⊂ Lρj

(Γj)

, α ∈ NM0 .

piecewise polynomials (requires bounded intervals Γj)[Deb, Babuška & Oden (2001)] [Babuška et al. (2004)]

global polynomials known as Polynomial Chaos (PC)when ξm independent Gaussian, and GeneralizedPolynomial Chaos for other probability distributions,[Ghanem & Spanos (1991)] [Xiu & Karniadakis (2002)][Babuška et al. (2004)]

reduced tensor products fixed total degree, sparse grids[Schwab & Todor (2003)] [Matthies & Keese (2004)] [Xiu(2007)] Silvester 2007 – p. 31/54

Page 35: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Galerkin Approximation

Inserting the trial and test functions

ph(~x, ξ) =∑

i,j

pi,j φi(~x) ψj(ξ), qh(~x, ξ) = φk(~x) ψℓ(ξ)

into the variational formulation gives

〈a(ph, qh)〉 = 0 ∀qh,

with

〈a(ph,qh)〉=∑

i,j

(∫

Γ

ρ(ξ)ψj(ξ)ψℓ(ξ)

D

T (~x, ξ)∇φi(~x)·∇φk(~x) d~x dξ

)

pi,j

=:∑

i,j

(∫

Γ

ρ(ξ)ψj(ξ)ψℓ(ξ)[K(ξ)]i,k dξ

)

pi,j .

Silvester 2007 – p. 32/54

Page 36: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Semi-Discrete System

The resulting systemAp = f

has dimension nx · nξ and has the block form

A =

A1,1 . . . A1,nξ

......

Anξ,1 . . . Anξ,nξ

, p =

p1

...pnξ

, f =

f1...

fnξ

,

with deterministic problem matrices defining the blocks

Aℓ,j = 〈ψjψℓK(ξ)〉 ∈ Rnx×nx , ℓ, j = 1, . . . , nξ.

Silvester 2007 – p. 33/54

Page 37: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Semi-Discrete System

The resulting systemAp = f

has dimension nx · nξ and has the block form

A =

A1,1 . . . A1,nξ

......

Anξ,1 . . . Anξ,nξ

, p =

p1

...pnξ

, f =

f1...

fnξ

,

with deterministic problem matrices defining the blocks

Aℓ,j = 〈ψjψℓK(ξ)〉 ∈ Rnx×nx , ℓ, j = 1, . . . , nξ.

The final ingredient is the representation of K(ξ) : there aretwo generic approaches ...

Silvester 2007 – p. 33/54

Page 38: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

(a) Karhunen-Loève (KL) expansion of RF (linear inξ)

If our postulated random field T has a continuouscovariance function, then we can define a set ofuncorrelated random variables ξ1, ξ2, . . . via

T (~x, ξ) = T (~x) + σ∞∑

m=1

λm Tm(~x)ξm,

where Tm are the eigenfunctions of the covariance operatorassociated with the random field T .

Truncating the series after M terms (setting T0 := T ) gives

[K(ξ)]i,k = [K0]i,k + σM∑

m=1

λm[Km]i,k ξm, [Km]i,k = (Tm∇φk,∇φi),

[G0]ℓ,k = 〈ψkψℓ〉, [Gm]ℓ,k = 〈ξmψkψℓ〉,

A = G0 ⊗ K0 +σ∑

m

λmGm ⊗ Km.Silvester 2007 – p. 34/54

Page 39: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Diagonalization of the Galerkin system

When the stochastic FE space consists of polynomials ofseparate degree d in M random variables, i.e.,nξ = (d+ 1)M , then the stochastic basis ψj can be chosenin such a way that Gm are diagonal, and hence A is blockdiagonal, [Babuška et al. (2004)].

This is not possible when using the (smaller) space ofcomplete polynomials of degree d, i.e., nξ =

(

M+dd

)

. [Ernst &Ullmann (in preparation)].

Silvester 2007 – p. 35/54

Page 40: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Example: M = 4 Gaussian RVs, G0, G1, G2, G3, G4.

0 2 4 6 8 10 12 14 160

2

4

6

8

10

12

14

16

nnz = 55

total degree ≤ 2

0 5 10 15 20 25 30 350

5

10

15

20

25

30

35

nnz = 155

total degree ≤ 3

Silvester 2007 – p. 36/54

Page 41: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Example: flow in a square

Ω = [0, 1] × [0, 1], with boundary conditions: p = 1 on0 × [0, 1], p = 0 on 1 × [0, 1] , ~u · ~n = 0 on (0, 1) × 0, 1 .

Mean problem solution

~u(x, y) =

[

1

0

]

, p(x, y) = 1 − x.

Deterministic discretization:

• RT (0) mixed approximation

• uniform square grid of size h

Silvester 2007 – p. 37/54

Page 42: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Example: flow in a square

Stochastic discretization:

• KL expansion of Gaussian RF T−1:

T (~x, ω) = T (~x)+σM∑

m=1

λmTm(~x)ξm(ω), T (~x) = 1, σ = 0.1

• Covariance function: double exponential,

c(~x, ~y) = σ2 exp(−|x1 − y1|c1

− |x2 − y2|c2

), c1 = c2 = 1.

• Stochastic FE space: Hermite polynomial chaos of(individual) degree ≤ d in M random variables.

• Biorthogonal basis, Galerkin system decouples into2 × 2 block diagonal system.

Silvester 2007 – p. 38/54

Page 43: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Computed Solutions

0

0.5

1

0

0.5

10

1

2

3

4

x 10−4

x

Variance of uy

y 0

0.5

1

0

0.5

16.5

7

7.5

x 10−3

x

Variance of ux

y

Silvester 2007 – p. 39/54

Page 44: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Convergence of Computed Variance

1 2 3 4 52.22

2.23

2.24

2.25

2.26

2.27

2.28

2.29

2.3

2.31x 10

−4

d

M=5M=6M=7M=8M=9

max varp

1 2 3 4 50

1

2

x 10−4

d

M=5M=6M=7M=8M=9

max varuy

Silvester 2007 – p. 40/54

Page 45: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Fast Solver

We solve the (symmetric–) block systems Lkx = fk

(

M0 + σ∑M

m=1Gkm

√λi Mm BT

B 0

)(

u

p

)

=

(

gk

0

)

with weighted mass matrices [Mm]i,j = (Tmφj , φi) andcoefficients Gk

m = 〈ξmψkψk〉 using ...

Silvester 2007 – p. 41/54

Page 46: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Fast Solver

We solve the (symmetric–) block systems Lkx = fk

(

M0 + σ∑M

m=1Gkm

√λi Mm BT

B 0

)(

u

p

)

=

(

gk

0

)

with weighted mass matrices [Mm]i,j = (Tmφj , φi) andcoefficients Gk

m = 〈ξmψkψk〉 using ... MINRES with the blockdiagonal preconditioning

P−1 =

(

M∗

−1 0

0 Q∗

−1

)

where• M∗ ∼M0 (mass matrix diagonal);

• Q∗ ∼ BM∗

−1BT (via amg).

Silvester 2007 – p. 41/54

Page 47: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Solver performance : dependence onh

Example: M = 4, d = 2, (nξ = 81)

h−1 av. its/system av. solve time setup time32 41 0.48 0.4764 41 1.92 1.60

128 42 9.03 11.3256 40 36.35 125.6

Silvester 2007 – p. 42/54

Page 48: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Solver performance: dependence ond,M

M = 2 M = 3 M = 4

d nξ its/system nξ its/system nξ its/system

1 4 39 8 40 16 40

2 9 39 27 41 81 41

3 16 40 64 42 256 42

4 25 40 125 42 625 43

M = 5 M = 6 M = 7

1 32 40 64 40 128 41

2 243 41 729 42 2187 42

3 1024 43 4096 43 16384 43

4 3125 43 15625 43 78125 43

Silvester 2007 – p. 43/54

Page 49: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Solver performance : dependence onσ

Example: M = 4, d = 2, (nξ = 81)

σ av. its/system0.1 41

0.2 46

0.3 53

0.4 indef. (1,1) block

Silvester 2007 – p. 44/54

Page 50: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Performance Summary

Our AMG preconditioner is robust with respect to:

• spatial mesh size h;• stochastic discretization parameters M and d.

Iteration counts are sensitive to the ratio σ/µ:the solver breaks down if σ/µ ∼ 1 (when the discreteproblem is not well-posed).

Silvester 2007 – p. 45/54

Page 51: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

For further details, check out

Michael Eiermann, Oliver Ernst & Elisabeth UllmannComputational Aspects of the Stochastic Finite ElementMethodComputing and Visualization in Science, 10, 3–15,2007.

Catherine Powell & Howard ElmanBlock-Diagonal Preconditioning for Spectral StochasticFinite Element SystemsMIMS Eprint 2007.xx, June 2007.

Next question:

Silvester 2007 – p. 46/54

Page 52: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

For further details, check out

Michael Eiermann, Oliver Ernst & Elisabeth UllmannComputational Aspects of the Stochastic Finite ElementMethodComputing and Visualization in Science, 10, 3–15,2007.

Catherine Powell & Howard ElmanBlock-Diagonal Preconditioning for Spectral StochasticFinite Element SystemsMIMS Eprint 2007.xx, June 2007.

Next question: How do we deal with lognormal randomfields?

Silvester 2007 – p. 46/54

Page 53: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

(b) Expansion in polynomial chaos (nonlinear inξ)

T (~x, ξ) = T0(~x) +

nξ∑

m=1

tm(~x) ψm(ξ), ξ = ξ(ω) ∈ RM ,

This generates a “less sparse” Galerkin matrix

A = G0 ⊗ K0 +

nξ∑

m=1

Hm ⊗ Km,

[Hm]ℓ,j = 〈ψm ψℓ ψj〉, [Km]i,k = (tm∇φk,∇φi).

Silvester 2007 – p. 47/54

Page 54: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

(b) Expansion in polynomial chaos (nonlinear inξ)

T (~x, ξ) = T0(~x) +

nξ∑

m=1

tm(~x) ψm(ξ), ξ = ξ(ω) ∈ RM ,

This generates a “less sparse” Galerkin matrix

A = G0 ⊗ K0 +

nξ∑

m=1

Hm ⊗ Km,

[Hm]ℓ,j = 〈ψm ψℓ ψj〉, [Km]i,k = (tm∇φk,∇φi).

This technique can be used with a lognormal RF

T (~x, ω) = exp(G(~x, ω)), with G a Gaussian RF.

A Polynomial Chaos expansion of T can then be explicitlyconstructed from the KL expansion of G ...

Silvester 2007 – p. 47/54

Page 55: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

For further details, look out for

Oliver Ernst, Catherine Powell, David Silvester &Elisabeth UllmannIterative Solution Methods for Mixed Formulations ofPotential Flow Problems with Random CoefficientsMIMS Eprint 2007.xx, August 2007.

Elisabeth UllmannPh.D Thesis, SFemLib software,December 2007.

Silvester 2007 – p. 48/54

Page 56: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Summary

There are two alternative SFEM formulations:

Expansion of RF linear in ξ using tensor products ofpolynomials of separate degree d in M randomvariables:• all Gm diagonal for doubly orthogonal basis,

• decoupled matrix: nξ = (d+ 1)M systems of size nx,

• beats Monte Carlo if nξ < nR (number ofrealizations) for same accuracy.

Expansion of RF nonlinear in ξ using polynomials oftotal degree d:• need to solve fully coupled system of size nξ · nx,

• significantly smaller problem overall: nξ =(

M+dd

)

.

Silvester 2007 – p. 49/54

Page 57: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

A Toy Problem (deterministic)

• P-IFISS Toolbox• HSL_MI20 Algebraic Multigrid

A Practical Problem (uncertain/stochastic)

• Monte Carlo Method• Stochastic Finite Element Method

• Dundee 1981–1987

Silvester 2007 – p. 50/54

Page 58: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Dundee 1981

A R Conn G Dahlquist L M Delves J Douglas JrI S Duff R Fletcher C W Gear G H GolubJ G Hayes P J v.d. Houwen T E Hull P LancasterR J Y Macleod M J D Powell L B Wahlbin

Silvester 2007 – p. 51/54

Page 59: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Dundee 1983

R H Bartels A Bjorck C de Boor H BrunnerT Dupont D M Gay P W Hemker M J D PowellP Raviart L F Shampine A Spence H J StetterPh L Toint G A Watson M F Wheeler J R Whiteman

Silvester 2007 – p. 52/54

Page 60: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Dundee 1985

P Alefeld F Brezzi G J Cooper D GoldfarbS-P Han A Iserles C Van Loan P MarcusA R Mitchell K W Morton M L Overton A H G Rinnooy KanJ M Sanz-Sena L N Trefethen J G Verwer W L Wendland

Silvester 2007 – p. 53/54

Page 61: Stochastic Finite Elements and Fast Iterative Solversnaconf/07/dun07djs.pdf · Stochastic Finite Element Discretization Primal problem: find p∈ Xg ⊗L2 P(Ω) such that h(T∇p,∇q)i

Dundee 1987

I Babuska H Brunner J R Cash R E EwingP Gaffney G H Golub D F Griffiths T J R HughesR Jeltsch U Kulisch K Madsen C A MicchelliM J D Powell D M Sloan G Wanner B Werner

Silvester 2007 – p. 54/54