Uncertainty Quantification: UQTk example problems...2012/04/04 · Uncertainty Quantiﬁcation:...

Uncertainty Quantification: UQTk example problems

Bert Debusschere1 Roger Ghanem2

1Sandia National LaboratoriesLivermore, CA, [email protected]

2University of Southern CaliforniaLos Angeles, CA, USA

SIAM 2012 Conference on Uncertainty Quantification

Debusschere (Sandia, USC) UQTk examples SIAM UQ 2012 1 / 81

Acknowledgement

K. Sargsyan, C. Safta, H.N. Najm— Sandia National Laboratories, CA

O.M. Knio — Duke Univ., Durham, NCA. Alexanderian — Johns Hopkins Univ., Baltimore, MD

This work was supported by:

US Department of Energy (DOE), Office of Advanced Scientific Computing Research(ASCR), Scientific Discovery through Advanced Computing (SciDAC)

DOE ASCR Applied Mathematics program.

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly ownedsubsidiary of Lockheed Martin Corporation, for the U.S. Department of Energys National Nuclear Security Administration undercontract DE-AC04-94AL85000.


Outline

1 Introduction to the UQ Toolkit (UQTk)

2 Spectral Polynomial Chaos Expansions (PCEs)

3 Propagation of Uncertainty through Computational Models

4 Characterization of Input Uncertainty

5 Case Study 1: Chemical Mechanism and Input Correlations

6 Case Study 2: Representation of Non-Gaussian Process with PCE

7 Advanced Topics


Introduction

Outline







7 Advanced Topics


Introduction

Uncertainty Quantification Toolkit (UQTk)

A library of C++ and Matlab functions for propagation of uncertaintythrough computational models

Mainly relies on spectral Polynomial Chaos Expansions (PCEs) forrepresenting random variables and stochastic processes

Target usage:

Rapid prototypingAlgorithmic researchTutorials / Educational

Version 1.0 released under the GNU Lesser General Public License

Downloadable from http://www.sandia.gov/UQToolkit/


Introduction

UQTk contents and development/release plans

Currently released (http://www.sandia.gov/UQToolkit/)

C++ Tools for intrusive UQ with PCEs

Under production, planned release Fall 2012

C++ Tools for non-intrusive UQMatlab tools for intrusive and non-intrusive UQKarhunen-Loeve decompositionBayesian inference toolsMany more examples and documentation

Under development

Adding support for multiwavelet based stochastic domaindecompositionsSupport for arbitrary basis types


Polynomial Chaos

Outline







7 Advanced Topics


Polynomial Chaos

Polynomial Chaos Expansions (PCEs)

Represent random variables with finite variance as polynomials ofstandard random variables

U(θ) ≃P∑

k=0

ukΨk(ξ)

Truncated to finite dimension n and order p

Numer of terms P + 1 = (n+p)!n!p!

ξ = (ξ1, ξ2, · · · , ξn) standard i.i.d. random variablesΨk standard orthogonal polynomialsuk spectral modes or PC coefficients

Most common standard Polynomial-Variable pairs:

(continuous) Gauss-Hermite, Legendre-Uniform(discrete) Poisson-Charlier

[Wiener, 1938; Ghanem & Spanos, 1991; Xiu & Karniadakis, 2002]


Polynomial Chaos

PCEs in the UQToolkit

// Initialize PC class

int ord = 5; // Order of PCE

int dim = 1; // Number of uncertain parameters

PCSet myPCSet("ISP",ord,dim,"LU"); // Legendre-Uniform PCEs

// Initialize PC class

int ord = 5; // Order of PCE

int dim = 1; // Number of uncertain parameters

PCSet myPCSet("NISP",ord,dim,"LU"); // Legendre-Uniform PCEs

Currently support Wiener-Hermite, Legendre-Uniform, andGamma-Laguerre (limited), Jacobi-Beta (development version)

PCSet class initializes PC basis type and pre-computes informationneeded for working with PC expansions


Polynomial Chaos

Operations on PCEs in the UQToolkit

// PC coefficients in double*

double* a = new double[npc];

double* b = new double[npc];

double* c = new double[npc];

// Initialization

a[0] = 2.0;

a[1] = 0.1;

...

// Perform some arithmetic

myPCSet.Subtract(a,b,c);

myPCSet.Prod(a,b,c);

myPCSet.Exp(a,c);

myPCSet.Log(a,c);

// PC coefficients in Arrays

Array1D<double> aa(npc,0.e0);

Array1D<double> ab(npc,0.e0);

Array1D<double> ac(npc,0.e0);

// Initialization

aa(0) = 2.0;

aa(1) = 0.1;

...

// Perform arithmetic

myPCSet.Subtract(aa,ab,ac);

myPCSet.Prod(aa,ab,ac);

myPCSet.Exp(aa,ac);

myPCSet.Log(aa,ac);

PC coefficients are either stored in double* vectors or in moreadvanced custom Array1D<double> classes

Functions can take either data type as argument


Propagation of Uncertainty

Outline







7 Advanced Topics



Surface Reaction Model

3 ODEs for a monomer (u),dimer (v), and inert species(w) adsorbing onto a surfaceout of gas phase.

du

dt= az − cu − 4duv

dv

dt= 2bz2 − 4duv

dw

dt= ez − fw

z = 1− u − v − w

u(0) = v(0) = w(0) = 0.0

a = 1.6 b = 20.75 c = 0.04 d = 1.0 e = 0.36 f = 0.016

0 200 400 600 800 1000Time [-]

0.0

0.2

0.4

0.6

0.8

1.0

Species Mass Fractions [-]

u v w

Oscillatory behavior for b ∈ [20.2, 21.2]

(Vigil et al., Phys. Rev. E., 1996; Makeev et al., J. Chem. Phys., 2002)



Surface Reaction Model: Intrusive Spectral Propagation(ISP) of Uncertainty

Assume PCE for uncertain parameter b and for the output variables,u, v ,w

Substitute PCEs into the governing equations

Project the governing equations onto the PC basis functions

Multiply with Ψk and take the expectation

Apply pseudo-spectral approximations where necessary

UQTk elementary operations



Surface Reaction Model: Specify PCEs for inputs andoutputs

Represent uncertain inputs with PCEs with known coefficients:

b =P∑

i=0

biΨi (ξ)

Represent all uncertain variables with PCEs with unknown coefficients:

u =P∑

i=0

uiΨi (ξ) v =P∑

i=0

viΨi (ξ) w =P∑

i=0

wiΨi (ξ) z =P∑

i=0

ziΨi (ξ)



Surface Reaction Model: Substitute PCEs into governingequations and project onto basis functions

du

dt= az − cu − 4duv

d

dt

P∑

i=0

uiΨi = a

P∑

i=0

ziΨi − c

P∑

i=0

uiΨi − 4dP∑

i=0

uiΨi

P∑

j=0

vjΨj

⟨

Ψk

d

dt

P∑

i=0

uiΨi

⟩

=

⟨

aΨk

P∑

i=0

ziΨi

⟩

−⟨

cΨk

P∑

i=0

uiΨi

⟩

−⟨

4dΨk

P∑

i=0

uiΨi

P∑

j=0

vjΨj

⟩



Surface Reaction Model: Reorganize terms

d

dtuk⟨Ψ2

k

⟩= azk

⟨Ψ2

k

⟩− cuk

⟨Ψ2

k

⟩− 4d

P∑

i=0

P∑

j=0

uivj 〈ΨiΨjΨk〉

d

dtuk = azk − cuk − 4d

P∑

i=0

P∑

j=0

uivj〈ΨiΨjΨk〉⟨Ψ2

k

⟩

d

dtuk = azk − cuk − 4d

P∑

i=0

P∑

j=0

uivjCijk

Triple products Cijk =〈ΨiΨjΨk〉〈Ψ2

k〉can be pre-computed and stored for

repeated use



Surface Reaction Model: Substitute PCEs into governingequations and project onto basis functions

dv

dt= 2bz2 − 4duv

d

dt

P∑

i=0

viΨi = 2P∑

h=0

bhΨh

P∑

i=0

ziΨi

P∑

j=0

zjΨj − 4dP∑

i=0

uiΨi

P∑

j=0

vjΨj

⟨

Ψk

d

dt

P∑

i=0

viΨi

⟩

=

⟨

2Ψk

P∑

h=0

bhΨh

P∑

i=0

ziΨi

P∑

j=0

zjΨj

⟩

−⟨

4dΨk

P∑

i=0

uiΨi

P∑

j=0

vjΨj

⟩



Surface Reaction Model: Reorganize terms

d

dtvk⟨Ψ2

k

⟩= 2

P∑

h=0

P∑

i=0

P∑

j=0

bhzizj 〈ΨhΨiΨjΨk〉 − 4d

P∑

i=0

P∑

j=0

uivj 〈ΨiΨjΨk〉

d

dtvk = 2

P∑

h=0

P∑

i=0

P∑

j=0

bhzizj〈ΨhΨiΨjΨk〉

⟨Ψ2

k

⟩ − 4dP∑

i=0

P∑

j=0

uivj〈ΨiΨjΨk〉⟨Ψ2

k

⟩

d

dtvk = 2

P∑

h=0

P∑

i=0

P∑

j=0

bhzizjDhijk − 4dP∑

i=0

P∑

j=0

uivjCijk

Pre-computing and storing the quad product Dhijk becomescumbersome

Use pseudo-spectral approach instead



Surface Reaction Model: Pseudo-Spectral approach forproducts

Introduce auxiliary variable g = z2

g = z2

f = 2bz2 = 2bg

gk =

P∑

i=0

P∑

j=0

zizjCijk

fk = 2

P∑

i=0

P∑

j=0

bigjCijk

Limits the complexity of computing product terms

Higher products can be computed by repeated use of the same binaryproduct rule

Does introduce errors if order of PCE is not large enough



Surface Reaction Model: UQTk implementation

// Build du/dt = a*z - c*u - 4.0*d*u*v

aPCSet.Multiply(z,a,dummy1); // dummy1 = a*z

aPCSet.Multiply(u,c,dummy2); // dummy2 = c*u

aPCSet.SubtractInPlace(dummy1,dummy2); // dummy1 = a*z - c*u

aPCSet.Prod(u,v,dummy2); // dummy2 = u*v

aPCSet.MultiplyInPlace(dummy2,4.e0*d); // dummy2 = 4.0*d*u*v

aPCSet.Subtract(dummy1,dummy2,dudt); // dudt = a*z - c*u - 4.0*d*u*v

All operations are replaced with their equivalent intrusive UQcounterparts

Results in a set of coupled ODEs for the PC coefficients

u, v ,w , z represent vector of PC coefficients

This set of equations is integrated to get the evolution of the PCcoefficients in time



Surface Reaction Model: Second equation implementation

// Build dv/dt = 2.0*b*z*z - 4.0*d*u*v

aPCSet.Prod(z,z,dummy1); // dummy1 = z*z

aPCSet.Prod(dummy1,b,dummy2); // dummy2 = b*z*z

aPCSet.Multiply(dummy2,2.e0,dummy1); // dummy1 = 2.0*b*z*z

aPCSet.Prod(u,v,dummy2); // dummy2 = u*v

aPCSet.MultiplyInPlace(dummy2,4.e0*d); // dummy2 = 4.0*d*u*v

aPCSet.Subtract(dummy1,dummy2,dvdt); // dvdt = 2.0*b*z*z - 4.0*d*u*v

// Build dw/dt = e*z - f*w

aPCSet.Multiply(z,e,dummy1); // dummy1 = e*z

aPCSet.Multiply(w,f,dummy2); // dummy2 = f*w

aPCSet.Subtract(dummy1,dummy2,dwdt); // dwdt = e*z - f*w

Dummy variables used where needed to build the terms in theequationsData structure is currently being enhanced to provide the operationresult as the function return value

Will allow more elegant inline replacement of operators with theirstochastic counterparts



Surface Reaction Model: ISP results

0 200 400 600 800 1000Time [-]

0.0

0.2

0.4

0.6

0.8

1.0


u v w

Assume 0.5% uncertainty in b around nominal value

Legendre-Uniform intrusive PC

Mean and standard deviation for u, v , and w

Uncertainty grows in time



Surface Reaction Model: ISP results

0 200 400 600 800 1000Time [-]

−0.1

0.0

0.1

0.2

0.3

0.4

0.5ui

[-]

u0

u1

u2

u3

u4

u5

Modes of u

Modes decay with higher order

Amplitudes of oscillations of higher order modes grow in time



Surface Reaction Model: ISP results: PDFs

0.310 0.315 0.320 0.325 0.330 0.335 0.340 0.345 0.350u

0

50

100

150

200

250

300

350

Prob. Dens. [-]

t = 330.5t = 756.5

0.10 0.15 0.20 0.25 0.30 0.35u

0

2

4

6

8

10

12

14

16

Prob. Dens. [-]

t = 377.8t = 803.0

Pdfs of u at maximum mean (left) and maximum standard deviation(right)

Distributions get broader and multimodal as time increases

Effect of accumulating phase errors



Surface Reaction Model: Non-Intrusive Propagation ofUncertainty

Consider two approaches for Non-Intrusive Spectral Projection(NISP):

QuadratureMonte Carlo or random-sampling

Assume PCE for uncertain parameter b and the output variables

Sample input parameters at quadratrure/random sample points

Run deterministic forward model for each of the sampled inputparameters

Perform Galerkin projection uk = 〈uΨk 〉

〈Ψ2k〉

Quadrature: 〈uΨk〉 =∑Nq

i=1 wiu(bi )

Monte Carlo: 〈uΨk〉 = 1Ns

∑Ns

i=1 u(bi )



Surface Reaction Model: NISP implementation in UQTk

Quadrature:

// Get the quadrature points

int nQdpts=myPCSet.GetNQuadPoints();

double* qdpts=new double[nQdpts];

myPCSet.GetQuadPoints(qdpts);

...

// Evaluate parameter at quad pts

for(int i=0;i<nQdpts;i++){

bval[i]=myPCSet.EvalPC(b,&qdpts[i]);

}

...

// Run model for all samples

for(int i=0;i<nQdpts;i++){

u_val[i] = ...

}

// Spectral projection

myPCSet.GalerkProjection(u_val,u);

myPCSet.GalerkProjection(v_val,v);

myPCSet.GalerkProjection(w_val,w);

Monte-Carlo Sampling:

// Get the sample points

int nSamples=1000;

Array2D<double> samPts(nSamples,dim);

myPCSet.DrawSampleVar(samPts);

...

// Evaluate parameter at sample pts

for(int i=0;i<nSamples;i++){

... // select samPt from samPts

bval[i]=myPCSet.EvalPC(b,&samPt)

}

...

// Run model for all samples

for(int i=0;i<nSamples;i++){

u_val[i] = ...

}

// Spectral projection

myPCSet.GalerkProjectionMC(samPts,u_val,u);

myPCSet.GalerkProjectionMC(samPts,v_val,v);

myPCSet.GalerkProjectionMC(samPts,w_val,w);



Surface Reaction Model: NISP results

0 200 400 600 800 1000Time [-]

0.0

0.2

0.4

0.6

0.8

1.0


NISP Quadrature

u v w

0 200 400 600 800 1000Time [-]

0.0

0.2

0.4

0.6

0.8

1.0


NISP MC

u v w

Mean and standard deviation for u, v , and w

Quadrature approach agrees well with ISP approach using 6quadrature points

Monte Carlo sampling approach converges slowly

With a 1000 samples, results are quite different from ISP and NISP



Surface Reaction Model: Comparison ISP and NISP

0 200 400 600 800 1000Time [-]

−0.010

−0.008

−0.006

−0.004

−0.002

0.000

0.002

0.004ui

[-]

u4,ISP

u4,NISP

u5,ISP

u5,NISP

Lower order modes agree perfectly

Very small differences in higher order modes

Difference increases with time



Surface Reaction Model: Comparison ISP and NISP

0.10 0.15 0.20 0.25 0.30 0.35u

0

2

4

6

8

10

Prob. Dens. [-]

ISP, t = 803.0NISP, t = 803.0

All pdf’s based on 50K samples each and evaluated with KernelDensity Estimation (KDE)

No difference in PDFs of sampled PCEs between NISP and ISP



Surface Reaction Model: Comparison ISP, NISP, and MC

0.10 0.15 0.20 0.25 0.30 0.35u

0

2

4

6

8

10

Prob. Dens. [-]

ISP, t = 803.0NISP, t = 803.0MC, t = 803.0

All pdf’s based on 50K samples each and evaluated with KernelDensity Estimation (KDE)

Good agreement between intrusive, non-intrusive projection, andMonte Carlo sampling



ISP pros and cons

Pros:

ElegantOne time solution of system of equations for the PC coefficients fullycharacterizes uncertainty in all variables at all timesTailored solvers can (potentially) take advantage of new hardwaredevelopments

Cons:

Often requires re-write of the original codeReformulated system is factor (P+1) larger than the original systemand can be challenging to solveChallenges with increasing time-horizon for ODEs

Many efforts in the community to automate ISP

UQToolkit http://www.sandia.gov/UQToolkit/Sundance http://www.math.ttu.edu/ klong/Sundance/html/Stokhos http://trilinos.sandia.gov/packages/stokhos/...



NISP pros and cons

Pros:

Easy to use as wrappers around existing codesEmbarassingly parallel

Cons:

Most methods suffer from curse of dimensionality Nq = nNd

Many develoment efforts for smarter sampling approaches anddimensionality reduction

(Adaptive) Sparse Quadrature approachesCompressive Sensing...

Sampling methods have found very wide spread use in the community

DAKOTA http://dakota.sandia.gov/...


Uncertainty Characterization

Outline







7 Advanced Topics



Uncertain Input Characterization

Use standard distribution

Normal distribution: often a good choice based on Central LimitTheoremLognormal: when positivity is required

Infer model parameters from data with inverse problem

Determine RV from available samples of RV

Fit standard distribution to data, e.g. MultiVariate Normal (MVN)approximationInverse Cumulative Distribution Function (CDF) mapping; Rosenblatttransformation

Dimensionality reduction for stochastic processes, Karhunen-LoeveExpansion (KLE)



Bayesian Inference

Bayes formulaPosterior

︷︸︸︷

P(c|D) ∝Likelihood

︷︸︸︷

P(D|c)Prior

︷︸︸︷

P(c)

Update prior distribution/knowledge about parameter c to posteriordistribution given data D, using likelihood function L(c) ≡ P(D|c).Data D = {di}Ni=1 - measurements of some quantities of interest (QoIs).

Prior distribution P(c) is based on expert opinion/previous literature.

Likelihood function measures goodness-of-fit and is the key component thatconnects the model inputs to measured QoIs, e.g.

L(c) = P(D|c) = 1√2πσ

exp

(

−N∑

i=1

(di − fi (c))2

2σ2

)

Input parameter → output QoI functions fi (·) could be expensive or noteven available.

Usually posterior distribution is not analytically tractable:need to resort to Markov Chain Monte Carlo sampling.



Markov Chain Monte Carlo

Single-site MCMC

• Set the current chain state c at an initial chain state c(0),

• Repeat for a predefined number (NMCMC ) of times,

• For k = 1, . . . ,K ,

• generate a single-site proposal c ′k from a Gaussiandistribution centered at the current chain statevalue of site ck with proposal width σk ,

• compute α = min {1,P(c′|D)/P(c|D)},• update the current chain state’s k-th element

ck = c ′k with probability α,

• End

• End



Markov Chain Monte Carlo

Adaptive MCMC [Haario,2002]

• Set the current chain state c at an initial chain state c(0),

• Repeat for a predefined number (NMCMC ) of times,

• generate a proposal c′ from a multivariate Gaussiandistribution centered at the current chain state value c withproposal covariance that is learnt from previous chain states,

• compute α = min {1,P(c′|D)/P(c|D)},• update the current chain state c = c

′ with probability α,

• End



Surface Reaction Model: parameter inference

500 600 700 800 900 1000

Time

0

0.1

0.2

0.3

0.4

Spec

ies

u

0.26 0.28 0.3 0.32 0.34Species u

0

0.02

0.04

0.06

0.08

0.1

Spec

ies

v

Synthetic data is generated for model outputs u,v at T = 1000.

Inferring two input parameters a and b using 10 samples on u and v

Likelihood function

P(D|a, b) = 1

2πσuσv

exp

(

−N∑

i=1

((d

(i)u − u(T ; a, b))2

2σ2u

+(d

(i)v − v(T ; a, b))2

2σ2v

)

)

Uncorrelated Gaussian noise model is assumed with standard deviationsproportional to the model value σu = 0.1u, σv = 0.1v .




500 600 700 800 900 1000

Time

0

0.01

0.02

0.03

0.04

0.05

Spec

ies

v

0.26 0.28 0.3 0.32 0.34Species u

0

0.02

0.04

0.06

0.08

0.1

Spec

ies

v

Synthetic data is generated for model outputs u,v at T = 1000.

Inferring two input parameters a and b using 10 samples on u and v

Likelihood function

P(D|a, b) = 1

2πσuσv

exp

(

−N∑

i=1

((d

(i)u − u(T ; a, b))2

2σ2u

+(d

(i)v − v(T ; a, b))2

2σ2v

)

)

Uncorrelated Gaussian noise model is assumed with standard deviationsproportional to the model value σu = 0.1u, σv = 0.1v .




10000 12000 14000 16000 18000 20000MCMC sample count

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Para

met

er a

10000 12000 14000 16000 18000 20000MCMC sample count

0

5

10

15

20

25

Para

met

er b

Posterior distribution based on 20000 adaptive MCMC samples.

First 4000 samples discarded

Shown only the second half of the chains



Posterior on inferred parameters

0.6 0.8 1 1.2 1.4 1.6Parameter a

5

10

15

20

Para

met

er b

Width of posterior indicates the amount of uncertainty in the inferredparameters

Uncertainty can be reduced by taking more data or by reducing noisein measurements



Surface Reaction Model: posterior to PC in 1d

20.73 20.74 20.75 20.76 20.77 20.78Parameter b

0

50

100

PDF

Posterior PDFPosterior PDF (PC)

The posterior describes random variable a with CDF F (·)CDF transformation F (a) = η maps random variable a touniform[0, 1] random variable η.η = Φ(ξ) maps uniform η to normal RV ξThe inverse CDF enables NISP projection

a =P∑

k=0

akΨk(ξ) ak ∝ 〈aΨk(ξ)〉 =∫

F−1(Φ(ξ))︸︷︷︸

a

Ψk(ξ)dξ



Surface Reaction Model: posterior to PC in multi-d

Rosenblatt transformation maps any (not necessarily independent) setof random variables (λ1, . . . , λn) to uniform i.i.d.’s {ηi}ni=1

(Rosenblatt, 1952).

η1 = F1(λ1)

η2 = F2|1(λ2|λ1)

...

ηn = Fn|n−1,...,1(λn|λn−1, . . . , λ1)

Rosenblatt transformation is a multi-D generalization of 1D CDFmapping.

Conditional CDFs are harder to evaluate in high dimensions



Projection of Rosenblatt transformed vars onto PCEs

0 0.2 0.4 0.6 0.8 1Parameter η1

0

0.2

0.4

0.6

0.8

1

Para

met

er

η 2

0.6 0.8 1 1.2 1.4 1.6Parameter a

5

10

15

20

Para

met

er b

NISP projection is enabled by inverse Rosenblatt transformation(a, b) = R

−1(ξ1, ξ2) ensures a well-defined quadrature integration

a =

P∑

k=0

akΨk(ξ) ak ∝

∫

R−1a (ξ)

︸︷︷︸

a

Ψk(ξ)dξ

b =

P∑

k=0

bkΨk(ξ) bk ∝

∫

R−1b (ξ)

︸︷︷︸

b

Ψk(ξ)dξ



Surface Reaction Model: predictive confidence

0 200 400 600 800 1000Time

0

0.1

0.2

0.3

0.4

Spec

ies

u

0 200 400 600 800 1000Time

0

0.2

0.4

0.6

0.8

Spec

ies

v

Uncertainty in inferred input parameters a and b is pushed throughthe forward model

Using 2D Wiener Hermite NISP Quadrature approach

Accounts for parametric uncertainty due to data noise



Multivariate Normal Approximation

Many distributions are unimodal and somewhat shaped like Gaussians

MultiVariate Normal (MVN) approximations capture the mean andcorrelation structure of the random variables

Easy to extract from a set of samples

In 1D: just compute mean and standard deviation: u = u0 + u1ξMulti-D: Cholesky factorization of covariance

# Compute mean parameter values

par_mean = numpy.mean(samples,axis=0)

# Compute the covariance

par_cov = numpy.cov(samples,rowvar=0)

# Compute the Cholesky Decomposition

chol_lower = numpy.linalg.cholesky(par_cov)



MVN approximation of Bayesian posterior from MCMCsamples

1.30 1.32 1.34 1.36 1.38 1.40

S15200

5250

5300

5350

5400

CS

0.000

0.075

0.150

0.225

0.300

0.375

0.450

0.525

0.600

1.31 1.32 1.33 1.34 1.35 1.36 1.37 1.38 1.39

S1

5240

5260

5280

5300

5320

5340

5360

5380

5400

CS

Comparison of Posterior (blue) with MVN (red)

S1 = 1.351 + 0.01367ξ1

CS = 5310− 26.25ξ1 + 20.26ξ2



Karhunen-Loeve (KL) Expansions

Assume stochastic process F (x , θ)

With covariance function Cov(x1, x2)

F can be written as

F (x , θ) = 〈F (x , θ)〉θ +∞∑

k=1

√

λkFk(x)ξk

Fk(x): eigenfunctions of Cov(x1, x2)

λk : corresponding eigenvalues, all positive

ξk : uncorrelated random variables, unit variance

Samples are obtained by projecting realizations of F onto Fk

Generally not independent

Special case: for Gaussian F , ξk are i.i.d. normal random variables



1D Gaussian Process: Realizations

δ = 0.02

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f(x)

δ = 0.1

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f(x)

Covariance Cov(x1, x2) = exp(−(x1 − x2)2/δ2)

Sample realizations are noisier as correlation length decreases



1D Gaussian Process: KL modes

δ = 0.02

0.0 0.2 0.4 0.6 0.8 1.0x

−3

−2

−1

0

1

2

3

f n

f1

f2

f3

f4

δ = 0.1

0.0 0.2 0.4 0.6 0.8 1.0x

−4

−3

−2

−1

0

1

2

3

4

5

f n

f1

f2

f3

f4

Eigenmodes of the covariance matrix

Data covariance matrix constructed from 512 Gaussian processrealizations

Higher modes are more oscillatory



1D Gaussian Process: KL random variables

δ = 0.02

−4 −3 −2 −1 0 1 2 3 4ξ(θ)

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

PDF(ξ)

ξ1

ξ2

ξ3

ξ4

δ = 0.1

−4 −3 −2 −1 0 1 2 3 4ξ(θ)

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

PDF(ξ)

ξ1

ξ2

ξ3

ξ4

Random variables obtained by projecting realizations onto KL modes

Uncorrelated by construction

Also independent due to nature of Gaussian Process



1D Gaussian Process: Eigenvalue spectrum

0 10 20 30 40Eigenvalue #

10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

Eigenvalue spectrum decays more slowly as correlation lengthdecreases

More oscillatory modes needed to represent fluctuations in x

KL expansion generally is truncated after enough modes are includedto capture a specified fraction of the total variance



1D Gaussian Process: Reconstructed realizations


10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

Large scale features can be resolved with small number of modes

Smaller scale features require higher modes





10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

2 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

4 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

6 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

8 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

10 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

14 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

16 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

18 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

1 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

2 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

3 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

4 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

5 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

6 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

7 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

8 terms







10-4

10-2

100

102

Eigenvalue M

agnitude

δ=0.02

δ=0.1

δ=0.2

δ=0.5

0.0 0.2 0.4 0.6 0.8 1.0x

−20

−10

0

10

20

f n

9 terms





KL of 2D Gaussian Process

δ = 0.1

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

δ = 0.2

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

δ = 0.5

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

2D Gaussian Process with covariance:Cov(x1, x2) = exp(−||x1 − x2||2/δ2)Realizations are smoother as covariance length δ increases



2D KL - Modes for δ = 0.1

√λ1f1

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ5f5

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ2f2

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ6f6

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ3f3

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ7f7

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ4f4

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ8f8

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

yDebusschere (Sandia, USC) UQTk examples SIAM UQ 2012 53 / 81



√λ1f1

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ5f5

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ2f2

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ6f6

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ3f3

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ7f7

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ4f4

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

√λ8f8

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0



2D KL - eigenvalue spectrum

0 20 40 60Eigenvalue #

10-2

100

102

104

Eigenvalue M

agnitude

δ=0.1

δ=0.2

δ=0.5

δ = 0.14 terms 16 terms

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

32 terms 64 terms

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y





10-2

100

102

104

Eigenvalue M

agnitude

δ=0.1

δ=0.2

δ=0.5


0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y

32 terms 64 terms

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0y

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

y


Case Study 1

Outline







7 Advanced Topics


Case Study 1

Uncertainty in Model Inputs

Probabilistic UQ requires specification of uncertain inputs

Require joint PDF on input space

PDF can be found given data

Typically such PDFs are not available from the literature

Summary information, e.g. nominals and bounds, is usually available

Uncertainty in computational predictions can depend strongly ondetailed structure of the missing parametric PDF

Need a procedure to reconstruct a PDF consistent with availableinformation in the absence of the raw data

“Data Free” Inference (DFI) (Berry et al., JCP 2012)


Case Study 1

The strong role of detailed input PDF structure

Simple nonlinear algebraic model (u, v) = (x2 − y2, 2xy)

Two input PDFs, p(x , y)

same nominals/boundsdifferent correlation structure

Drastically different output PDFs

different nominals and bounds


Case Study 1

Generate ignition “data” using a detailed model+noise

Ignition using a detailedchemical model formethane-air chemistry

Ignition time versus InitialTemperature

Multiplicative noise errormodel

11 data points:

di = tGRIig,i (1 + σǫi )

ǫ ∼ N(0, 1)

1000 1100 1200 1300Initial Temperature (K)

0.01

0.1

1

Igni

tion

time

(sec

)

GRI

GRI+noise


Case Study 1

Fitting with a simple chemical model

Fit a global single-stepirreversible chemical model

CH4 + 2O2 → CO2 + 2H2O

R = [CH4][O2]kf

kf = A exp(−E/RoT )

Infer 3-D parameter vector(lnA, lnE , lnσ)

Good mixing with adaptiveMCMC when start at MLE

28

30

32

34

36

lnA

10.6

10.8

lnE

0 2000 4000 6000 8000 10000Chain Step

-3-2.5

-2-1.5

-1-0.5

lnσ


Case Study 1

Bayesian Inference Posterior and Nominal Prediction

30 31 32 33 34 35

10.6

10.65

10.7

10.75

10.8

10.85

1000 1100 1200 1300Initial Temperature (K)

0.01

0.1

1

Igni

tion

time

(sec

)

GRIGRI+noiseFit Model

GRI

GRI+noise

Marginal joint posterior on(lnA, lnE ) exhibits strongcorrelation

Nominal fit model is consis-tent with the true model


Case Study 1

Correlation Slope χ and Chemical Ignition

0 0.5 1Time (sec)

0

0.05

0.1

0.15

0.2

0.25

Mas

s Fr

actio

n

Means

1000

1500

2000

2500

3000

Tem

pera

ture

(K

)CH

4

O2

CO2

H2O

T

0.46 0.462 0.464 0.466 0.468 0.47Time (sec)

0

0.01

0.02

0.03

0.04

0.05

0.06

Mas

s Fr

actio

n

CH4

O2

CO2

H2O

Standard Deviations

0

100

200

300

400

Tem

pera

ture

(K

)

T

4th Order Multiwavelet PC, Multiblock, Adaptive

σT ,max ∼ 400 K during ignition transient, χ ∼ 0.03


Case Study 1

Time evolution of Temperature PDFs in preheat stage

1300 1400 1500 1600Temperature (K)

0

0.05

0.1

0.15

Prob

abili

ty D

ensi

ty

MC

t=0.455 sec

0.459 sec

0.462 sec

0.464 sec

1300 1400 1500 1600Temperature (K)

0

0.05

0.1

0.15

Prob

abili

ty D

ensi

ty

MW

t=0.455 sec

0.459 sec

0.462 sec

0.464 sec

Similar results from MC (20K samples) and MW PC

Increased uncertainty, and long high–T PDF tails, in time


Case Study 1

Evolution of Temp. PDF – Fast Ignition Transient

0

0.01

0.02

MC

0

0.01

0.02

0

0.01

0.02

0

0.01

0.02

1000 1500 2000 2500 3000Temperature (K)

0

0.01

0.02

Den

sity

t = 0.4642 sec

0.4660 sec

0.4664 sec

0.4668 sec

0.4671 sec

0

0.01

0.02

MW

0

0.01

0.02

0

0.01

0.02

0

0.01

0.02

1000 1500 2000 2500 3000Temperature (K)

0

0.01

0.02D

ensi

ty

t = 0.4642 sec

0.4660 sec

0.4664 sec

0.4668 sec

0.4671 sec

Transition from unimodal to bimodal PDFs

Leakage of probability mass from pre-heat PDF high–T tail


Case Study 1

Time evolution of Temperature PDFs for different χ

1500 2000 2500 3000Temperature (K)

0

0.001

0.002

0.003

0.004

0.005

0.006

Prob

abili

ty D

ensi

ty

0.022875

1500 2000 2500 3000Temperature (K)

0

0.002

0.004

0.006

0.008

0.01

Prob

abili

ty D

ensi

ty

0.036675

1500 2000 2500 3000Temperature (K)

0

0.005

0.01

0.015

0.02

0.025

0.03

Prob

abili

ty D

ensi

ty

0.042425

1500 2000 2500 3000Temperature (K)

0

0.01

0.02

0.03

0.04

0.05

0.06

Prob

abili

ty D

ensi

ty

0.04325

1500 2000 2500 3000Temperature (K)

0

0.05

0.1

0.15

0.2

0.25

Prob

abili

ty D

ensi

ty

0.04395

1500 2000 2500 3000Temperature (K)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

Prob

abili

ty D

ensi

ty

0.04475

1500 2000 2500 3000Temperature (K)

0

0.002

0.004

0.006

0.008

0.01

Prob

abili

ty D

ensi

ty

0.052475

1500 2000 2500 3000Temperature (K)

0

0.001

0.002

0.003

0.004

0.005

0.006

Prob

abili

ty D

ensi

ty

0.066275

Bimodal solution PDFs for high uncertainty growth

Unimodal for low uncertainty growth, with χ ≈ 0.044


Case Study 1

Central Challenge for UQ in Chemical Kinetic Models

Need joint PDF on model parameters for forward UQ

Joint PDF structure is crucial

Joint PDF not available for chemical kinetic parameters

At best, have

Nominal parameter valuesBounds, e.g. marginal 5%, 95% quantiles

PDF can be constructed by repeating experimentsor access to original raw data

– Neither is feasible

Is there a way to construct an approximate PDF without access toraw data?

– Yes!


Case Study 1

Data Free Inference (DFI) (Berry et al., JCP, in review)

Intuition: In the absence of data, the structure of the fit model,combined with the nominals and bounds, implicitly inform thecorrelation between the parameters

Goal: Make this information explicit in the joint PDF

DFI: discover a consensus joint PDF on the parameters consistentwith given information:

– Nominal parameter values– Bounds– The fit model– The data range– ... potentially other/different constraints


Case Study 2

Outline







7 Advanced Topics


Case Study 2

Schlogl Model is a prototype bistable model

• ReactionsA+ 2X

a1−→

←−

a2

3X

Ba3−→

←−

a4

X .

• Propensitiesa1 = k1AX (X − 1)/2,a2 = k2X (X − 1)(X − 2)/6,a3 = k3B ,a4 = k4X .

• Nominal parametersk1A 0.03k2 0.0001k3B = λ 200k4 3.5A 105

B 2 · 105X (0) 250

0 5 10 15 200

100

200

300

400

X(t

)

0 5 10 15 200

200

400

600

800

X(t

)0 5 10 15 20

200

300

400

500

600

700

800

X(t

)

λ = 130

λ = 200

λ = 350

Time, t


Case Study 2

Polynomial Chaos expansion represents any randomvariable as a polynomial of a standard random variable

• Truncated PCE: finite dimension n and order p

X (θ) ≃P∑

k=0

ckΨk(η)

with the number of terms P + 1 = (n+p)!n!p! .

• η = (η1, · · · , ηn) standard i.i.d. r.v.Ψk standard orthogonal polynomialsck spectral modes.

• Most common standard Polynomial-Variable pairs:(continuous) Gauss-Hermite, Legendre-Uniform,(discrete) Poisson-Charlier.

[Wiener, 1938; Ghanem & Spanos, 1991; Xiu & Karniadakis, 2002]


Case Study 2

Galerkin Projection is typically needed

PC expansion: X (θ) ≃∑Pk=0 ckΨk(η) = gD(η)

Orthogonal projection: ck =〈X (θ)Ψk (η)〉

〈Ψ2k(η)〉

• Intrusive Spectral Projection (ISP)

⋆ Direct projection of governing equations⋆ Leads to deterministic equations for PC coefficients∗ No explicit governing equation for SRNs

• Non-intrusive Spectral Projection (NISP)

⋆ Sampling based⋆ No explicit evolution equation for X needed∗ Galerkin projection not well-defined for SRNs


Case Study 2

Karhunen-Loeve decomposition reduces stochastic processto a finite number of random variables

• KL decomposition:

X (t, θ) = X (t) +∞∑

n=1

ξn(θ)√

λnfn(t)

• Uncorrelated, zero-mean KL variables:

〈ξn〉 = 0, 〈ξnξm〉 = δnm

• SSA(continuum) ←→ KL(discrete)

X (t)←→ ξ = (ξ1, ξ2, . . . )

0 2 4 6 8 10 12 14 16 18 20

Time, t-300

-200

-100

0

100

KL

mod

es,

λi1/

2 fi(t

)

0 1 2 3 4 5 6 7 8 9n

1000

10000

1e+05

1e+06

Eig

enva

lues

, λ n

-2 -1 0 1 2

ξ1

-4

-2

0

2

ξ 2


Case Study 2

K-L decomposition captures each realization


Case Study 2

K-L decomposition captures each realization

0 2 4 6 8 10 12 14 16 18 20Time, t

0

100

200

300

400

500

600

700

X(t

) L=10L=30L=60X(t)


Case Study 2

PC expansion of a random vector

ξ =P∑

k=0

ckΨk(η)

Galerkin projection

ck =〈ξΨk(η)〉〈Ψ2

k(η)〉

is not well-defined,since ξ and η do not belong to the same stochastic space.


Case Study 2

PC expansion of a random vector

ξ =P∑

k=0

ckΨk(η)

Galerkin projection

ck =〈ξΨk(η)〉〈Ψ2

k(η)〉

is not well-defined,since ξ and η do not belong to the same stochastic space.

Need a map ξ ↔ η.


Case Study 2

Rosenblatt transformation

• Rosenblatt transformation maps any (not necessarilyindependent) set of random variables (ξ1, . . . , ξn) to uniformi.i.d.’s {ηi}ni=1 (Rosenblatt, 1952).

η1 = F1(ξ1)

η2 = F2|1(ξ2|ξ1)η3 = F3|2,1(ξ3|ξ2, ξ1)...

ηn = Fn|n−1,...,1(ξn|ξn−1, . . . , ξ1)

• Inverse Rosenblatt transformation ξ = R−1(η) ensures awell-defined quadrature integration

〈ξiΨk(η)〉 =∫

R−1(η)iΨk(η)dη


Case Study 2

KL+PC+Data Partitioning represent the dynamics of a bimodal process

0 5 10 15 20

Time, t0

100

200

300

400

500

600

700X

KL

PC(t

)

KL-PC representation, 5 KL modes, 3rd PC order


Advanced topics

Outline







7 Advanced Topics


Advanced topics

Advanced Topics

Sensitivity analysis

Domain decomposition methods; multiwavelets

Adaptive Sparse Quadrature

Stochastic preconditioning (time rescaling)

Data Free Inference (DFI)

Bayesian Compressive Sensing (BCS)

PCEs with random coefficients versus Gaussian Processes

Model uncertainty, comparison, selection

... Stay Tuned ...


Uncertainty Quantification: UQTk example problems...2012/04/04 · Uncertainty Quantiﬁcation:...

Documents

Transcript of Uncertainty Quantification: UQTk example problems...2012/04/04 · Uncertainty Quantiﬁcation:...