GOal-Oriented Optimal Experimental Design · pr /ˇ like(bjm); where the data likelihood is given...

GOal-Oriented Optimal Experimental DesignGO-OED for PDE-based Bayesian Linear Inverse Problems

Ahmed Attia

Statistical and Applied Mathematical Science Institute (SAMSI)19 TW Alexander Dr, Durham, NC 27703

attia@ samsi.info || vt.edu

Alen Alexanderian, and Arvind Krishna Saibaba

Department of MathematicsNorth Carolina State University

SAMSIJanuary 26, 2017

GO-OED for PDE-based Bayesian Linear Inverse Problems [1/28]

January 26, 2017: Goal-Oriented Optimal Experimental Design, Ahmed Attia. (http://samsi.info)

Outline

MotivationInverse problemsData collection, and sensor placement

Optimal Experimental Design (OED)Optimal experimental design (OED)Sensor placement as OED problem

Goal-Oriented OED

GO-OED preliminary results

Conclusions and Future Work

GO-OED for PDE-based Bayesian Linear Inverse Problems [2/28]




Goal-Oriented OED



GO-OED for PDE-based Bayesian Linear Inverse Problems Motivation [3/28]


Motivation

I Inverse problems, and data assimilation, enable making efficient and accuratepredictions about the state of large-scale systems such as the atmosphere.

I The solution of the inverse problem is sensitive to the quality of the collectedmeasurements.

I It is intuitive that we should collect as much data as possible, howevermeasurement sensors are not always cheap!

I Generally, observational grids are designed such that the collectedmeasurements lead to predictions with minimum uncertainty (OED).

I We might be interested in designing an observational grid that minimizes aQoI dependent on the model state, rather than the state itself (GO-OED).



Example: Advection-Diffusive transport

I Consider the concentration of a contaminant u in the domain Ω ∈ R2.

I Assume, we are interested in inferring the initial distribution of thecontaminant, from measurements b taken after the contaminant has beensubjected to diffusive transport. For example, consider measuring u on someparts Γm of (or all) the domain boundary.



Example: Initial Condition in Advection-Diffusive transport

I Inverse Problem: given measurements b over time interval [T1,T ], find theinitial condition u0 such that the evolution of the contaminant distribution isgoverned by the PDE, and is consistent with the collected measurements.

I A Continuous Formulation: solve the optimization problem:

minu0

J(u0) :=1

2

∫ T

T1

∫Γm

(B(u)− b)2dxdt +γ

2

∫Ω

u20dx

where u is constrained by the advection diffusion PDEs.

I A Discrete Formulation: solve the optimization problem:

minu0

J(u0) :=1

2

m∑i=1

(B(ui )− bi )TΓ−1

noise(B(ui )− bi ) +1

2uT

0 R−1u0

where u is a spatial discretization of u, and is constrained by the discretizedadvection diffusion PDEs, and bii=1,2,...,m are discrete-time observations.



Example: Initial Condition in Advection-Diffusive transportA Discrete Bayesian Formulation:

I Let u0 be a random vector with prior distribution N (0, Cprior).

I Assume a likelihood function:

P(b1, . . . ,bm|u0) ∝ exp

(−1

2

m∑i=1

(bi − B(ui ))TΓ−1noise(bi − B(ui ))

)

I The posterior (Bayes’ theorem):

P(u0|b1, . . . ,bm) ∝ exp(− 1

2

∑mi=1(B(ui )− bi )

TΓ−1noise(bi − B(ui ))

)+ 1

2 uT0 C−1

prioru0

where Γnoise is an observation error covariance matrix, and Cprior is aparameter prior covariance matrix.

I Solution strategy: Given observation(s), and a prior, describe the posteriorPDF, and use it to make predictions.



Data collection, and sensor placement

I Design of experiments (DOE) involves applying engineering principles andtechniques for data collection so as to ensure the conclusions made based onthe experiment are valid and reliable.

I Suppose we have Nobs candidate observation locations. Where do we putsensors, to actually collect measurements?

I The main idea: Find the measurement locations such that the uncertaintyin the QoI is minimized.



Problem Formulation:I Consider an additive noise model:

b = F(m) + η .

where F is the “Forward Operator” e.g. (B S).

I In a Bayesian formulation, we obtain the posterior law for m through,

dµbpost

dµpr∝ πlike(b|m) ,

where the data likelihood is given by, πlike(b|m) = ρnoise(d −F(m)).

I An experimental design, ξ, will specify the way data is collected (e.g. sensorplacement). In OED problem, we seek ξopt that minimizes the uncertainty inthe inferred parameter m, i.e. we consider:

dµb|ξpost

dµpr∝ πlike(b|m; ξ) .

GO-OED for PDE-based Bayesian Linear Inverse Problems Optimal Experimental Design (OED) [9/28]


Optimal Experimental Design (OED)

I The way one chooses to quantify “posterior uncertainty” leads to the choiceof the design criterion.

I Alphabetical criteria:

1. minimize the trace of the posterior covariance (A-Optimality)

2. minimize the determinant of the posterior covariance (D-Optimality)

3. minimize the largest eigenvalue of the posterior covariance (E-Optimality)

4. etc.



Optimal Experimental Design (OED)

I In the context of optimal sensor placement, the design ξ is a binary vector w ,corresponding to candidate sensor locations.

I We solve a relaxed version, where the design ξ is a vector w of weights∈ [0, 1] corresponding to a set of candidate locations for sensors.



OED for Sensor PlacementI w enters the Bayesian inverse problem through the data likelihood,

amounting to a weighted data likelihood:

πlike(b|m; w) ∝ exp

ß−1

2

(F(m)− b

)TW1/2Γ−1

noiseW1/2(F(m)− b

)™,

where W = diag(w1, . . . ,wNs ).

I Posterior (Gaussian Linear case) N (0, Cpost) with

Cpost(w) = (F∗WσF + C−1prior)

−1 ≡ H−1 ,

where Wσ = W1/2Γ−1noiseW1/2.

I OED: find w that minimizes (Cpost trace, determinant, etc.)

I what if we are interested in a prediction quantity p = P(m) rather than theparameter itself? e.g. the average contaminant concentration within aspecific distance from the buildings’ walls. (GO-OED)





Goal-Oriented OED



GO-OED for PDE-based Bayesian Linear Inverse Problems Goal-Oriented OED [13/28]


GO-OED: problem formulationI We consider a linear Θpred of the form:

Θpred(m) = Pm ,

where P is a linear prediction operator, e.g. a forward solve followed by arestriction operator.

I In the standard Linear-Gaussian settings:

νprior = NÄ‹mpr, Cpr

ä, ν

y |wpost = N

Ä‹mpost, Cpost

ä,

with ‹mpr = Pmpr , ‹mpost = PmMAP , and,

Cpr = PCprP∗ , Cpost = PCpostP∗ ,where, the posterior MAP, and the posterior covariance, of the inverseproblem, are given by:

mMAP = Cpost

(F∗Wσy + C−1

pr mpr

);

Cpost(w) = (F∗WσF + C−1pr )−1 = (Hmisfit(w) + C−1

pr )−1 .



GO-OED: A-Optimality

I The optimal design (wAoptimal) is given by:

wAoptimal = arg min

w∈RNs

trÄ

Cpost

ä:= tr

ÄP [H(w)]−1 P∗

ä,

where H(w) is the Hessian of the posterior negative-log.

I With temporally-independent observation errors, the Hessian reads:

H(w) =Nτ∑k=1

F∗0,kWσF0,k + C−1pr = Hmisfit(w) + C−1

pr ,

where F0,k is the parameter-to-observable map from (discrete) time 0 to timetk , and F∗0,k is it’s adjoint.



GO-OED: A-Optimality

I The gradient of A-optimality objective, with respect to the design, is given by:

∇w trÄ

Cpost

ä= −

Nτ∑k=1

∑j

ÄΓ−1/2N F0,k [H(w)]−1 P∗ e j

äÄ

Γ−1/2N F0,k [H(w)]−1 P∗ e j

ä ,where e i is the i th coordinate vector in RNpred .



GO-OED: D-Optimality

I The optimal design (wDoptimal) is given by:

wDoptimal = arg min

w∈RNs

log detÄ

Cpost

ä:= log det

ÄP [H(w)]−1 P∗

ä.

I The gradient of D-optimality objective reads:

∇w log detÄ

Cpost

ä= −

Nτ∑k=1

∑j

ÄΓ−1/2N F0,k [H(w)]−1 P∗Σ−1/2

pred (w)e j

äÄ

Γ−1/2N F0,k [H(w)]−1 P∗Σ−1/2

pred (w)e j

ä ,where e i is the i th coordinate vector in RNpred , andÄ

Cpost

ä−1= (Σpred(w))−1/2

ÄΣT

pred(w)ä−T/2



GO-OED: D-Optimality

I Alternatively, the gradient of D-optimality objective can be written on theform:

∂

∂wi

Älog det

ÄCpost

ää= −

Nτ∑k=1

(l k,iÄP [H(w)]−1 P∗

ä−1lTk,i),

where:

lTk,i =ÄP [H(w)]−1 F∗0,kΓ

−T/2N

äe i ; ∀i = 1, 2, . . . ,Nobs ,

with e i being the i th coordinate vector in RNobs .



GO-OED: Bayesian D-Optimality (KL-Divergence)I Consider the expected Kullback-Leibler divergence from the prior and

posterior predictive distributions of Θpred:

DKL(νy |wpost‖νprior) ,

I The expected information gain can be defined as:

Ψ(w) :=

∫ ∫DKL(ν

y |wpost‖νprior)πlike(y |m)dy µpr(dm)

= EmEy |mDKL(νy |wpost‖νprior)

where the KL divergence measure given by:

DKL(νy |wpost‖νprior) =

1

2

ÄtrÄ

C−1pr Cpost

ää+

1

2(µpost − µpr)

T C−1pr (µpost − µpr)

− Npred

2+

1

2log

Çdet Cpr

det Cpost

å.



GO-OED: Bayesian D-Optimality (KL-Divergence)

I The expected information gain reduces to:

Ψ(w) =1

2tr(ÄPC1/2

pr

ä† ÄPC1/2

pr

ä)− Npred

2

+1

2log det

(PCprPT

)− 1

2log det

(PCpostPT

).

I Maximizing the expected information gain is equivalent to minimizing thefollowing quantity: ‹Ψ(w) = log det

(PCpostPT

).

This is exactly the standard D-optimality criterion we addressed before.





Goal-Oriented OED



GO-OED for PDE-based Bayesian Linear Inverse Problems GO-OED preliminary results [21/28]


Example: Advection-Diffusive transportI Let u be the solution of:

ut − κ∆u + v · ∇u = 0 in Ω× [0,T ]

u(0, x) = u0 in Ω

κ∇u · n = 0 on ∂Ω× [0,T ]

where κ is the diffusivity, and v is the velocity field:

- Here, Dof = 7836,- T0 = 0, T1 = 1, Tfinal = 3, dt = 0.2,- Nobs = 20 observation points are randomly selected in the domain,- the observation noise level is 0.05 of the magnitude of the initial true solution.



Preliminary results: the AD problem

The solution of the inverse problem (using hIPPYlib):

(c) True m (d) A prior sample (e) The MAP

Figure: Inverse problem components and solution (MAP)fig:IP_solution



GO-OED Results: Advection-Diffusion modelLet P be a solution operator from the initial time T0 = 0 to a future time T = 4,followed by a restriction operator that observes the concentration of thecontaminant within a distance of ε = 0.009 around the two buildings.

Assuming an `1 norm with penalty α = 10−3:

(a) A-Optimal Design (b) D-Optimal Design

Figure: Goal-Oriented OED A, and D-Optimality results



GO-OED Results: Advection-Diffusion model

(a) A-Optimal Design (b) D-Optimal Design

Figure: Goal-Oriented OED A, and D-Optimality results



GO-OED Results

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20Candidate locations

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4w

A-Optimal

D-Optimal

Figure: Goal-Oriented OED A, and D-Optimality resultsfig:IP_solution




I We have extended the standard OED framework to the case where the QoI isa linear transformation of the inverse problem solution rather than thesolution itself.

I Developed the A-optimality, the D-optimality criteria and the associatedgradients,

I Showed the equivalence between standard D-optimality, and BayesianD-optimality (expected information gain).

I We are currently comparing results to the standard approach, i.e. with P = I,to understand the impact of incorporating Goals, on the optimal design.

I We will consider other regularization norms in addition to the `1 norm.

GO-OED for PDE-based Bayesian Linear Inverse Problems Conclusions and Future Work [27/28]


References

I Alexanderian, Alen, et al. ”A-Optimal Design of Experiments forInfinite-Dimensional Bayesian Linear Inverse Problems with Regularized`0-Sparsification.” SIAM Journal on Scientific Computing 36.5 (2014):A2122-A2148.

I Chaloner, Kathryn, and Isabella Verdinelli. ”Bayesian experimental design: Areview.” Statistical Science (1995): 273-304.

I Haber, Eldad, Lior Horesh, and Luis Tenorio. ”Numerical methods forexperimental design of large-scale linear ill-posed inverse problems.” InverseProblems 24.5 (2008): 055012.

I Villa, Umberto, Petra, Noemi, and Ghattas, Omar. ”hIPPYlib: An ExtensibleSoftware Framework for Large-Scale Deterministic and Linearized BayesianInversion” (2016): url = http://hippylib.github.io

Thank You!GO-OED for PDE-based Bayesian Linear Inverse Problems Conclusions and Future Work [28/28]


GOal-Oriented Optimal Experimental Design · pr /ˇ like(bjm); where the data likelihood is given...

Documents

Transcript of GOal-Oriented Optimal Experimental Design · pr /ˇ like(bjm); where the data likelihood is given...