Iterative surrogate model development...Iterativesurrogate model development with applications to...

HIPAD LAB: HIGH PERFORMANCE SYSTEMS LABORATORY DEPARTMENT OF CIVIL AND ENVIRONMENTAL ENGINEERING AND EARTH SCIENCES

Iterative surrogatemodeldevelopmentwithapplications

tomulti‐objectivedesignunderuncertainty[andstochasticsampling]

Alexandros TaflanidisAssociate Professor

and Frank M. Freimann Collegiate Chair in Structural Engineering Department of Civil & Environmental Engineering & Earth SciencesConcurrent Associate ProfessorDepartment of Aerospace and Mechanical Engineering

1 UQ tasks: design under uncertainty, stochastic sampling

• Metamodels (surrogate models): simple, data-driven approximations of the input/output relationship of complex numerical models

• Iterative, adaptive development of metamodels to support specific UQ1 tasks

• Goal is not to establish a globally accurate characterization of the input/output relationship, rather to accurately “perform” the UQ task

Seminar overview

Outline

• Motivation (why use metamodels?)

• Surrogate modeling overview

• Design under uncertainty using metamodels

• Iterative metamodel implementation for multi-objective design under uncertainty

• Iterative metamodel implementation for stochastic sampling

xnX x

ModelResponse

Performance evaluation

[ | , ] ( , )h h z x θ x θ

nΘ θ

θ1θ2

p(θ)

Design under uncertainty

Performance measure

( , ) znz x θ

Design problem under uncertainty

[ | , ] ( ) ( , ) (( )= [ ] )Θ Θp h p d h pH E h d z x θ θ θ x θ θ θx

arg min ( ) *

XH

xx x

Design under uncertainty [multi-objective]Design problem under uncertainty

( )= [ ( , )] ( , ) ( ) [ | , ] ( )i ip iΘi Θh pE d pH dh h x θ θ θ z x θx θ θx θ

arg min { ( )} P iX

H

x

X x



arg min { ( )} P iX

H

x

X x

Pareto Front

HP=H(XP) Feasible objective space

11 ( , ))= (( )Θ

h p dH x θ θ θx

22 ( , ))= (( )Θ

h p dH x θ θ θx



arg min { ( )} P iX

H

x

X x

H1(x)

H2(

x)

Model

nX xx

nΘ θθ~ ( )pθ θ

( , )z x θ[ | ],ih z x θPerformance

evaluation

Simulation-based optimization I

zn ( , )ih x θ

arg min { ( |{ })} jP i

XH

xX x θ

1

1 ( )( |{ }) ( , ) ; ~ ( ) ( )

jNj j j

i i jj

pH h qN q

θx θ x θ θ θθ

( , )( ) ( )ii Θh p dH x θ θ θx

arg min { ( )} P iX

H

x

X x

Simulation-based optimization II

Challenges Estimation error (accuracy of stochastic simulation)

needs to be addressed

Computational cost for a single evaluation significant since we need N model evaluations for each objective function calculation

Numerical differentiation might be only possibility for getting derivative information since we have assumed a black-box numerical model

Reduction of relative (common random

numbers) or absolute importance of error

(importance sampling)

Parallel computing

Algorithms for costly global optimization oralternative gradient

free approaches

1

1 ( ) ( |{ }) ( , ) ; ~ ( ) ( )

jNj j j

i jj

pH h qN q

θx θ x θ θ θθ

arg min { ( |{ })} jP i

XH

xX x θ

Outline






-2

0

2

-2

0

20

0.1

0.2

0.3

0.4

Real model

Experiments Surrogate model approximationy1y2

z y1y2

z

y1y2

z

• Data-driven mathematical approximations of the input/output (y/z) relationship of complex numerical models (frequently references as process or computer code)

• Formulated based on a database of simulations for the complex process. This database is frequently referenced as experiments or training (or support) points

Surrogate modeling I

Gaussian process metamodel (GPM) I

( ) ( ) ( )Tz n y b y β yReal function is approximated as a realization of a stochastic process

(Gaussian metamodel or Gaussian process emulator)

* *

* *

( ) ( ) + ( )

p

T T

n n

z

y b y β r y α

β α

( ) ~ ( ( ), ( ))z N z y y y

* *

( ) [ ( ) ( ) ( ) ( ) ( )]

where ( ) ( ) ( )

( ) ( ) /

2 2 T T 1 1 T

T 1

2 T 1

1

n

y u y B R B u y r y Rr y

u y B R r y b y

F Bβ R F Bβ

Provides also local estimate for the predictive variance (“estimation error”)

10

20

30Exact function

Experiments

z(y)

Gaussian process metamodel (GPM) II

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1y

10

20

30Exact function

Predictive mean

Experiments

z(y)


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1y

1

1( , | ) exp

nyysn I J

I J k ki

k k

Rs

y yy y s

Optimize hyper parameters of

correlation function

10

20

30Exact function

Predictive mean

Experiments

z(y)


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

2

4

6

σ(y)

y

Predictive variance

Predictive mean ± σ

( ) ~ ( ( ), ( ))z N z y y y

10

20

30Exact function

Experiments

z(y)

Design of experiments

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1y

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

2

4

6

σ(y)

y

Predictive variance

Predictive mean

0

10

20

30Response

Predictive meanExperimentz(y)

Adaptive design of experiments

y0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

2

4

6

σ(y)

y

Select experiment in regions of low accuracy

2

2

1 ( )

( ) ([ |

)]z

GPMi

j

ni

jhjj z

hzz

y

yy y

Target only domain of interest based on some

preference function (density)0

0.2

0.4

0.6

Preference function

πp(y)

Utility metric U(y)

0

10

20

30Response

Predictive meanExperimentz(y)

Sample-based design of experiments I

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

2

4

6

σ(y)

y

2

2

1 ( )

( ) ([ |

)]z

GPMi

j

ni

jhjj z

hzz

y

yy y

0

0.2

0.4

0.6

Preference function

πp(y)

• Simulate large number of samples from preference function

• Maintain only the samples that have larger associated error function

• Cluster them to avoid close proximity of experiments

Utility metric

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

2

4

60

σ(y)

y

0

0.2

0.4

0.6

DOE

0

10

20

30

Experiment (new)z(y)

previous errorcurrent error

Response

Predictive mean

Experiment

• Simulate large number of samples from utility function

• Maintain only the samples that have larger associated error function

• Cluster them to avoid close proximity of experiments

Preference function

πp(y)

Sample-based design of experiments II

-2

0

2

-2

0

20

0.1

0.2

0.3

0.4

Real model

Surrogate model approximationy1y2

z y1y2

z

y1y2

z

• Computationally VERY efficient (matrix manipulations that are easy to vectorize, no matrix inversion in implementation)

• Exact interpolation and accurate for approximating complex functions and can provide easily gradient information

• Local estimate for the predictive variance (can be used for adaptive DoE)

GPM

* *( ) ( ) + ( )T Tz y b y β r y α

* *( ) ( )+ ( )b rz y β J y α J y

2 2 1 1

1 1

( ) [1 ( ) ( ) ( )

( ) ( )]; ( ) ( ) ( )

T T

T T

y u y F R F u y

r y R r y u y F R r y b y

( ) ~( ( ), ( ))

zN z

yy y

Experiments (Support points)

Surrogate modeling II (GPM)

Outline






Model

nX xx

nΘ θθ~ ( )pθ θ

( , )z x θ[ | ],ih z x θPerformance

evaluation

Surrogate model–aided optimization I

zn ( , )ih x θ

arg min { ( |{ })} jP i

XH

xX x θ

1

1 ( )( |{ }) ( , ) ; ~ ( ) ( )

jNj j j

i i jj

pH h qN q

θx θ x θ θ θθ

What space to formulate the metamodel in (input)?

What is output?

[ | ],ih z x θModel

nX xx

nΘ θθ~ ( )pθ θ

( , )z x θPerformance

evaluation

zn( , )ih x θ

[ ]y x θAugmented input space:Taflanidis, A.A. and J.L. Beck (2008).

“Stochastic Subset Optimization for problems with reliability objectives”. Probabilistic Engineering Mechanics, 23 (2-3): 324-338.

Zhang, J., Taflanidis A.A, and J.C. Medina (2017). “Sequential approximate optimization for design under uncertainty problems utilizing Kriging metamodeling in augmented input space”. Computer Methods in Applied Mechanics and Engineering. 31: 369-395

Surrogate model–aided optimization II

[ | ],ih z x θ

[ ]y x θAugmented input space:

GPMΘX

y

Model

nX xx

nΘ θθ~ ( )pθ θ

( , )z x θPerformance

evaluation

zn( , )ih x θ

( ( ), ( ))N z y σ y

( , )ih x θ Performance

evaluation

[ ( ), ( ) | , ]GPMih zz y σ y x θ

Surrogate model–aided optimization III

z(y1)z(y2)

z(y n)

...

Model

Model

Modely1

y2

y n

...

[ ]y x θ

Response

Metamodeling in augmented input space I

Get experiments covering variation in both x and θ

training points

GPMDevelop metamodel in

augmented space and use it to replace initial objective function with metamodel-

based approximation

Solve optimization using metamodel-based

approximation, exploit also the gradient information

( ) ( , ) ( )

( ) ( ) ( , )

GPMi iΘ

GPMi iΘ

H h p d

H p h d

x x

x x θ θ θ

x θ x θ θ

θ 1

θ N

...

( )j kqθ ~ θ

Samples

GPM

...

GPM

GMP

...

( , )k Nih x θ

1( , )kz x θ

( , )k Nz x θ

1( , )kih x θ

1

1

1 ( )( , )( )

1 ( ) ( , )(

( )

))

(

jNk j

i k jj

jNk j

ik j

GPM ki

GPMi

j

k

H

H

phN q

p hN q

x x

θx θθ

θ x θθ

x

x

Use single surrogate model to

simultaneouslycalculate probabilistic integrals (uncertainty

propagation) and explore different design

solutions (design optimization)

Metamodeling in augmented input space II

1

1 ( ) { } ( , )( |)

)(

jNj k j

iGPM

k jij

H phN q

θθ x θθ

x

arg min { ( |{ })} jP i

X

GPMH

x

X x θ

Multi-objective optimizer I

H1(x)H

2(x)x2

x1

H1

H2( )= [ ( , ) ( ) )] , (i p

GPMiΘiH h p dE h x θx θ θx θ

arg min{ ( )}P iX

GPMH

x

X x

( )= [ ( , ) ( ) )] , (i pGPM

iΘiH h p dE h x θx θ θx θ

1

2

arg min ( )

such that ( ) ; 1,...,

* GPM

M rPX

Gp

H

H r n

x

x x

x

H1(x)H

2(x)x2

x1

H1

H2

Multi-objective optimizer II

Epsilon-constraint

1

2

arg min ( )

such that ( ) ; 1,...,

* GPM

M rPX

Gp

H

H r n

x

x x

x

H1(x)H

2(x)x2

x1

{XP}

Multi-objective optimizer III

( )= [ ( , ) ( ) )] , (i pGPM

iΘiH h p dE h x θx θ θx θ

Outline






1st iteration (k= 1)Metamodel Accuracy

(Initial) training points

z(y1)

z(y n)

...Modely1

y n...

Response

Model evaluationGPM development

H1GPM

H2G

PM PredictivePF

H1

H2 Validation

vs.Actual PF



z(y1)

z(y n)

...Modely1

y n...

Response


H1GPM

H2G

PM PredictivePF

Should we built first a metamodel with very high accuracy and then

perform optimization?

Can we efficiently converge to the correct solution with an inaccurate

metamodel?



z(y1)

z(y n)

...Modely1

y n...

Response


H1GPM

H2G

PM PredictivePFIs it possible to progressively update the

metamodel, in an informative manner, such that we can more efficiently approach

the Pareto front?

1st iteration (k= 1)Iterative implementation

Modely1

y n...

Response

Stopping criteriaSatisfied?

Termination

Improve the metamodel Y

N

z(y1)

z(y n)

...GPM development

H1GPM

H2G

PM PredictivePF

Newtraining points

New+ existing model evaluations

GPM update

Stopping criteria I

Iteration 2Iteration 1

H1(x)

H2(

x)

Iteration-wise improvement: we monitor the progress by comparing

current Pareto front and theprecedent one

Significant discrepancy:Pareto front is still evolving, so it justifies the need for the next

iteration

Iteration 3

Stopping criteria I


H1(x)

H2(

x)

Iteration-wise improvement: we monitor the progress by comparing

current Pareto front and theprecedent one

Minor discrepancy:Pareto front convergence is reached, so it’s not worthy to invest in

additional iterations

Iteration 3


H1(x)

H2(

x)

Pairwise comparison: The discrepancy is quantified by the

closeness of points in current Pareto set to ones in the precedent one

At converged iterations, for each current solution, there should exist at least one precedent

solution close to it

Remaining task: definition of closeness

Stopping criteria II

( ) ( ) sgn ( ) sgn ( ) ( ) ( ) 0 ( ) ( )m krig cr krig cr cr crl l l ll l lp P G G G G p G dG x x x x x x x

' ( ) ( ')

i

i iM H H

x x

x x

For the i-th objective, the probability of x being superior to x', under current metamodel uncertainties:

Far from 50%: one design clearly outperforms another, ‘not close’;

Around 50%: two designs displays indifferent performance, ‘close’;

Using indifference 50% as reference, the closeness on th objective can be measured with

| ' 0.5 |i

i

x x

2( ') ( ) ( ') ( ), ( ') ( ) ( ')cr cr cr cr cr cr

i i i i i i iH H p H H dH dH

x x x x x x x x

Probabilistic superiority and closeness

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1-0.5

00.5

1-3

-2

-1

0

1

2


x θ

crz • Conditionally realize the response in augmented space

• Extract relevant responses, and estimate one sample value of

[′ ]

• Iterate here

• Repeat sufficient times, to obtain an estimate of probability of superiority

) ( ) (( )j ji

cri

j ji iz z e yy y

Probabilistic superiority I


1

1( ) ( ') [ ( , ( ', ]

where( , ( ', | , | '

( ) ) )(

,

)

) )

Ncr cr cr j cr ji i i i

j

cr j cr j cr j cr ji i i

j

i

jH H h hN

h

p

h h h

q

x x x θ x θ

x θ x θ z x θ z

θ

θ

θ

x

x θ

crz

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1-0.5

00.5

1-3

-2

-1

0

1

2• Conditionally realize

the response in augmented space


• Iterate here


x' ( ) ( ')cr cri iH Hx x

) ( ) (( )j ji

cri

j ji iz z e yy y

Probabilistic superiority II


-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1-0.5

00.5

1-3

-2

-1

0

1

2

x θ



• Iterate to get multiple samples


1

( ) ) )( )

1( ) ( ') [ ( , ( ', ]

Ncr cr cr j cr ji

j

iji ij

H H h hpN q

x θ x θθθ

x x

) ( ) (( )j ji

cri

j ji iz z e yy y

( ) ( ')cr cri iH Hx x

Probabilistic superiority III

x θ





-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-1-0.5

00.5

1-2

0

2

4

6

1

( ) ) )( )

1( ) ( ') [ ( , ( ', ]

Ncr cr cr j cr ji

j

iji ij

H H h hpN q

x θ x θθθ

x x

) ( ) (( )j ji

cri

j ji iz z e yy y


Probabilistic superiority III


• Conditionally realize the response in augmented space



• Estimate probability of superiority based on available samples

-1 -0.5 0 0.5 1 1.5 20

5

10

15

20

25

Occurrence

1

1( ) ( ') 0 [ ( ) ( ')] 0

crNcr cr j

M i i i ijcr

H H H HN

x x x x



Probabilistic superiority IV


H1(x)

H2(

x)

( 1)

( 1)

' 1,2( | ) m

Convergence

in max | '

ind

0.5 |

icator:

kP

kconv P ii

I

x Xx X x x

Comparison to all precedent ones

We require each member in current Pareto set to stay close enough to at least one of

precedent members

Take account into all objectivesAggregation of

local indicators

Stopping criteria III

( )

( 1)max ( | ) [Threshold constanGlobal convergence indicator

t:

]k

P

kconv PI

x Xx X


H1(x)

H2(

x)

We require each member in current Pareto set to stay close enough to at least one of

precedent members

Take account into all objectives

Stopping criteria III

Sample–based design of DoE

• Generate candidate DoE sampled for y=[x, θ] based on some preference density

• Keep (small) percentage of the candidate samples based on some utility metric U(y)

• Cluster retained samples to desired number of experiments and keep one experiment per cluster, the one that has the largest value of U(y)

Sample–based design of DoE

• Generate candidate DoE sampled for y=[x, θ] based on some preference density πp(y)= πp(x) πp(θ|x) with separate characteristics for x, θ

• Keep (small) percentage of the candidate samples based on some utility metric U(y)

• Cluster retained samples to desired number of experiments and keep one experiment per cluster, the one that has the largest value of U(y)

Iteration 2

H1(x)

H2(

x)

x2

x1

Pareto set iteration 2 XP2

Domains close to potential members of pareto set (exploration)

DoE preference density for x I

x2

x1


H1(x)

H2(

x)


Exclude current solutions ‘close to’ precedent ones


XDoE = XP - {set of convergent points}

1

1( | ) ( ) DoE

jDoE DoE

n

pjDoE

Kn b

X xx x

( 1)

( 1)

1,21,2, ,( | ) min max | ' 0.5 |

kp

kconv p iir n

I

x X x x

Domains close to potential members of pareto set (exploration)

DoE preference density for x I

θ

Integrand( , ) ( )

ih pθx θ

1

1

Aggregating all objectives , , :1( | ) ( | ), where ( | ) ( , ) ( )

n

n

p pi i ij

h h

h pn

θ x θ x θ x x θ θ

Regions that contribute more to the integrand(importance sampling densities)

Preference density in augmented ( , ) ( |

space) ( | )p p DoE p x θ x X θ x

DoE preference density for θ

VARM[hi(x,θ)] (predicted) variance of ith performance measure (hi) under metamodel uncertainties for z

“Performance” Function

p[z(y)] p[h(y)]

2

1 ( )

VAR ( , ) ( )[ | ]z

j j

ni

M i ij j z z

hhz

x,θ

x,θx θ x,z θ

( )= max VAR ( , )M iiU hy x θ

Utility metric

Illustrative example IZhang, J. and A.A. Taflanidis (2017). “Multi-objective

optimization for design under uncertainty problems through surrogate modeling in augmented input

space”. Structural and Multidisciplinary optimization, under review.

Design of half-car nonlinear suspension system (4 design variables)

Random Road Surface (ISO 8608 spectral

characterization)

15 uncertain variables

1 2arg min{ ( ),

( , ) (

( )}

( ) )X

Θ

P

ii

H H

h p dH

x

X x x

x θ θ θx

2 ( , ) θx ftf ftrSD SDh

1ln( ) ln( ) (, )

x θ ac

b

R bh MS

~

1 2 2

ln( ) ln( )[ ( ), ( ) | , ]acb RMS

acGPM RMS bh

zz y σ y x θ

~ ~

2 [ ( ), ( ) | , ]GPMftf ftrh SD SD zz y σ y x θRoad holding: Average (RMS) force for

front & rear tire force

Passenger comfort: Probability (RMS) acceleration will exceed acceptable threshold

~~ ~

21 2 2 3/22 2 2 2

~ ~

2

ln( ) ln( ) (ln( )[ ( ), ( ) | , ](

[

ln

( ), ( ) |

( ) ln( ))

, ]

ac

acac ac

Racac acGPM

GPMft

MSb RMSb RMS b

f ft

RMS

r

RMSRMS b RMSh

h SD S

b

D

xx x

x x

z

z x

z y σ y x θ

z y σ y x θ

Illustrative example II

1 2arg min{ ( ), (

(

)}

( ) , ) ( )

GPM GPM

GP

PX

iM

iΘh p d

H H

H

x

x θ θ

x x

x θ

X

Illustrative results I

H1(x)

H2(

x)

Illustrative sample iteration

H1(x)

H2(

x)

Pareto front across iterations (performance evaluated by metamodel)

Illustrative results II

H1(x)

H2(

x)

Pareto front across iterations (performance evaluated by actual model)

Illustrative results III

H1(x)

H2(

x)

Comparison of pareto front to benchmark solution

3500 model evaluations


Illustrative results IV

3500 model evaluations 2200 model evaluations

Illustrative results V

H1(x)

H2(

x)

Müller, J. (2017). SOCEMO: Surrogate Optimization of Computationally Expensive Multiobjective Problems. INFORMS Journal on Computing, 29(4), 581-596



Reference solutions

SOCEMOProposed approach


Illustrative results VI

Outline






Model

nΘ θ

θ1

θ2

Stochastic sampling I

PDF p(θ)

( )z θ

response

( )h θsystem function

“Performance” evaluation

Target probability density

( ) ( )( )= ( ) ( )( ) ( )

Θ

h p h ph p d

θ θθ θ θθ θ θ

θ1

θ2

Target PDF π(θ)

Examples

Importance Sampling (IS),

Posterior (Bayesian) analysis, Subset

Simulation (SS) …

How to sample from the target distribution when the response is evaluated under complex numerical model (simulator)?

Different approaches exist for stochastic sampling Rejection Sampling Markov chain Monte Carlo (MCMC) …

Higher efficiency requires the proposal density q(θ) being close to the target PDF π(θ)

Typically non-trivial task for tough (rare event, peaked posterior) target distributions

Use samples (trials) from some proposal density q(θ) to obtain

samples from target density π(θ)

EfficiencyIssue

[number of trialsneeded to obtain one equivalent independent sample]

Stochastic sampling II

• Construct a series of intermediate densities between p(θ) and π(θ)

• Smaller change between adjacent densities possible to efficiently sample from πj (θ) using πj-1(θ) as proposal density

Sequential sampling

π0(θ) ≡ p(θ)

πn(θ) ≡ π(θ)

π1(θ)

πj(θ)

πj+1(θ)

SubsetSimulation,CrossEntropyIS,…

TransitiveMCMC…

nΘ θ

θ1

θ2

Metamodel-aided stochastic sampling

PDF p(θ)

( )z θ

Predicted response

( )GPMh θ

Approximated target probability density

( ) ( )( )= ( ) ( )( ) ( )

GPMGPM G

Θ

PMGPM

h p h ph p d

θ θθ θ θθ θ θ

θ1

θ2

Approximated Target PDF πGPM(θ)

Surrogatemodel

θ1θ2

z

Predictive system function

“Performance” evaluation

( )z θ

Metamodelj

πn(θ) ≡ π(θ)

π1(θ)

πj(θ)

πj+1(θ)

π0(θ) ≡ p(θ)

Metamodel0

Metamodelj+1

Metamodeln

Global metamodel

Sequential sampling

1st iteration (k= 1)

InitializationGet initial experiments and

evaluate response

Utilize all available training points to formulate the GPM

Stochastic sampling to simulate samples {θ}s

(k)

from πGPM(k)(θ)

Iterative metamodel-aided stochastic sampling

Iterative implementation, to gradually converge to target

density

Preference for rejection sampling o Independent samples

o Exploit metamodel capability in providing vectorised predictions

1st iteration (k= 1)

InitializationGet initial experiments and

evaluate response

Utilize all available training points to formulate the GPM

Stochastic sampling to simulate samples {θ}s

(k)

from πGPM(k)(θ)

Iterative metamodel-aided stochastic sampling

Stochastic sampling to obtain samples from π(θ) using πGPM(θ)

as proposal density

Stopping criteriaSatisfied?

k=k+1no

Proceed to next iteration

Perform refinement DoE and evaluate

response

STOP

Iterative implementation, to gradually converge to target

density

yes

Zhang, J. and A.A. Taflanidis (2017). “Adaptive Kriging stochastic sampling and

density approximation and its implementation to rare-event estimation”. ASCE-ASME

Journal of Risk and Uncertainty in Engineering Systems, in press.

Stopping criteria

2( ) ( 1) ( ) ( 1) 21ˆ ( ), ( ) [ ( ) ( )]2

GPM k k k kGPM GPM GPMH Θ

D d θ θ θ θ θ Hellinger distance

Approximated target PDF in kth iteration πGPM(k)(θ)

Approximated target PDF in k-1th iteration πGPM(k-1)(θ)

Bounded symmetric metric ∈[0, 1], where higher value → higher discrepancy

Stopping criteria

2( ) ( 1) ( ) ( 1) 21ˆ ( ), ( ) [ ( ) ( )]2

GPM k k k kGPM GPM GPMH Θ

D d θ θ θ θ θ Hellinger distance

Approximated target PDF in kth iteration πGPM(k)(θ)

Approximated target PDF in k-1th iteration πGPM(k-1)(θ)

2( ) ( 1)

( ) ( 1) ( ) ( 1)1

1 1 ( ) ( )2 ( |{ } ,{ } ) ( |{ } ,{ } )

dN k r k r

r k k r k krd d s

GPM G

s d

P

s s

M

N q q

θ θ

θ θ θ θ θ θ

IS density qd for estimation based on samples from πGPM(k)(θ) [{θ}s

(k)] and πGPM(k-1)(θ) [[{θ}s(k-1)]

DoE strategy I: Target-based

Objective: enhance the metamodel accuracy around the sampling domain of interest (target PDF)

– Preference function: πp(θ) =πGPM(k)(θ)

– Utility metric: predicted variance of system function (h) under response prediction (z) metamodel uncertainties

“Performance” Function

Propagating the uncertaintiesp[z(θ)] p[h(θ)]

2

1 ( )

[VAR ( , ) ( )| ]z

j j

ni

M i ij j z z

hhz

θ

z θx θ θ

DoE strategy II: Response-based

Objective: enhance the metamodel accuracy around domains with insufficient information about metamodel output (response)

– Preference function: πp(θ) =πjGPM(k)(θ)

– Utility metric: differential entropy between prior/ posterior response prediction

Obtain the information gain by differential entropy DEz(θ)

Posterior (predictive distribution)Prior

(process variance) Lower DEz(θ)less information extracted from experiments for point (new

information might change predictions)

2 2

1 1

2 2

1

1DE ( ) [log(2 )] ( ( ))2

exp [log( ) log( ( ))]

z zz

z

n nn

i ii i

n

i ii

e

z θ θ

θ

• Combines the target-based and response-based design of experiments

•Simulate large number of samples from target-basedDoE PDF•Maintain only the samples that have high VARM [h(θ)]

Hybrid DoE

•Simulate large number of samples from response-based DoE PDF•Maintain only the samples that have small DEz(θ)

Combine, cluster and maintain only nearest neighbor to the centroids

Final Hybrid DoE

formulating an IS based on πGPM(θ) and relying again on established metamodel [AK-MCS (Echard et al. 2011)]

formulating an IS based on πGPM(θ) and relying on actual model [eg meta-IS (Dubourg et al. 2013), AM-SIS (Pedroni and Zio 2015)]

Once convergence to πGPM(θ) has been established we can calculate P(F)

1

1 ( ) ( )ˆ ( ) ; ~ ( )( )

cN GPM r rGPM r r

crrc c

h pP F qN q

θ θ θ θθ

1

1 ( ) ( )( ) ; ~ ( )( )

cN r rr r

crrc c

h pP F qN q

θ θ θ θθ

Based on πGPM(θ)

(samples)

IS-based rare event estimation

( ) ( ) ( )

( ) ( ) ( ) ( )( ) ( | )( )( ) ( )

FΘ

F F

FΘ

P F I p d

I p I pp FP FI p d

θ θ θ

θ θ θ θθ θθ θ θ

[ ]Tlf lrC Cx

Illustrative examplesP1

P2 P3 P4 P5 P6A1, E1

A1, E1

A2, E2

ul um ur

AK-SSD applied with

addition of 2nθexperiments per

iteration

Case 1: Failure related to um

Case 2: Failure related to um,ur

and ul

server

uy kp

Isolator Restoring force

Isolator displacement

Ground Motion

(Stochastic model incorporating near fault effects)

δi

Fy,i

ki

-δy,i

Restoring force Fi for ith story as function of drift δi

δy,i

aiki

-Fy,i

Fim4=400 ton

k1 =460 MN/m

m3=500 ton

m2=500 ton

m1=500 ton

k2 =368 MN/m

k3 =276 MN/m

k1 =184 MN/m

kl

nθ=10 variables

Truss example

FIS-example

nθ=36 variables

Results IHellinger distance between approximated target PDFs in subsequent iterations

• Large initial improvement and then plateau reached in latter iterations small discrepancy between distributions less impactful refinement DoE

Results II

• Decreasing trend approximated distribution approaches actual one

Hellinger distance between approximated target PDFs and actual target PDF across k

• Large initial improvement and then plateau reachedconvergence of approximated PDFs, good indicator for convergence to target PDF

Results IIIFailure probability estimates using metamodel or exact numerical model

ˆ ( )GPMP F

ˆ ( )GPMP F

ˆ ( )GPMP F

Results IV

• Large initial improvement and then plateau reached as convergence establishedconvergence of approximated PDFs good indicator for efficiency of IS densities

Efficiency of second stage (N required to get cov of 5%) if samples from πkrig(k)(θ)were to be used to form IS densities

: estimated failure probability fully based on Kriging metamodel: estimated failure probability using stochastic simulation

n : total high-fidelity simulations needed for constructing Kriging metamodelNtot : total high-fidelity simulations needed to establish a stochastic-simulation

estimation with 5% c.o.v.

Examples Approaches ˆ( )P F ˆ ( )GPMP F n Ntot

Truss-example case 1

MCS 3.5×10-5 - - 1.14×107 PCE - 3.2×10-5 78 -

AK-SSD 3.5×10-5 3.49×10-5 230 322 Truss-

example case 2

MCS 8.14×10-5 - - 4.84×106

AK-SSD 8.14×10-5 8.18×10-5 240 315

FIS- example

MCS 1.93×10-4 - - 2.07×106 SS-AKSD 2.16×10-4 - - 1.44×105 AK-SSD 1.93×10-4 1.93×10-4 684 2454

MCS: Direct Monte CarloPCE: Polynomial-Chaos Kriging approach (PCE) by (Schöbi et al. 2016)SS-AKSD: SS with adaptive kernel sampling densities (SS-AKSD) by (Jia et al. 2015)

ˆ( )P Fˆ ( )krigP Fˆ( )P Fˆ ( )krigP F

Results V

Conclusions

• Surrogate modelling can facilitate significant computational benefits for UQ applications.

• Gaussian process metamodels are especially relevant in this context due to their holistic treatment of uncertainty in metamodel predictions.

• Metamodel accuracy should be adaptively controlled (coupled with adaptive DoE) with goal to converge to accurate solutions leveraging (perhaps) inaccurate models (but accurate enough in domains of interest)

• For design uncertainty development of metamodels in augmented input space can facilitate additional computational benefits

• Sample-based DoE is an efficient approach for identifying new experiments and can provide significant accuracy improvement

Acknowledgments

• Students– Jize Zhang– Camilo Medina

• Funding Agencies

Thank You!

Iterative surrogate model development...Iterativesurrogate model development with applications to...

Documents

Transcript of Iterative surrogate model development...Iterativesurrogate model development with applications to...