Bayesian Experimental Design for Models with …...Model discrimination for intractable likelihoods...

Motivating Example Static Design Design for Intractable Models Examples Ending

Bayesian Experimental Design for Models with

Intractable Likelihoods

Chris Drovandi (me) and Tony Pettitt

November 6, 2012

Chris Drovandi Bayes on the Beach


Macroparasite Immunity

Estimate parameters of a Markov process model explainingmacroparasite population development with host immunity

212 hosts (cats) i = 1, . . . , 212. Each cat injected with lijuvenile Brugia pahangi larvae (approximately 100 or 200).

At time ti host is sacrificed and the number of matures arerecorded

Host assumed to develop an immunity

Three variable problem: M(t) matures, L(t) juveniles, I (t)immunity.

Only L(0) and M(ti) is observed for each host



0 200 400 600 800 1000 12000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time

Pro

porto

n of

Mat

ures



Trivariate Markov Process of Riley et al (2003)

M(t) L(t)

I(t)

Mature Parasites Juvenile Parasites

Immunity

Maturation

Gain of immunity Loss of immunity

Death due to im

munity

Natural death

Natural death

Invisible

Invisible

Invisible

γL(t)

νL(t) µI I (t)

βI(t)L

(t)

µLL(t)

µMM

(t)



The Model and Intractable Likelihood

Deterministic form of the model

dL

dt= −µLL− βIL− γL,

dM

dt= γL− µMM,

dI

dt= νL− µI I ,

µm, γ fixed. ν, µL, µI , β require estimation

Likelihood based inference appears intractable.

Drovandi et al (2011) fit the model to data using approximateBayesian computation



The problem

ν

µ L

posterior from dataA

0 0.5 1 1.5 2 2.5 3

x 10−3

−0.01

0

0.01

0.02

0.03

ν

µ L

estimated posterior from dataB

0 0.5 1 1.5 2 2.5 3

x 10−3

−0.01

0

0.01

0.02

0.03

Figure : (a) bivariate density estimate of (ν, µL) obtained from Drovandiet al (2011). (b) bivariate density estimate based on samples from theparametric approximation of the posterior in (a).

Posterior becomes the prior for future experiments (figureabove)

How to choose the sacrifice times and initial larvae injectionto gain most information about parameters (ν and µL)???



Static Experimental Design

Set-up: Have prior distribution p(θ) for parameter ofstatistical model, with likelihood p(y|θ,d).Wish to design for next n observations. Design variable:d = (d1, . . . , dn).

Define (general) utility function u(d, y,θ). y is future data.

u(d) = Eθ,y[u(d, y, θ)] =

∫

y

∫

θ

u(d, y, θ)p(y|d, θ)p(θ)dθdy,

Objective

d∗ = argmaxd∈D

u(d).

Too difficult to do directly



MCMC approach - Muller 1999

Turn optimisation problem into simulation problem

h(d,θ, y) ∝ u(d, y,θ)p(y|d,θ)p(θ)p(d).

Admits ∝ u(d)p(d) as marginal distribution

Optimal design d∗ is mode of marginal

If marginal is flat, may be beneficial to sample from

h(d,θ1, . . . ,θJ, y1, . . . , yJ) ∝ p(d)

J∏

j=1

u(d, yj,θj)p(yj|d,θj)p(θj),

for large J. A marginal is ∝ u(d)Jp(d)

Muller (1999) propose MCMC for sampling.q(d∗, y∗,θ∗|d, y,θ) = p(y∗|θ∗,d∗)p(θ∗)q(d∗|d).Amzal et al (2006). Use SMC by gradually increasing J.



Utility functions and Computational issues

Frequentist designs use Fisher information matrix. u(d,θ)(utility independent of data, fun times!). Called robust orpseudo-Bayesian designs.

Bayesian utilities typically u(d, y). Based on posterior, soindependent of θ. Still need θ for proposal (not so fun times!)

Kullback-Leibler divergence u(d, y) = KL(p(θ|y,d)||p(θ))Concentration of Posterior distribution

Set u(d, y) as entropy of p(θ|y, d)Posterior precision u(d, y) = 1/det(var(θ|y, d))

u(d, y) must usually be approximated (also not so cool...)

Bayesian designs more difficult computationally: Sampling overlarge design space + parameter + future data (hard slog)



Muller Algorithm

A closer look at the Muller algo... di−1 is current, with utilityui−1 = u(di−1,θi−1, yi−1)

Propose d∗ ∼ q(d|di−1), θ∗ ∼ p(θ), y∗ ∼ p(y|θ∗,d∗).Compute u∗ = u(d∗,θ∗, y∗)

Compute α = min(

1, u∗p(d∗)q(di−1|d∗)

ui−1p(di−1)q(d∗|di−1)

)

Likelihood functions cancel as per ABC MCMC (Marjoram etal (2003))

Now simply need utility function that does not requirelikelihood evaluation



Utility for intractable likelihoods

Fisher information not always helpful here (It’s not Bayesiananyway!). Can be done for Markov processes (see Pagendamand Ross (2012)), but expensive.

Bayesian utilities based on posterior.

Approximate true posterior via approximate Bayesiancomputation (later)

Utility

KLD between prior and ABC posterior (?)

Concentration of ABC posterior (e.g. entropy or precision).Precision straightforward to calculate



Approximate Bayesian Computation

Simulation based method that does not involve likelihoodevaluations

Involves a joint ‘approximate’ posterior distribution

p(θ, x|y, ǫ) ∝ g(y|x, ǫ)p(x|θ)p(θ)where g(y|x, ǫ) is a weighting function. Popular choiceg(y|x, ǫ) = 1(ρ(y, x) ≤ ǫ)How to choose ρ(y, x)?

Usually based on a set of low-dimensional set of summarystatisticsHere use full data (low-dimensional designs)

Choice of ǫ trade-off between accuracy and efficiency (andMonte Carlo error)

The effect of the approximation

Idea is that marginal p(θ|y, ǫ) ≈ p(θ|y)Errors from insufficient summaries and ǫ > 0



ABC Algorithms

Many algorithms available: ABC rejection (Beaumont et al2002), MCMC ABC (Marjoram et al (2003)), SMC ABC(Sisson et al (2007), CD+TP (2011))

MCMC ABC and SMC ABC suitable for single data analysis

In context of static design, need ABC posterior at eachiteration for a new y

ABC rejection amendable to multiple datasets from samemodel



ABC Rejection

1 Generate θi ∼ p(θ) for i = 1, . . . ,N

2 Simulate xi ∼ p(y|θi,d) for i = 1, . . . ,N

3 Compute discrepancies ρi = ρ(y, xi) for i = 1, . . . ,N, creatingparticles {θi, ρi}Ni=1

4 Sort the particle set via the discrepancy ρ

5 Discard (1− α)N of the particles with the highestdiscrepancy. Effectively ǫ = ραN

Steps 1 and 2 are independent of data and {θi, xi}Ni=1 can bestored.Discrepancy function:

ρ(y, x) =D∑

i=1

|yi − xi |stdp(θ)(xi )

, (1)



Discretising the Design Space

Discretise design (time) space: tmin, tmax , tinc

Prior simulations get recorded at each design point in designspace

Prior simulations are done before Muller algorithm

Metropolis-Hastings sample over discrete design space



Example - pure death process

SI Model, pure death, closed population.P(S(t + δt) = i − 1|S(t) = i) = b1iδt + o(δt).b1 ∼ log normal(−0.005, 0.01)

Binomial likelihood. Allows comparison of likelihood-free(100,000 prior simulations), likelihood-based designs

tmin = 0.01, tmax = 10 and tinc = 0.01.

Likelihood-free: ABC posterior. Likelihood-based: Posteriorvia importance sampling

0 2 4 6 8 100

10

20

30

40

50

days

Num

ber o

f inf

ecte

ds

A



Results

−4 −2 0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25

t

dens

ity

ABCimportance

(a) 1 design point

−2 0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25

0.3

0.35

t

dens

ity

ABCimportance

(b) 2 design points

−2 0 2 4 6 8 10 120

0.05

0.1

0.15

0.2

0.25

0.3

0.35

t

dens

ity

ABCimportance

(c) 3 design points

−2 0 2 4 6 8 10 120

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

t

dens

ity

ABCimportance

(d) 4 design points

Figure : Experimental design results for death model for up to 4 designpoints. Solid lines are based on the ABC approximation to the trueChris Drovandi Bayes on the Beach


Macroparasite Population Evolution

Prior on (ν,√µL) is bivariate normal with mean

(0.0013,0.0854), variance of (1.0469 × 10−7, 0.0012) and acorrelation of -0.6865.

Lot’s of prior predictive variability: trial 200,000 and 800,000prior simulations

tmin = 1, tmax = 300 and tinc = 1 days

0 50 100 150 200 250 3000

10

20

30

40

50

60

70

80

90

100

days

Number o

f matures

C



Results

23

23.5

24

24.5

25

25.5

26

1 2 3 4

number of design points

log(u

(d,y)

)

(a) death process

132

134

136

138

140

142

number of design points

log(u

(d,y)

)

1 2 3 4

(b) macroparasite proces

Figure : Change in utility



Results

−100 0 100 200 300 4000

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

−3

t

dens

ity

200000800000

(a) 1 design point

−50 0 50 100 150 200 250 300 350 4000

1

2

3

4

5

6

7x 10

−3

t

dens

ity

200000800000

(b) 2 design points

−50 0 50 100 150 200 250 300 3500

1

2

3

4

5

6

7

8

9x 10

−3

t

dens

ity

200000800000

(c) 3 design points

−50 0 50 100 150 200 250 300 3500

0.002

0.004

0.006

0.008

0.01

0.012

t

dens

ity

200000800000

(d) 4 design points

Figure : Experimental design results for the macroparasite model for upto 4 design points. The solid line and dashed line results are based onChris Drovandi Bayes on the Beach


Results - Gain in utility

1 2 3 4

6.5

7

7.5

8

x 1011

no. of design points

utilit

y

Figure : The expected utility (with error bars) of the designs d1 = (100),d2 = (50, 300), d3 = (50, 150, 300) and d4 = (25, 100, 175, 300) for themacroparasite model.



Stochastic DE - PK Model

dXt =(

DkakeCl

e−kat − keXt

)

dt + σdWt

Log-normal priors on ka, ke and Cl

Exact simulation not available. Approximate simulation viaEuler’s method

200,000 prior simulations with D = 4

tmin = 0.25, tmax = 12 and tinc = 0.25 days



Results

0 2 4 6 8 10 12 140

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

sampling time

dens

ity

(a) 1 design point

−4 −2 0 2 4 6 8 10 12 14 160

0.05

0.1

0.15

0.2

0.25

sampling time

dens

ity

(b) 2 design points

−2 0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25

sampling time

dens

ity

(c) 3 design points

−2 0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25

sampling time

dens

ity

(d) 4 design points

Figure : PK Example.Chris Drovandi Bayes on the Beach


Limitations and Future Work

Limitations

Only suitable for low dimensional designs. Bayesian highdimensional designs difficult even with likelihood!

Initial condition needs to be fixed, or only a few options

ABC rejection needs informative priors → but so doesBayesian design in general

Future Work

Macroparasite Example: Extend to choose either 100 or 200initial larvae

Extend to higher dimensional design (ABC rejection might die)

Model discrimination for intractable likelihoods

Indirect inference?



References

Drovandi and Pettitt (2012). Bayesian experimental design formodels with intractable likelihoods.http://eprints.qut.edu.au/53924/.Cook, A. R., Gibson, G. J., and Gilligan, C. A. (2008).Optimal observation times in experimental epidemic processes.Biometrics, 64(3):860-868.Muller, P. (1999). Simulation based optimal design. InBayesian statistics 6: proceedings of the Sixth ValenciaInternational Meeting.Beaumont, M. A., Zhang, W., and Balding, D. J. (2002).Approximate Bayesian computation in population genetics.Genetics, 162(4):2025-2035.Drovandi, C. C., Pettitt, A. N., and Faddy, M. J. (2011).Approximate Bayesian computation using indirect inference.Journal of the Royal Statistical Society: Series C (AppliedStatistics), 60(3):503-524.


Bayesian Experimental Design for Models with …...Model discrimination for intractable likelihoods...

Documents

Transcript of Bayesian Experimental Design for Models with …...Model discrimination for intractable likelihoods...