Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum...
Transcript of Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum...
![Page 1: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/1.jpg)
Observations to Models Lecture 3
!
Exploring likelihood spaceswith CosmoSIS
Joe ZuntzUniversity of Manchester
![Page 2: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/2.jpg)
BoltzmannP(k) and DA(z) Nonlinear P(k)
Photometric redshift n(z)
Theory shearCℓ
Binned Theory Cb
Likelihood
MeasuredCb
Defining Likelihood Pipelines
![Page 3: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/3.jpg)
BoltzmannP(k) and DA(z) Nonlinear P(k)
Photometric redshift n(z)
Theory shearCℓ
Binned Theory Cb
Likelihood
MeasuredCb
Defining Likelihood Pipelines
Photo-z Systematics
Shape Errors
![Page 4: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/4.jpg)
Modular Likelihood Pipelines
• Each box is an independent piece of code (module)
• With well-defined inputs and outputs
• This lets you replace, compare, insert, and combine code more easily
![Page 5: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/5.jpg)
CosmoSIS
• CosmoSIS is a parameter estimation framework
• Connect inference methods to cosmology likelihoods
https://bitbucket.org/joezuntz/cosmosis
![Page 6: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/6.jpg)
Setting up CosmoSIS on teaching machines
• Open VirtualBox
• Click “start”
• Password cosmosis
• Click Terminal
> su
• use password cosmosis
> cd /opt/cosmosis
> source config/setup-cosmosis
![Page 7: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/7.jpg)
Pipeline Example> cosmosis demos/demo2.ini
> postprocess -o plots -p demo2 demos/demo2.ini
Parameters CMB Calculation
Planck Likelihood
Bicep Likelihood
Saved Cosmological Theory Information TotalLikelihood
https://bitbucket.org/joezuntz/cosmosis/wiki/Demo2
![Page 8: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/8.jpg)
Basic Inference Methods• You’ve made a posterior function from a
likelihood function and a prior
• Now what?
• Explore the parameter space and build constraints on parameters
• Remember: “the answer” is a probability distribution
![Page 9: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/9.jpg)
Basic Inference: Grids
• Low number of parameters ⇒can try grid of all possible parameter combinations
• Number of posterior evaluations = (grid points per param)(number params)
• Approximate PDF integrals as sums
· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·· · · · · · · · · ·
x
y
![Page 10: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/10.jpg)
Basic Inference: Grids
• Get constraints on lower dimensions by summing along axes
P (X) =
ZP (XY )dy
⇡X
i
�YiP (X,Yi)
/X
i
�P (X,Yi)
![Page 11: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/11.jpg)
Grid Example> cosmosis demos/demo7.ini
> postprocess -o plots -p demo7 demos/demo7.ini
ParametersGrowth
FunctionBOSS
Likelihood
Saved Cosmological Theory Information TotalLikelihood
https://bitbucket.org/joezuntz/cosmosis/wiki/Demo7
![Page 12: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/12.jpg)
Basic Inference: Maximum Likelihood/Posterior
• The modal value of a distribution can be of interest, especially for near-symmetric distributions
• Find mode by solving gradient=0
• Maximum-likelihood (ML)Maximum a Posteriori (MAP)
![Page 13: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/13.jpg)
Basic Inference: Maximum Likelihood/Posterior
• In 1D cases solve:
!
• Multi-dimensional case:
d logP
dx
= 0
r logP = 0rP = 0
dP
dx= 0 or
or
![Page 14: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/14.jpg)
Basic Inference: Maximum Likelihood/Posterior
• Around the peak of a likelihood the curvature is related to distribution width
• Usually easiest to do with logs, for example for a Gaussian:
d
2logP
dx
2
����x=µ
= � 1
�
2
![Page 15: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/15.jpg)
ML Exercise• Find the maximum likelihood of our LMC cepheid
likelihood, assuming fixed σi:
P (V obs
i |p) / 1
�2
int
+ �2
i
exp�0.5
✓(V obs
i � (↵+ � log
10
Pi))2
�2
int
+ �2
i
◆
• Much easier to use log(P)
![Page 16: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/16.jpg)
ML Example> cosmosis demos/demo4.ini
> postprocess -o plots -p demo4 demos/demo4.ini
Parameters CMB Calculation
Planck Likelihood
Saved Cosmological Theory Information TotalLikelihood
https://bitbucket.org/joezuntz/cosmosis/wiki/Demo4
![Page 17: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/17.jpg)
Basic Inference: Maximum Likelihood/Posterior• In simple cases you can solve these analytically.
Otherwise need some kind of optimization
• Numerical solvers - various algorithms; google scipy.minimize for a nice list
• Usually easier with gradients e.g. Newton-Raphson
• See also Expectation-Maximization algorithm
![Page 18: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/18.jpg)
Monte Carlo Methods• Collection of methods
involving repeated sampling to get numerical results
• e.g. chance of dealing two pairs in five cards?
• Multi-dimensional integrals, graphics, fluid dynamics, genetics & evolution, AI, finance
![Page 19: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/19.jpg)
Sampling Overview
• “Sample” = “Generate values with given distribution”
• Easy: distribution written down analytically
• Harder: can only evaluate distribution for given parameter choice
![Page 20: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/20.jpg)
Sampling Motivation• Simulate systems
• Short-cut to expectations if you hate maths
• Approximate & characterize distributions
• Estimate distributions mean, std, etc.
• Make constraint plots
• Compare with other data
![Page 21: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/21.jpg)
![Page 22: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/22.jpg)
Direct Sampling• Simple analytic distribution:
can generate samples pseudo-randomly
• Actually incredibly complicated!
• import scipy.stats X = scipy.stats.poisson(5.0) x = X.rvs(1000)"
• Be aware of random seeds
![Page 23: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/23.jpg)
Direct Sampling
• e.g. approx solution to Lecture 1 homework problem
!
!
!
• Often easier than doing sums/integrals
![Page 24: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/24.jpg)
Monte Carlo Markov Chains• Often cannot directly sample from a distribution
• But can evaluate P(x) for any given x
• Markov chain = memory-less sequence
!
• MCMC: Random generation rule for f(x) which yields xn with stationary distribution P(x) i.e. chain histogram tends to P
xn = f(xn�1)
![Page 25: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/25.jpg)
MCMC• Jump around parameter space
• Number of jumps near x ~ P(x)
!
!
!
• Standard algorithms: x = single parameter setAdvanced algorithms: x = multiple parameter sets
![Page 26: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/26.jpg)
Metropolis-Hastings• Given current sample xn in parameter space
• Generate new proposed sample
• For simplest algorithm
x
n+1 =
(p
n
with probability max
⇣P (pn)P (xn)
, 1
⌘
x
n
otherwise
pn ⇠ Q(pn|xn)
Q(p|x) = Q(x|p)
![Page 27: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/27.jpg)
Metropolis-Hastings
xn
![Page 28: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/28.jpg)
Metropolis-Hastings
Q(pn|xn)
![Page 29: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/29.jpg)
Metropolis-Hastings
P (pn) > P (xn)
Definite accept
![Page 30: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/30.jpg)
Metropolis-Hastings
P (pn) < P (xn)
Let ↵ = P (pn)/P (xn)
Generate u ⇠ U(0, 1)
Accept if ↵ > u
![Page 31: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/31.jpg)
Metropolis-Hastings
maybe accept
unlikely toaccept
![Page 32: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/32.jpg)
Metropolis-Hastings
![Page 33: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/33.jpg)
Metropolis-Hastings Proposal
• MCMC will always converge in ∞ samples
• Proposal function Q determines efficiency of MCMC
• Ideal proposal has cov(Q) ~ cov(P)
• (Multivariate) Gaussian centred on xn usually good Tune σ (+covariance) to match P
• Ideal acceptance fraction ~ 0.2 - 0.3
![Page 34: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/34.jpg)
Metropolis Example> cosmosis demos/demo10.ini
> #This will take too long! Figure out how to change demo 7 to use Metropolis!
Parameters CMB Calculation
WMAPLikelihood
Saved Cosmological Theory Information TotalLikelihood
https://bitbucket.org/joezuntz/cosmosis/wiki/Demo10
![Page 35: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/35.jpg)
Metropolis-Hastings Convergence
• Have I taken enough samples?
• Many convergence tests available
• Easiest is visual!
![Page 36: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/36.jpg)
Metropolis-Hastings Convergence
• Bad Mixing
• Structure and correlations visible
Chain Position
x
![Page 37: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/37.jpg)
Metropolis-Hastings Convergence
• Good Mixing
• Looks like noise
• Must be true for all parameters in space
Chain Position
x
![Page 38: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/38.jpg)
Metropolis-Hastings Convergence
• Run several chains and compare results
• (Variance of means) / (Mean of variances) Less than e.g. 3%
![Page 39: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/39.jpg)
Metropolis-Hastings Burn in
• Chain will explore peak only after finding it
• Cut off start of chain
Chain Position
Prob
abilit
y
![Page 40: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/40.jpg)
Metropolis-Hastings Analysis
• Probability proportional to chain multiplicity
• Histogram chain, in 1D or 2D
• Can also thin e.g. remove every other sample
Prob
abilit
y
x
![Page 41: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/41.jpg)
Metropolis-Hastings Analysis
• Can also get expectations of derived parameters
E[f(x)] =
ZP (x)f(x)dx ⇡ 1
N
X
i
f(xi)
![Page 42: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/42.jpg)
Emcee Sampling• Goodman & Weare algorithm
• Group of live “walkers” in parameter space
• Parallel update rule on connecting lines - affine invariant
• Popular in astronomy for nice and friendly python package
![Page 43: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/43.jpg)
Emcee Sampling• Goodman & Weare algorithm
• Group of live “walkers” in parameter space
• Parallel update rule on connecting lines - affine invariant
• Popular in astronomy for nice and friendly python package
![Page 44: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/44.jpg)
Emcee Example> cosmosis demos/demo5.ini
> postprocess demos/demo5.ini -o plots -p demo5
Parameters Distance Calculation
Supernova Likelihood
Saved Cosmological Theory Information TotalLikelihood
https://bitbucket.org/joezuntz/cosmosis/wiki/Demo5
H0 Likelihood
![Page 45: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/45.jpg)
Model Selection• Given two models, how can we compare them?
• Simplest approach = compare ML
• Does not include uncertainty or Occam’s Razor
• Recall that all our probabilities have been conditional on the model, as in Bayes:
P (p|M) =P (d|pM)P (p|M)
P (d|M)
![Page 46: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/46.jpg)
Model Selection: Bayesian Evidence
• Can use Bayes Theorem again, on model level:
!
!
• Only really meaningful when comparing models:
P (M |d) = P (d|M)P (M)
P (d)
Model PriorsP (M1|d)P (M2|d)
=P (d|M1)
P (d|M2)
P (M1)
P (M2)
Bayesian Evidence Values
![Page 47: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/47.jpg)
Model Selection: Bayesian Evidence
• Likelihood of parameters within model:
!
!
• Evidence of model:
P (d|pM)
P (d|p)
![Page 48: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/48.jpg)
Model Selection: Bayesian Evidence
• Evidence is the bit we ignored before when doing parameter estimation
• Given by an integral over prior space
!
• Hard to evaluate - posterior usually small compared to prior
P (d|M) =
ZP (d|pM)P (p|M)dp
![Page 49: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/49.jpg)
Model Selection: Evidence Approximations
• Nice evidence approximations for some cases:
• Savage-Dickey Density ratio (for when one model is a subset of another)
• Akaike information criterion AICBayesian information criterion BIC Work in various circumstances
![Page 50: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/50.jpg)
Model Selection: Nested Sampling
ZL(✓)p(✓)d✓ =
ZL(X)dX
⇡X
Li�Xi
dX ⌘ P (✓)d✓
X = remaining prior volume
![Page 51: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/51.jpg)
Model Selection: Nested Sampling
• Also uses ensemble of live points
• Computes constraints as well as evidence
• Each iteration, replace lowest likelihood point with one higher up
• By cleverly sampling into envelope of active points
• Multinest software is extremely clever
• C, F90, Python bindings
![Page 52: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/52.jpg)
Multinest Example> cosmosis demos/demo9.ini
> postprocess -o plots -p demo9 demos/demo9.ini --extra demos/extra9.py
Parameters Distance Calculation
Supernova Likelihood
Saved Cosmological Theory Information TotalLikelihood
https://bitbucket.org/joezuntz/cosmosis/wiki/Demo9
H0 Likelihood
![Page 53: Observations to Models Lecture 3 Exploring likelihood ... · Basic Inference: Maximum Likelihood/Posterior • In simple cases you can solve these analytically. Otherwise need some](https://reader035.fdocuments.net/reader035/viewer/2022071212/6024792d3562ca696f3b8c00/html5/thumbnails/53.jpg)
Exercise
• With a prior s ~ N(0.5, 0.1) and other priors of your choice, write a Metropolis Hastings sampler to draw from the LMC Cepheid likelihood. Plot 1D and 2D constraints on the parameters.