Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU...
Transcript of Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU...
![Page 1: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/1.jpg)
CSE586, PSURobert Collins
Intro to Sampling Methods
CSE586 Computer Vision II
Penn State Univ
![Page 2: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/2.jpg)
CSE586, PSURobert Collins
A Brief Overview of Sampling
Monte Carlo Integration
Sampling and Expected Values
Inverse Transform Sampling (CDF)
Rejection Sampling
Importance Sampling
Markov Chain Monte Carlo
![Page 3: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/3.jpg)
CSE586, PSURobert Collins
Integration and Area
http://videolectures.net/mlss08au_freitas_asm/
![Page 4: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/4.jpg)
CSE586, PSURobert Collins
Integration and Area
http://videolectures.net/mlss08au_freitas_asm/
![Page 5: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/5.jpg)
CSE586, PSURobert Collins
Integration and Area
http://videolectures.net/mlss08au_freitas_asm/
total # boxes
![Page 6: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/6.jpg)
CSE586, PSURobert Collins
Integration and Area
http://videolectures.net/mlss08au_freitas_asm/
![Page 7: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/7.jpg)
CSE586, PSURobert Collins
Integration and Area
• As we use more samples, our answer should get more and more accurate
• Doesn’t matter what the shape looks like
(0,0)
(1,1)
area under curveaka integration!
(0,0)
(1,1)
arbitrary region(even disconnected)
![Page 8: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/8.jpg)
CSE586, PSURobert Collins
Monte Carlo Integration
upper boundc
f(x)
a b
Goal: compute definite integral of function f(x) from a to b
count the K samples that fall below the f(x) curve
K
Generate N uniform random samples in upper bound volume
N
KN
Answer= * Area of upper bound volume
![Page 9: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/9.jpg)
CSE586, PSURobert Collins
Monte Carlo Integration
Sampling-based integration is useful for computing the normalizing constant that turns an arbitrary non-negative function f(x) into a probability density function p(x).
Compute this via sampling (Monte Carlo Integration). Then:
Z
1Z
Note: for complicated, multidimensional functions, this is the ONLY way we can compute this normalizing constant.
![Page 10: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/10.jpg)
CSE586, PSURobert Collins
Sampling: A Motivation
If we can generate random samples xi from a given distribution P(x), then we can estimate expected values of functions under this distribution by summation, rather than integration.
That is, we can approximate:
by first generating N i.i.d. samples from P(x) and then forming the empirical estimate:
![Page 11: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/11.jpg)
CSE586, PSURobert Collins
Expected Values and Sampling
P(x) P(1) = 1/4P(2) = 1/4P(3) = 2/41 2 3
Example:
a discrete pdf
![Page 12: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/12.jpg)
CSE586, PSURobert Collins
Expected Values and Sampling (cont)
P(x)
1 2 3
generate 10 samples from P(x)
1 3 3 2 2 3 1 3 3 1x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
10
~
![Page 13: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/13.jpg)
CSE586, PSURobert Collins
Inverse Transform Sampling
It is easy to sample from a discrete 1D distribution, using the cumulative distribution function.
1 N
wk
k 1 Nk
1
0
![Page 14: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/14.jpg)
CSE586, PSURobert Collins
Inverse Transform Sampling
It is easy to sample from a discrete 1D distribution, using the cumulative distribution function.
1 Nk
1
0
1) Generate uniform u in the range [0,1]
u
2) Visualize a horizontal line intersecting bars
3) If index of intersected bar is j, output new sample xk=j j
xk
![Page 15: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/15.jpg)
CSE586, PSURobert Collins
Inverse Transform Sampling
Why it works:
inverse cumulative distribution function
cumulative distribution function
![Page 16: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/16.jpg)
CSE586, PSURobert Collins
Efficient Generating Many Samples
Basic idea: choose one initial small random number; deterministically sample the rest by “crawling” up the cdf function. This is O(N).
Due to Arulampalam
odd property: you generate the “random”numbers in sorted order...
(naive approach is NlogN, but we can do better)
![Page 17: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/17.jpg)
CSE586, PSURobert Collins
Example: Sampling From an Image
“Likelihood” image
500 samples 1000 samples 5000 samples 10000 samples
![Page 18: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/18.jpg)
CSE586, PSURobert Collins
A Brief Overview of Sampling
Inverse Transform Sampling (CDF)
Rejection Sampling
Importance Sampling
For these two, we can sample from an unnormalized distribution function.
That is, to sample from distribution P, we only need to know a function P*, where P = P* / c , for some normalization constant c.
![Page 19: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/19.jpg)
CSE586, PSURobert Collins
Rejection Sampling
Need a proposal density Q(x) [e.g. uniform or Gaussian], and a constant c such that c(Qx) is an upper bound for P*(x)
Example with Q(x) uniformgenerate uniform random samples in upper bound volumeupper bound
cQ(x)
P*(x)
accept samples that fallbelow the P*(x) curve
the marginal density of thex coordinates of the pointsis then proportional to P*(x)
Note the relationship toMonte Carlo integration.
![Page 20: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/20.jpg)
CSE586, PSURobert Collins
Rejection Sampling
More generally:
1) generate sample xi from a proposal density Q(x)
2) generate sample u from uniform [0,cQ(xi)]
3) if u <= P*(xi) accept xi; else reject
Q(x)
cQ(xi)
P*(xi)
P*(x)cQ(x)
xiuaccept!
![Page 21: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/21.jpg)
CSE586, PSURobert Collins
Importance “Sampling”
Not for generating samples. It is a method to estimate the expected value of a function f(xi) directly
1) Generate xi from Q(x)
2) an empirical estimate of EQ(f(x)), the expected value of f(x) under distribution Q(x), is then
3) However, we want EP(f(x)), which is the expected value of f(x) under distribution P(x) = P*(x)/Z
![Page 22: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/22.jpg)
CSE586, PSURobert Collins
Importance Sampling
Q(x)
P*(x)
When we generate from Q(x), values of x where Q(x) is greater than P*(x) are overrepresented, and values where Q(x) is less than P*(x) are underrepresented.
To mitigate this effect, introduce a weighting term
![Page 23: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/23.jpg)
CSE586, PSURobert Collins
Importance Sampling
New procedure to estimate EP(f(x)):
1) Generate N samples xi from Q(x)
2) form importance weights
3) compute empirical estimate of EP(f(x)), the expected value of f(x) under distribution P(x), as
![Page 24: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/24.jpg)
CSE586, PSURobert Collins
Resampling
Note: We thus have a set of weighted samples (xi, wi | i=1,…,N)
If we really need random samples from P, we can generate them by resampling such that the likelihood of choosing value xi is proportional to its weight wi
This would now involve now sampling from a discrete distribution of N possible values (the N values of xi )
Therefore, regardless of the dimensionality of vector x, we are resampling from a 1D distribution (we are essentially sampling from the indices 1...N, in proportion to the importance weights wi). So we can using the inverse transform sampling method we discussed earlier.
![Page 25: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/25.jpg)
CSE586, PSURobert Collins
Note on Proposal Functions
Computational efficiency is best if the proposal distribution looks a lot like the desired distribution (area between curves is small).
These methods can fail badly when the proposal distribution has 0 density in a region where the desired distribution has non-negligeable density.
For this last reason, it is said that the proposal distribution should have heavy tails.
![Page 26: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/26.jpg)
CSE586, PSURobert Collins
Sequential Monte Carlo Methods
Sequential Importance Sampling (SIS) and the closely related algorithm Sampling Importance Sampling (SIR) are known by various names in the literature:
- bootstrap filtering- particle filtering- Condensation algorithm- survival of the fittest
General idea: Importance sampling on time series data, with samples and weights updated as each new data term is observed. Well-suited for simulating Markov chains and HMMs!
![Page 27: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/27.jpg)
CSE586, PSURobert Collins
Markov-Chain Monte Carlo
![Page 28: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/28.jpg)
CSE586, PSURobert Collins
References
![Page 29: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/29.jpg)
CSE586, PSURobert Collins
Recall: Sampling Motivation
If we can generate random samples xi from a given distribution P(x), then we can estimate expected values of functions under this distribution by summation, rather than integration.
That is, we can approximate:
by first generating N i.i.d. samples from P(x) and then forming the empirical estimate:
![Page 30: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/30.jpg)
CSE586, PSURobert Collins
Problem
Intuition: In high dimension problems, the “Typical Set” (volume of nonnegligable prob in state space) is a small fraction of the total space.
![Page 31: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/31.jpg)
CSE586, PSURobert Collins
High-Dimensional Spaces
http://www.cs.rpi.edu/~zaki/Courses/dmcourse/Fall09/notes/highdim.pdf
consider ratio of volumes of hypersphere inscribed inside hypercube
2D 3D
Asymptotic behavior:
most of volume of the hypercube lies outside of hypersphere as dimension d increases
d
![Page 32: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/32.jpg)
CSE586, PSURobert Collins
High Dimensional Spaces
each pixel has twostates: on and off
![Page 33: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/33.jpg)
CSE586, PSURobert Collins
, but...
![Page 34: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/34.jpg)
CSE586, PSURobert Collins
ignores neighborhoodstructure of pixel latticeand empirical evidence that images are “smooth”
![Page 35: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/35.jpg)
CSE586, PSURobert Collins
![Page 36: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/36.jpg)
CSE586, PSURobert Collins
Recall: Markov Chains
Markov Chain:
• A sequence of random variables X1,X2,X3,...
• Each variable is a distribution over a set of states (a,b,c...)
• Transition probability of going to next state only depends on the current state. e.g. P(Xn+1 = a | Xn = b)
a
b
c
d
transition probs can be arranged in an NxN table of elements
kij = P(Xn+1=j | Xn = i)
where the rows sum to one
![Page 37: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/37.jpg)
CSE586, PSURobert Collins
General Idea: MCMC Sampling
Start in some state, and then run the simulation for some number of time steps. After you have run it “long enough”start keeping track of the states you visit.
{... X1 X2 X1 X3 X3 X2 X1 X2 X1 X1 X3 X3 X2 ...}
These are samples from the distribution you want, so you can now compute any expected values with respect to that distribution empirically.
![Page 38: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/38.jpg)
CSE586, PSURobert Collins
The Theory Behind MCMC Sampling
If the Markov chain is positive recurrent, there exists a stationary distribution. If it is positive recurrent and irreducible, there exists a unique stationary distribution. Then, the average of a function f over samples of the Markov chain is equal to the average with respect to the stationary distribution
expected return time to every state is finiteevery state is accessible fromevery other state.
We can compute this empirically aswe generate samples.
This is what we want to compute,and is infeasible to compute inany other way.
![Page 39: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/39.jpg)
CSE586, PSURobert Collins
K= transpose of transitionprob table {k ij} (cols sum to one. We do this for computational convenience (next slide)
![Page 40: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/40.jpg)
CSE586, PSURobert Collins
Question:
Assume you start in some state, and then run the simulation for a large number of time steps. What percentage of time do you spend at X1, X2 and X3?
![Page 41: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/41.jpg)
CSE586, PSURobert Collins
four possible initial distributions
[.33 .33 .33]
initial distribution
distribution after one time step
all eventually end up with same distribution -- this is the stationary distribution!
![Page 42: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/42.jpg)
CSE586, PSURobert Collins
in matlab:[E,D] = eigs(K)
(Perron-Frobenius theorem)
![Page 43: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/43.jpg)
CSE586, PSURobert Collins
The PageRank of a webpage as used by Google is defined by a Markov chain. It is the probability to be at page i in the stationary distribution on the following Markov chain onall (known) webpages. If N is the number of known webpages, and a page i has ki links then it has transition probability (1-q)/ki + q/N for all pages that are linked to and q/N for all pages that are not linked to. The parameter q is taken to be about 0.15.
![Page 44: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/44.jpg)
CSE586, PSURobert Collins
![Page 45: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/45.jpg)
CSE586, PSURobert Collins
Another Question:
Assume you want to spend a particular percentage of time at X1, X2 and X3. What should the transition probabilities be?
X1
X2X3
P(x1) = .2P(x2) = .3P(x3) = .5 K = [ ? ? ?
? ? ?? ? ? ]
![Page 46: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/46.jpg)
CSE586, PSURobert Collins
Thought ExperimentConsider only two states. What transition probabilities should we use so that we spend roughly equal time in each of the two states? (i.e. 50% of the time we are in state 1 and 50% of the time we are in state 2)
1 2
?
?
??
![Page 47: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/47.jpg)
CSE586, PSURobert Collins
Detailed Balance
• Consider a pair of configuration nodes r,s
• Want to generate them with frequency relative to their likelihoods L(r) and L(s)
• Let q(r,s) be relative frequency of proposing configuration s when the current state is r (and vice versa)
r
s
L(r)
L(s)
q(r,s)
q(s,r)
A sufficient condition to generate r,s with the desired frequency is
L(r) q(r,s) = L(s) q(s,r)
“detailed balance”
![Page 48: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/48.jpg)
CSE586, PSURobert Collins
Detailed Balance• In practice, you just propose some transition probabilities.
• They typically will NOT satisfy detailed balance (unless you are extremely lucky).
• Instead, you “fix them” by introducing a computational fudge factor
r
s
L(r)
L(s)
a * q(r,s)
Detailed balance:
a* L(r) q(r,s) = L(s) q(s,r)
Solve for a:
a = L(s) q(s,r)L(r) q(r,s)
q(s,r)
![Page 49: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/49.jpg)
CSE586, PSURobert Collins
diff with rejection sampling: instead ofthrowing away rejections, you replicatethem into next time step.
Note: you can just make uptransition probability q on-the-fly, using whatevercriteria you wish.
![Page 50: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/50.jpg)
CSE586, PSURobert Collins
Note: the transition probabilities q(x(t),x) can be arbitrary distributions. They can depend on the current state and change at every time step, if you want.
![Page 51: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/51.jpg)
CSE586, PSURobert Collins
Metropolis Hastings Example
X1
X2X3
P(x1) = .2P(x2) = .3P(x3) = .5
Proposal distributionq(xi, (xi-1)mod3 ) = .4q(xi, (xi+1)mod3) = .6
Matlab demo
![Page 52: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/52.jpg)
CSE586, PSURobert Collins
Variants of MCMC
• there are many variations on this general approach, some derived as special cases of the Metropolis-Hastings algorithm
![Page 53: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/53.jpg)
CSE586, PSURobert Collins
q(x’,x)q(x, x’)
cancels
e.g. Gaussian
![Page 54: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/54.jpg)
CSE586, PSURobert Collins
simpler version, using 1D conditional distributions
or line search, or ...
![Page 55: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/55.jpg)
CSE586, PSURobert Collins
1D conditional distr
1D conditional distr interleave
![Page 56: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/56.jpg)
CSE586, PSURobert Collins
Gibbs SamplerSpecial case of MH with acceptance ratio always 1 (so you always accept the proposal).
S.Brooks, “Markov Chain Monte Carlo and its Application”
where
![Page 57: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/57.jpg)
CSE586, PSURobert Collins
Simulated Annealing
• introduce a “temperature” term that makes it more likely to accept proposals early on. This leads to more aggressive exploration of the state space.
• Gradually reduce the temperature, causing the process to spend more time exploring high likelihood states.
• Rather than remember all states visited, keep track of the best state you’ve seen so far. This is a method that attempts to find the global max (MAP) state.
![Page 58: Intro to Sampling Methods - cse.psu.edurtc12/CSE586/lectures/cse586samplingIntro.pdf · CSE586, PSU Robert Collins Note on Proposal Functions Computational efficiency is best if the](https://reader036.fdocuments.net/reader036/viewer/2022081402/6079f1125d69b03e98444f98/html5/thumbnails/58.jpg)
CSE586, PSURobert Collins
Trans-dimensional MCMC
• Exploring alternative state spaces of differing dimensions (example, when doing EM, also try to estimate number of clusters along with parameters of each cluster).
• Green’s reversible-jump approach (RJMCMC) gives a general template for exploring and comparing states of differing dimension.