Post on 13-Dec-2015
Outline
Introduction of the Stick-Breaking process
Presentation of fundamental representation The Dirichlet process The Pitman-Yor process The Indian buffet process
Outline
Introduction of the Stick-Breaking process
Presentation of fundamental representation The Dirichlet process The Pitman-Yor process The Indian buffet process
Definition of the Beta process
Outline
Introduction of the Stick-Breaking process
Presentation of fundamental representation The Dirichlet process The Pitman-Yor process The Indian buffet process
Definition of the Beta process A Stick-Breaking construction of Beta
process
Outline
Introduction of the Stick-Breaking process
Presentation of fundamental representation The Dirichlet process The Pitman-Yor process The Indian buffet process
Definition of the Beta process A Stick-Breaking construction of Beta
process Conclusion and current work
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining
stick is broken by sampling the proportion to cut
The Stick-Breaking process
Assume a stick of unit length At each iteration, a part of the remaining stick
is broken by sampling the proportion to cut How should we sample these proportions?
Beta random proportions
Let be the proportion to cut at iteration
The remaining length can be expressed as
Beta random proportions
Let be the proportion to cut at iteration
The remaining length can be expressed as
Thus, the broken part is defined by
Beta random proportions
Let be the proportion to cut at iteration
The remaining length can be expressed as
Thus, the broken part is defined by
We first consider the case where
The Dirichlet process
Dirichlet processes are often used to produce infinite mixture models
Each observation belongs to one of the infinitely many components
The Dirichlet process
Dirichlet processes are often used to produce infinite mixture models
Each observation belongs to one of the infinitely many components
The model ensures that only a finite number of components have appreciable weight
The Dirichlet process
A Dirichlet process, , can be constructed according to a Stick-Breaking process
Where is the base distribution and is a unit mass at
The Pitman-Yor process
A Pitman-Yor process, , can be constructed according to a Stick-Breaking process
Where and
Evolution of the Beta cuts
The parameter controls the speed at which the Beta distribution changes
The parameter determines initial shapes of the Beta distribution
Evolution of the Beta cuts
The parameter controls the speed at which the Beta distribution changes
The parameter determines initial shapes of the Beta distribution
When , there is no changes over time and its called a Dirichlet process
Evolution of the Beta cuts
The parameter controls the speed at which the Beta distribution changes
The parameter determines initial shapes of the Beta distribution
When , there is no changes over time and its called a Dirichlet process
MATLAB DEMO
The Indian Buffet process
The Indian Buffet process was initially used to represent latent features
Observations are generated according to a set of unknown hidden features
The Indian Buffet process
The Indian Buffet process was initially used to represent latent features
Observations are generated according to a set of unknown hidden features
The model ensure that only a finite number of features have appreciable probability
The Indian Buffet process
Recall the basic Stick-Breaking process Here, we only consider the remaining
parts
The Indian Buffet process
Recall the basic Stick-Breaking process Here, we only consider the remaining
parts
The Indian Buffet process
Recall the basic Stick-Breaking process Here, we only consider the remaining
parts Each value corresponds to a feature
probability of appearance
Summary
The Dirichlet process induces a probability over infinitely many classes
This is the underlying de Finetti mixing distribution of the Chinese restaurant process
De Finetti theorem
It states that the distribution of any infinitely exchangeable sequence can be written
where is the de Finetti mixing distribution
Summary
The Dirichlet process induces a probability over infinitely many classes
This is the underlying de Finetti mixing distribution of the Chinese restaurant process
The Indian Buffet process induces a probability over infinitely many features
Summary
The Dirichlet process induces a probability over infinitely many classes
This is the underlying de Finetti mixing distribution of the Chinese restaurant process
The Indian Buffet process induces a probability over infinitely many features
Its underlying de Finetti mixing distribution is the Beta process
Beta with Stick-Breaking
The Beta distribution has a Stick-Breaking representation which allows to sample from
Beta with Stick-Breaking
The Beta distribution has a Stick-Breaking representation which allows to sample from
The construction is
Beta with Stick-Breaking
The Beta distribution has a Stick-Breaking representation which allows to sample from
The construction is
Conclusion
We briefly described various Stick-Breaking constructions for Bayesian nonparametric priors
These constructions help to understand the properties of each process
It also unveils connections among existing priors
The Stick-Breaking process might help to construct new priors