Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.
-
Upload
dwayne-george -
Category
Documents
-
view
217 -
download
0
Transcript of Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.
![Page 1: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/1.jpg)
Bayesian Generalized Kernel Mixed Models
Zhihua Zhang, Guang Dai and Michael I. Jordan
JMLR 2011
![Page 2: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/2.jpg)
Summary of contributions
• Propose generalized kernel models (GKMs) as a framework in which sparsity can be given an explicit treatment and in which a fully Bayesian methodology can be carried out
• Data augmentation methodology to develop a MCMC algorithm for inference
• Approach shown to be related Gaussian processes and provide a flexible approximation method for GPs
![Page 3: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/3.jpg)
Bayesian approach for kernel supervised learning
• The form of the regressor or classifier is given by
• For a Mercer kernel, there exists a corresponding mapping (say ), from the input space , such that
• This provides an equivalent representation in the feature space, where,
![Page 4: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/4.jpg)
Generalized Kernel Models
![Page 5: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/5.jpg)
Prior for regression coefficients
![Page 6: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/6.jpg)
Sparse models
• Recall that the number of active vectors is the number of non-zero components of– We are thus interested in a prior for which
allows some components of to be zero
![Page 7: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/7.jpg)
Methodology
For the indicator vector
![Page 8: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/8.jpg)
Graphical model
![Page 9: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/9.jpg)
Inference
• Gibbs for most parameters• MH for kernel parameters• Reversible jump Markov Chain for – takes 2^n distinct values– For small n, posterior may be obtained by calculating
the normalizing constant by summing over all possible values of
– For large n, a reversible jump MC sampler may be employed to identify high posterior probability models
![Page 10: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/10.jpg)
Automatic choice of active vectors
• We generate a proposal from the current value of by one of the three possible moves:
Prediction :
![Page 11: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/11.jpg)
Sparse Gaussian process for classification
Given a function , then is a Gaussian process with zero mean and covariance function and vice versa.
Also,
![Page 12: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/12.jpg)
Sparse GP classification
![Page 13: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/13.jpg)
Results
![Page 14: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/14.jpg)
![Page 15: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/15.jpg)
![Page 16: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/16.jpg)
![Page 17: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/17.jpg)
![Page 18: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/18.jpg)
![Page 19: Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649f445503460f94c65986/html5/thumbnails/19.jpg)