Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian...
Transcript of Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian...
![Page 1: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/1.jpg)
Bayesian Optimization Meets Bayesian Optimal Stopping
Zhongxiang Dai 1 Haibin Yu 1 Bryan Kian Hsiang Low 1 Patrick Jaillet 2
1 Department of Computer Science, National University of Singapore2 Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology
![Page 2: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/2.jpg)
Choice of hyperparameters• Reason: high computational cost
Bayesian Optimization (BO)• Sample efficiency: requires a small number of function evaluations• Theoretical guarantee: Gaussian Process-Upper Confidence Bound (GP-UCB)
Problem:
Solution:
Machine Learning (ML) models have achieved unprecedented level of performances
![Page 3: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/3.jpg)
• Many ML models require iterative training• e.g. stochastic gradient descent (neural network)
training epochs
valid
atio
n e
rro
r
5 BO iterations
wasted computation
![Page 4: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/4.jpg)
• Many ML models require iterative training• e.g. stochastic gradient descent (neural network)
training epochs
valid
atio
n e
rro
r
5 BO iterations
wasted computation
Can we early-stop unpromisinghyperparameters to save resource?
![Page 5: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/5.jpg)
a principled optimal stopping rule is needed
Bayesian Optimal Stopping (BOS)
• Many ML models require iterative training• e.g. stochastic gradient descent (neural network)
training epochs
valid
atio
n e
rro
r
5 BO iterations
wasted computation
Can we early-stop unpromisinghyperparameters to save resource?
![Page 6: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/6.jpg)
• Contribution• We propose BO-BOS, which unifies BO (GP-UCB) and BOS to achieve epoch-
efficient hyper-parameter tuning
• We show that BO-BOS is asymptotically no-regret and performs effectively in practice
![Page 7: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/7.jpg)
• for 𝑡 = 1,2…• choose the hyperparameter to query by maximizing the GP-UCB acquisition function
The BO-BOS Algorithm
![Page 8: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/8.jpg)
• for 𝑡 = 1,2…• choose the hyperparameter to query by maximizing the GP-UCB acquisition function
• Formulate & solve the BOS problem
The BO-BOS Algorithm
![Page 9: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/9.jpg)
• for 𝑡 = 1,2…• choose the hyperparameter to query by maximizing the GP-UCB acquisition function
• Formulate & solve the BOS problem
• Train the ML model using , early-stop the training if BOS outputs the stopping decision
The BO-BOS Algorithm
BOS outputs the stopping decision if is believed to end up performing worse than the currently found optimum
Stop?
Continue to epoch 𝑛 + 1?training epochs
epoch 𝑛
expected loss of stopping < expected loss of continuing
![Page 10: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/10.jpg)
• for 𝑡 = 1,2…• choose the hyperparameter to query by maximizing the GP-UCB acquisition function
• Formulate & solve the BOS problem
• Train the ML model using , early-stop the training if BOS outputs the stopping decision
• Update Gaussian process belief of the objective function
The BO-BOS Algorithm
BOS outputs the stopping decision if is believed to end up performing worse than the currently found optimum
Stop?
Continue to epoch 𝑛 + 1?training epochs
epoch 𝑛
expected loss of stopping < expected loss of continuing
![Page 11: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/11.jpg)
• Matches the simple regret of GP-UCB,thus asymptotically vanishes
• BO aims to minimize the simple regret:
• Asymptotically vanish if the BOS parameters are chosen with consideration of the exploration vs. exploitation trade-off
expectation over the uncertainty introduced by BOS
Theoretical Analysis
![Page 12: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/12.jpg)
• Hyperparameter Tuning• Convolutional Neural Networks
CIFAR-10 dataset SVHN dataset
Experiments
![Page 13: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/13.jpg)
• Beyond Hyperparameter Tuning:• Policy search for Reinforcement Learning (Swimmer-v2 from OpenAI, Mujoco)• Joint hyperparameter tuning and feature selection
Experiments
![Page 14: Bayesian Optimization Meets Bayesian Optimal Stopping12-14-00)-12... · 2019-06-07 · Bayesian Optimization Meets Bayesian Optimal Stopping Zhongxiang Dai 1Haibin Yu Bryan Kian Hsiang](https://reader034.fdocuments.net/reader034/viewer/2022042313/5edf2b12ad6a402d666a84a6/html5/thumbnails/14.jpg)
Thank you!
For more information, please visit our poster at Pacific Ballroom #228