Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep...
Transcript of Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep...
![Page 1: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/1.jpg)
Geometry and Uncertainty in Deep Learning
for Computer Vision
Alex Kendall, University of Cambridge, March 2017
@alexgkendall alexgkendall.com [email protected]
![Page 2: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/2.jpg)
Why is uncertainty important?
![Page 3: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/3.jpg)
Bayesian SegNet for probabilistic scene understanding
Input Image Semantic Segmentation Uncertainty
![Page 4: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/4.jpg)
1. What uncertainty can we model with deep learning and what
are the benefits?
2. How do we model uncertainty using Bayesian deep learning
for regression and classification tasks?
3. Why should we formulate deep learning models for vision
which leverage our knowledge of geometry?
Outline of Talk
![Page 5: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/5.jpg)
Uncertainty
![Page 6: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/6.jpg)
Epistemic uncertainty
• Measures what you’re model doesn’t know
• Can be explained away by unlimited data
Aleatoric uncertainty
• Measures what you can’t understand from the data
• Can be explained away by unlimited sensing
What kind of uncertainty can we model?
1
2
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
![Page 7: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/7.jpg)
Epistemic uncertainty is modeling uncertainty
Aleatoric uncertainty is sensing uncertainty
What kind of uncertainty can we model?
![Page 8: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/8.jpg)
Deep learning is required to achieve state of the art results in computer vision
applications but doesn’t provide uncertainty estimates.
• Bayesian neural networks are a framework for understanding uncertainty in deep learning
• They have distributions over network parameters (rather than deterministic weights)
• Traditionally they have been tricky to scale
Modeling Uncertainty with Bayesian Deep Learning
MacKay, David JC. A practical Bayesian framework for backpropagation networks. Neural Computation, 4(3): 448–472, 1992.
?
![Page 9: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/9.jpg)
We can model epistemic uncertainty in deep learning models using
Monte Carlo dropout sampling at test time.
Dropout sampling can be interpreted as sampling from a distribution over models.
Modeling Epistemic Uncertainty with Bayesian Deep Learning
Alex Kendall, Vijay Badrinarayanan and Roberto Cipolla Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for
Scene Understanding. arXiv preprint arXiv:1511.02680, 2015.
![Page 10: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/10.jpg)
Modeling Aleatoric Uncertainty with Probabilistic Deep Learning
ො𝑦, ො𝜎2 = 𝑓(𝑥)
𝐿𝑜𝑠𝑠 = 𝑦 − ො𝑦 2
ො𝑦t = ො𝑦 + 𝜖𝑡 𝜖𝑡 ∼ 𝑁(0, ො𝜎2)
𝐿𝑜𝑠𝑠 =1
𝑇
𝑡
𝑆𝑜𝑓𝑡𝑚𝑎𝑥𝐶𝑟𝑜𝑠𝑠𝐸𝑛𝑡𝑟𝑜𝑝𝑦(ො𝑦t)
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
ො𝑦 = 𝑓(𝑥)
𝐿𝑜𝑠𝑠 =𝑦 − ො𝑦 2
2 ො𝜎2+ log ො𝜎2
𝐿𝑜𝑠𝑠 = 𝑆𝑜𝑓𝑡𝑚𝑎𝑥𝐶𝑟𝑜𝑠𝑠𝐸𝑛𝑡𝑟𝑜𝑝𝑦(ො𝑦t)
Deep Learning Probabilistic Deep Learning
Model
Regression
Classification
![Page 11: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/11.jpg)
Semantic Segmentation Performance on CamVid
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
CamVid Results IoU Accuracy
DenseNet (State of the art baseline) 67.1
+ Aleatoric Uncertainty 67.4
+ Epistemic Uncertainty 67.2
+ Aleatoric & Epistemic 67.5
![Page 12: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/12.jpg)
Monocular Depth Regression Performance
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
NYU Depth Results Rel. Error
DenseNet (State of the art baseline) 0.167
+ Aleatoric Uncertainty 0.149
+ Epistemic Uncertainty 0.162
+ Aleatoric & Epistemic 0.145
![Page 13: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/13.jpg)
![Page 14: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/14.jpg)
Aleatoric vs. Epistemic Uncertainty for Out of Dataset Examples
Aleatoric uncertainty remains constant while epistemic
uncertainty increases for out of dataset examples!!
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
![Page 15: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/15.jpg)
• One reason why computer vision has progressed so rapidly is because we can
benchmark and compare algorithms easily
• Often leaderboards rank prediction accuracy and algorithm speed
• Leaderboards should also rank algorithms probabilistically and quantify
uncertainty accuracy
Uncertainty Benchmarks
![Page 16: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/16.jpg)
Calibration Plots
• For a prediction with probability 𝑝, the model should be correct with a frequency of 𝑝
• Perfect calibration corresponds to the line, 𝑦 = 𝑥
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
![Page 17: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/17.jpg)
Precision Recall Plots
• Uncertainty should correlate well with accuracy
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
![Page 18: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/18.jpg)
Putting it all Together: Multi-Task Learning
![Page 19: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/19.jpg)
We want to simultaneously learn multiple tasks: 𝐿𝑜𝑠𝑠 = σ𝑖𝑤𝑖𝐿𝑖
Task performance is very sensitive to choice of weights, so how do you choose??
Multitask Learning
Scene Understanding [1] Localisation [2]
[1] Alex Kendall, Yarin Gal and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arxiv preprint 1705.07115, 2017.
[2] Alex Kendall, Matthew Grimes and Roberto Cipolla PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. ICCV, 2015.
![Page 20: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/20.jpg)
Heteroscedastic aleatoric uncertainty
• Data dependent aleatoric uncertainty
Homoscedastic aleatoric uncertainty
• Aleatoric uncertainty which doesn’t depend on the data
• Task uncertainty
Types of Aleatoric Uncertainty
1
2
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
![Page 21: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/21.jpg)
Homoscedastic uncertainty, σ2, captures uncertainty of the entire task itself – not dependant on input data.
We propose to use this to learn a weighting for each loss term.
Combine Losses Using Homoscedastic Uncertainty
𝐿𝑜𝑠𝑠 =𝐿𝑟𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 1
𝜎12 + log𝜎1
2 +𝐿𝑟𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 2
𝜎22 + log𝜎2
2 + 𝑆𝑜𝑓𝑡𝑚𝑎𝑥𝐶𝑟𝑜𝑠𝑠𝐸𝑛𝑡𝑟𝑜𝑝𝑦𝑦
𝜎32
Depth Regression Instance Regression Semantic Segmentation
Regularizes homoscedastic
uncertainty from going to
infinity
Learnt uncertainty
tempers each loss term
[1] Alex Kendall, Yarin Gal and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arxiv preprint 1705.07115, 2017.
![Page 22: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/22.jpg)
Multi Task Scene Understanding Model
[1] Alex Kendall, Yarin Gal and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arxiv preprint 1705.07115, 2017.
![Page 23: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/23.jpg)
• Homoscedastic uncertainty can learn the optimal weighting
• Multitask learning can improve performance compared with training separate models for each
individual task
Multitask Learning Results
[1] Alex Kendall, Yarin Gal and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arxiv preprint 1705.07115, 2017.
![Page 24: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/24.jpg)
Qualitative Multitask Learning Results
[1] Alex Kendall, Yarin Gal and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arxiv preprint 1705.07115, 2017.
![Page 25: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/25.jpg)
Geometry
![Page 26: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/26.jpg)
• Geometry was once the most exciting topic in computer vision
• Now machine learning models are the solution to most tasks
• These black boxes can learn many representations with end-to-
end supervised learning
• Often naïve architectures are used
Geometry in Computer Vision?
![Page 27: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/27.jpg)
• However, geometry provides a rich source of training data
• Motion, pose and depth can be leveraged for supervised and
unsupervised training
• Geometric priors and architectural designs can
significantly improve model performance
Geometry in Computer Vision?
![Page 28: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/28.jpg)
PoseNet: trained end-to-end to regress camera
position, x and orientation, q
𝑙𝑜𝑠𝑠 𝐼 = ො𝑥 − 𝑥 2 + 𝛽 ො𝑞 −𝑞
𝑞2
Naïve deep learning approach to learning camera pose
Alex Kendall, Matthew Grimes and Roberto Cipolla. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. ICCV, 2015.
![Page 29: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/29.jpg)
training data in green, test data in blue, PoseNet results in red
Camera Pose Regression
Alex Kendall, Matthew Grimes and Roberto Cipolla. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. ICCV, 2015.
![Page 30: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/30.jpg)
Train with reprojection loss of 3-D geometry
with predicted and ground truth camera poses.
Where 𝜋 is the projection function of 3-D point 𝑔𝑖
Learning camera pose, with geometry
Alex Kendall and Roberto Cipolla. Geometric loss functions for camera pose regression with deep learning. CVPR, 2017 (to appear).
![Page 31: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/31.jpg)
Camera Pose Regression
Alex Kendall and Roberto Cipolla. Geometric loss functions for camera pose regression with deep learning. CVPR, 2017 (to appear).
Using geometry in our model structure improves performance
![Page 32: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/32.jpg)
Epistemic uncertainty to estimate loop closure
Alex Kendall and Roberto Cipolla. Modelling Uncertainty in Deep Learning for Camera Relocalization. ICRA, 2016.
Increased uncertainty from strong occlusion, motion blur, visually ambiguous landmarks
Determine if the
model has seen the
landmark before
(loop closure)
We can use epistemic uncertainty to estimate metric relocalisation error
![Page 33: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/33.jpg)
End to end deep learning for stereo vision
Alex Kendall et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint 1703.04309, 2017.
• Form differentiable cost volume and sub-pixel regression network with soft argmax function
• Use 3-D convolutions to learn to regularise the volume
![Page 34: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/34.jpg)
Soft ArgMin / ArgMax
Alex Kendall et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint 1703.04309, 2017.
Loss > 3px Error
Classification Loss 12.2
Soft Classification 12.3
Regression 9.34
![Page 35: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/35.jpg)
Scene Flow Dataset Results
Alex Kendall et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint 1703.04309, 2017.
![Page 36: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/36.jpg)
Probabilistic Deep Learning for Stereo Vision
Alex Kendall et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint 1703.04309, 2017.
Alex Kendall and Roberto Cipolla. Uncertainty and Unsupervised Learning for Stereo Vision with Probabilistic Deep Learning. Under Review, 2017.
Depth Prediction Depth Prediction Uncertainty
Input Left Image Input Right Image
![Page 37: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/37.jpg)
1st Place on the 2012 & 2015 KITTI Stereo Challenge
Alex Kendall et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint 1703.04309, 2017.
![Page 38: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/38.jpg)
Autonomous Drone Prototype
Skydio Inc. http://www.skydio.com/
![Page 39: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/39.jpg)
Aleatoric uncertainty is important for
• Large data situations, where epistemic uncertainty is explained away,
• Real-time applications, because we can form aleatoric models without expensive Monte Carlo
samples,
• Multitask applications, because we can appropriately weight each loss.
Epistemic uncertainty is important for
• Safety-critical applications, because epistemic uncertainty is required to understand examples which
are different from training data,
• Small datasets, where the training data is sparse,
• Exploratory applications, such as loop closure and reinforcement learning.
Conclusions
1
2
![Page 40: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/40.jpg)
It is important to quantify the accuracy of uncertainty
estimates
We should leverage our knowledge of geometry when
designing machine learning models for computer vision
• Reprojection loss
• Stereo cost volume
Conclusions
3
4
![Page 41: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/41.jpg)
CVPR Tutorial
Hawaii
July 26th 2017
See you there?
![Page 42: Geometry and Uncertainty in Deep Learning · 2017-05-22 · Geometry and Uncertainty in Deep Learning for Computer Vision Alex Kendall, University of Cambridge, March 2017 @alexgkendall](https://reader034.fdocuments.net/reader034/viewer/2022042202/5ea257cff98dd6603c40baf7/html5/thumbnails/42.jpg)
• Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint 1703.04977, 2017.
• Alex Kendall and Roberto Cipolla. Geometric loss functions for camera pose regression with deep learning. CVPR, 2017 (to appear).
• Alex Kendall et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint 1703.04309, 2017.
• Alex Kendall and Roberto Cipolla. Uncertainty and Unsupervised Learning for Stereo Vision with Probabilistic Deep Learning. Under Review, 2017.
• Alex Kendall et al. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arxiv preprint 1705.07115, 2017.
• Vijay Badrinarayanan, Alex Kendall and Roberto Cipolla. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. PAMI 2017.
• Alex Kendall and Roberto Cipolla. Modelling Uncertainty in Deep Learning for Camera Relocalization. ICRA, 2016.
• Alex Kendall et al. Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding. arXiv 1511.02680, 2015.
• Alex Kendall, Matthew Grimes and Roberto Cipolla. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. ICCV, 2015.
Thank You & References