Disparity of access_quality_review_of_maternal_mortality_in_5_regions_in_indonesia
Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with...
Transcript of Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with...
![Page 1: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/1.jpg)
![Page 2: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/2.jpg)
![Page 4: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/4.jpg)
![Page 5: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/5.jpg)
●●●
![Page 7: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/7.jpg)
ROI-10D: Monocular Lifting of2D Detection to 6D Pose and Metric Shape
F Manhardt, W Kehl, A Gaidonhttps://arxiv.org/abs/1812.02781
Learning to Fuse Things and Stuff
J Li, A Raventos, A Bhargava, T Tagawa, A Gaidonhttps://arxiv.org/abs/1812.01192
![Page 8: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/8.jpg)
Credit: Ed Olson, May Mobility
![Page 11: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/11.jpg)
●●●
![Page 12: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/12.jpg)
●○○
●○○
■■
Toyota Safety Sense 2.0 Camera
![Page 13: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/13.jpg)
ICRA 2019 [arxiv + video]
![Page 14: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/14.jpg)
Easy to acquire
Expensive / Difficult to acquire
![Page 15: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/15.jpg)
Easy to acquire
![Page 16: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/16.jpg)
![Page 17: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/17.jpg)
![Page 18: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/18.jpg)
18
Photometric lossvia view-synthesis
Occlusion Regularization
Depth Regularization(edge-aware depth smoothing)
Depth Model Parameters
![Page 19: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/19.jpg)
●● →
Resolution Matters for View Synthesis!
![Page 20: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/20.jpg)
A. Odena, V. Dumoulin, and C. Olah, “Deconvolution and checkerboard artifacts,” Distill, vol. 1, no. 10, p. e3, 2016.W. Shi, J. Caballero, F. Husza ́r, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super- resolution
using an efficient sub-pixel convolutional neural network,” CVPR 2016
●○○
![Page 21: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/21.jpg)
●○
Modified DispNet Architecture
![Page 22: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/22.jpg)
M. Jaderberg, K. Simonyan, A. Zisserman, et al., “Spatial transformer networks,” NIPS 2015 C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017
Left Disparity
Flipped LeftDisparity
Priors learned by model due to occluded boundariesin fronto-parallel stereo case
Spatial Transformer
Network
Fused left
●○○
![Page 23: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/23.jpg)
Sub-pixel convolutions (SP), Differentiable Flip Augmentation (FA)
![Page 24: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/24.jpg)
![Page 25: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/25.jpg)
![Page 26: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/26.jpg)
![Page 27: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/27.jpg)
●●●
![Page 30: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/30.jpg)
30
Gaidon et al, "Virtual worlds as proxy for multiobject tracking analysis.", CVPR'16
Ros et al, "The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes", CVPR'16
de Souza et al, "Procedural Generation of Videos to Train Deep Action Recognition Networks.", CVPR'17
![Page 31: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/31.jpg)
31
![Page 32: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/32.jpg)
32
adversarial loss
privileged regularization
perceptual regularization(self-regularization)
task loss(this is what we care about)
![Page 33: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/33.jpg)
→
33
![Page 34: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/34.jpg)
→
34
![Page 35: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/35.jpg)
→
35
![Page 36: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/36.jpg)
→
36
![Page 37: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/37.jpg)
→
37
![Page 38: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/38.jpg)
38
![Page 39: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/39.jpg)
●
●
●
![Page 40: Nvidia...C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised monocular depth estimation with left-right consistency,” CVPR 2017 Left Disparity Flipped Left Disparity Priors](https://reader033.fdocuments.net/reader033/viewer/2022050103/5f41fb1d4e30c656a8078137/html5/thumbnails/40.jpg)