A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision...
-
Upload
monserrat-lindley -
Category
Documents
-
view
228 -
download
0
Transcript of A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision...
![Page 1: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/1.jpg)
A Fast Local Descriptor for Dense Matching
Engin Tola, Vincent Lepetit, Pascal Fua
Computer Vision LaboratoryEPFL
2008-06-10
![Page 2: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/2.jpg)
MotivationNarrow baseline : Pixel Difference + Graph Cuts*
groundtruth
pixel difference
input frame
input frame
* Y. Boykov et al. Fast Approximate Energy Minimization via Graph Cuts. PAMI’01.
![Page 3: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/3.jpg)
MotivationWide baseline : Pixel Difference + Graph Cuts
groundtruth
USE A DESCRIPTOR
input frame
input frame
pixel difference
![Page 4: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/4.jpg)
MotivationWide baseline : SIFT Descriptor*+ Graph Cuts
groundtruth
SIFT
250 Seconds
* D. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. IJCV’04
input frame
input frame
![Page 5: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/5.jpg)
MotivationWide baseline : DAISY Descriptor+ Graph Cuts
groundtruth
DAISY
5 Seconds
input frame
input frame
![Page 6: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/6.jpg)
MotivationHistogram Based Descriptors: SIFT, GLOH, SURF…
- Perspective robustness- Proven good performance- Robustness to many image transformations
Cons
- No efficient implementation exists for dense computation- Do not consider occlusions
Design a descriptor that is as robust as SIFT or GLOH but can be computed much more effectively and handle occlusions.
![Page 7: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/7.jpg)
Problem Definition
epipolar lineepipolar
line
Virtual Camera
Input Frames
![Page 8: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/8.jpg)
descriptor
Histogram based Descriptors…SIFT Computation
…
![Page 9: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/9.jpg)
Histogram based Descriptors…SIFT Computation
![Page 10: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/10.jpg)
SIFT -> DAISY
SIFT
+ Good Performance- Not suitable for
dense computation
![Page 11: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/11.jpg)
SIFT -> DAISY
SIFT Sym.SIFT
+ Gaussian Kernels : Suitable for Dense Computation
GLOH*
+ Good Performance+ Better Localization- Not suitable for
dense computation
+ Good Performance- Not suitable for
dense computation
* K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local Descriptors. PAMI’04.
![Page 12: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/12.jpg)
SIFT -> DAISY
DAISY
+ Suitable for dense computation + Improved performance:*
+ Precise localization+ Rotational Robustness
Sym.SIFT
+ Suitable for Dense Computation
GLOH
+ Good Performance+ Better Localization- Not suitable for
dense computation
* S. Winder and M. Brown. Learning Local Image Descriptors in CVPR’07
![Page 13: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/13.jpg)
DAISY Computation
…
…
…
![Page 14: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/14.jpg)
DAISY Computation
…
…
…
![Page 15: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/15.jpg)
DAISY Computation DAISY : 5sSIFT : 250s
- Rotating the descriptor only involves reordering the histograms. - The computation mostly involves 1D convolutions, which is fast.
![Page 16: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/16.jpg)
Depth Map Estimation
x
NN OZpOZxDpDOZp ),(),|)(()|,( :1:1
Descriptors
Occlusion
Depthmap Evidence Smoothness Prior
Occlusions should be handled explicitly!
![Page 17: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/17.jpg)
Depth Map Estimation
OZxMpxMOZxDpOZxDp mm
mNN ,|)()(,,|)(,|)( :1:1
Evidence
P. of a specific Occlusion Mask
Occlusion Masks
![Page 18: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/18.jpg)
Depth Map Estimation
OZxMpxMOZxDpOZxDp mm
mNN ,|)()(,,|)(,|)( :1:1
Evidence
Occlusion Masks
P. of a specific Occlusion Mask
![Page 19: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/19.jpg)
Experiments
DAISY SIFT
SURF NCCPixel Diff
Laser Scan
Comparing against other Descriptors
![Page 20: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/20.jpg)
100
90
80
70
60
50
40
30
20
10
0
Correct Depth % for Image Pairs
ExperimentsComparison with other Descriptors
DAISY
SIFT
SURF
NCC
PIXEL
![Page 21: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/21.jpg)
100
90
80
70
60
50
40
30
20
10
0
Correct Depth % for Image Pairs
ExperimentsComparison with other Descriptors
DAISY
SIFT
SURF
NCC
PIXEL
Correct Depth % vs Error Threshold
![Page 22: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/22.jpg)
Herz-Jesu Sequence
87.4 % 83.9 % 83.8 %
84.9 % 91.8 % 91.8 %
90.8 %83.2 % 93.5 %
89.4 %80.2 % 90.7 %
Truly Occluded
Missed Depths
Missed Occlusions
![Page 23: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/23.jpg)
Herz-Jesu Sequence
Ground TruthDAISY
![Page 24: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/24.jpg)
Comparison with Strecha’05
Strecha’05: Wide baseline stereo from Multiple Views: A probabilistic Account
Strecha: 3072x2048
![Page 25: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/25.jpg)
Comparison with Strecha’05
Strecha’05: Wide baseline stereo from Multiple Views: A probabilistic Account
768x512
![Page 26: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/26.jpg)
Image TransformsContrast Change
Scale
Blurry Webcam Images
SIFTNCC
![Page 27: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/27.jpg)
Image TransformsContrast Change
Scale
Blurry Webcam Images
DAISYNCC
![Page 28: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/28.jpg)
Conclusion
DAISY:• Efficient descriptor for dense wide baseline matching.• Handles occlusions correctly. • Robust to perspective distortions.• Robust to lighting changes. • Can handle low quality imagery.
Future work:• Image-based rendering from widely spaced cameras. • Object detection and recognition.
![Page 29: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/29.jpg)
DAISY Source Codehttp://cvlab.epfl.ch/software
Stereo Data and Ground Truthhttp://cvlab.epfl.ch/data
C. Strecha et al. On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery, CVPR’08
Source Code & Data
![Page 30: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/30.jpg)
Questions
DAISY Source Codehttp://cvlab.epfl.ch/software
Imageshttp://cvlab.epfl.ch/data
http://cvlab.epfl.ch/~tolaEngin Tola
![Page 31: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/31.jpg)
DAISY Source Codehttp://cvlab.epfl.ch/software
Imageshttp://cvlab.epfl.ch/data
http://cvlab.epfl.ch/~tolaEngin Tola
QUESTIONS ?
![Page 32: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/32.jpg)
Parameter Selection
THQ
=2TH
Q=4
R: 5->30R: 5->30
R: 5->30
THQ
=8
HQ=2 HQ=4 HQ=8
RQ:2->5 RQ:2->5 RQ:2->5
![Page 33: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/33.jpg)
R: 5->30R: 5->30
R: 5->30
THQ
=2TH
Q=4
THQ
=8
HQ=2 HQ=4 HQ=8
RQ:2->5 RQ:2->5 RQ:2->5
Parameter Selection
R: 5->30R: 5->30
R: 5->30
THQ
=2TH
Q=4
THQ
=8
HQ=2 HQ=4 HQ=8
RQ:2->5 RQ:2->5 RQ:2->5
Wide BaselineNarrow Baseline
Max: 87 %> 86 %
V:328R=15, RQ=5,
THQ=8, HQ=8
V:52R=10, RQ=3,
THQ=4, HQ=4
V:104R=10, RQ=3,
THQ=4, HQ=8Max: 78%
V:328R=15, RQ=5,
THQ=8, HQ=8
V:200R=15, RQ=3,
THQ=8, HQ=8
V:104R=10, RQ=3,
THQ=4, HQ=8
> 77%
![Page 34: A Fast Local Descriptor for Dense Matching Engin Tola, Vincent Lepetit, Pascal Fua Computer Vision Laboratory EPFL Engin Tola, Vincent Lepetit, Pascal.](https://reader036.fdocuments.net/reader036/viewer/2022062417/551c4e1f550346b1458b4bc4/html5/thumbnails/34.jpg)
Parameter SelectionWide BaselineNarrow Baseline
R: 5->30R: 5->30
R: 5->30
TQ=2
TQ=4
TQ=8
Q:1->5 Q:1->5 Q:1->5
H=2 H=4 H=8
R: 5->30R: 5->30
R: 5->30
TQ=2
TQ=4
TQ=8
Q:1->5 Q:1->5 Q:1->5
H=2 H=4 H=8
0
100