Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a...
-
Upload
christine-kelly -
Category
Documents
-
view
220 -
download
0
Transcript of Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a...
![Page 1: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/1.jpg)
Feature Matching and RANSAC
15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005
with a lot of slides stolen from Steve Seitz and Rick Szeliski
© Krister Parmstrand
![Page 2: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/2.jpg)
Feature matching
?
![Page 3: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/3.jpg)
Feature matching• Exhaustive search
• for each feature in one image, look at all the other features in the other image(s)
• Hashing• compute a short descriptor from each feature vector, or hash
longer descriptors (randomly)
• Nearest neighbor techniques• k-trees and their variants
![Page 4: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/4.jpg)
What about outliers?
?
![Page 5: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/5.jpg)
Feature-space outlier rejectionLet’s not match all features, but only these that have
“similar enough” matches?
How can we do it? • SSD(patch1,patch2) < threshold• How to set threshold?
![Page 6: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/6.jpg)
Feature-space outlier rejectionA better way [Lowe, 1999]:
• 1-NN: SSD of the closest match• 2-NN: SSD of the second-closest match• Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN• That is, is our best match so much better than the rest?
![Page 7: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/7.jpg)
Feature-space outliner rejection
Can we now compute H from the blue points?• No! Still too many outliers… • What can we do?
![Page 8: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/8.jpg)
Matching features
What do we do about the “bad” matches?
![Page 9: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/9.jpg)
RAndom SAmple Consensus
Select one match, count inliers
![Page 10: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/10.jpg)
RAndom SAmple Consensus
Select one match, count inliers
![Page 11: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/11.jpg)
Least squares fit
Find “average” translation vector
![Page 12: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/12.jpg)
RANSAC for estimating homography
RANSAC loop:
1. Select four feature pairs (at random)
2. Compute homography H (exact)
3. Compute inliers where SSD(pi’, H pi) < ε
4. Keep largest set of inliers
5. Re-compute least-squares H estimate on all of the inliers
![Page 13: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/13.jpg)
RANSAC
![Page 14: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/14.jpg)
Example: Recognising Panoramas
M. Brown and D. Lowe,
University of British Columbia
![Page 15: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/15.jpg)
Why “Recognising Panoramas”?
![Page 16: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/16.jpg)
Why “Recognising Panoramas”?
1D Rotations ()• Ordering matching images
![Page 17: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/17.jpg)
Why “Recognising Panoramas”?
1D Rotations ()• Ordering matching images
![Page 18: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/18.jpg)
Why “Recognising Panoramas”?
1D Rotations ()• Ordering matching images
![Page 19: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/19.jpg)
Why “Recognising Panoramas”?
• 2D Rotations (, )– Ordering matching images
1D Rotations ()• Ordering matching images
![Page 20: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/20.jpg)
Why “Recognising Panoramas”?
1D Rotations ()• Ordering matching images
• 2D Rotations (, )– Ordering matching images
![Page 21: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/21.jpg)
Why “Recognising Panoramas”?
1D Rotations ()• Ordering matching images
• 2D Rotations (, )– Ordering matching images
![Page 22: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/22.jpg)
Why “Recognising Panoramas”?
![Page 23: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/23.jpg)
OverviewFeature Matching
Image Matching
Bundle Adjustment
Multi-band Blending
Results
Conclusions
![Page 24: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/24.jpg)
RANSAC for Homography
![Page 25: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/25.jpg)
RANSAC for Homography
![Page 26: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/26.jpg)
RANSAC for Homography
![Page 27: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/27.jpg)
Probabilistic model for verification
![Page 28: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/28.jpg)
Finding the panoramas
![Page 29: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/29.jpg)
Finding the panoramas
![Page 30: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/30.jpg)
Finding the panoramas
![Page 31: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/31.jpg)
Finding the panoramas
![Page 32: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/32.jpg)
Parameterise each camera by rotation and focal length
This gives pairwise homographies
Homography for Rotation
![Page 33: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/33.jpg)
Bundle AdjustmentNew images initialised with rotation, focal length of best
matching image
![Page 34: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/34.jpg)
Bundle AdjustmentNew images initialised with rotation, focal length of best
matching image
![Page 35: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/35.jpg)
Multi-band BlendingBurt & Adelson 1983
• Blend frequency bands over range
![Page 36: Feature Matching and RANSAC 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649db05503460f94a9efee/html5/thumbnails/36.jpg)
Results