11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.
-
date post
21-Dec-2015 -
Category
Documents
-
view
232 -
download
1
Transcript of 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.
![Page 1: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/1.jpg)
11
Scale Invariant Feature Transform (SIFT)
David G. Lowe
University of British Columbia
![Page 2: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/2.jpg)
2
References
[1] D.G. Lowe, “Object recognition from local scale-invariant features,” in Proc. Seventh IEEE Int’l Conf. Computer Vision, vol. 2, pp. 1150 -1157, 1999.
[2] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int’l J. Computer Vision, vol. 2, no. 60, pp. 91-110, 2004.
2
![Page 3: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/3.jpg)
33
Outline
• Introduction• Algorithm• Applications• Conclusions
![Page 4: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/4.jpg)
44
Outline
• Introduction• Algorithm• Applications• Conclusions
![Page 5: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/5.jpg)
55
An Example: Build a Panorama
![Page 6: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/6.jpg)
66
How Do We Build Panorama?
• We need to match (align) images
![Page 7: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/7.jpg)
77
Matching with Features
• Detect feature points in both images
• Find corresponding pairs
• Use these pairs to align images
![Page 8: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/8.jpg)
8
An Example: Harris Detector
• It is non-invariant to image scale
8
All points will be classified as edges
Corner
![Page 9: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/9.jpg)
99
Scale Invariant Detection
• Consider regions (e.g. circles) of different sizes around a point
• Regions of corresponding sizes will look the same in both images
![Page 10: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/10.jpg)
1010
Scale Invariant Detection
• The problem: how do we choose corresponding circles independently in each image?
![Page 11: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/11.jpg)
11
Laplacian of Gaussian Operator
• Maxima of Laplacian gives best notion of scale:
• Characteristic scale of a local structure
scale
bad
scale
Good !
![Page 12: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/12.jpg)
1212
Introduction
• Generates image features, “keypoints”– invariant to image scaling and rotation– partially invariant to change in illumination
and 3D camera viewpoint– many can be extracted from typical images– highly distinctive
![Page 13: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/13.jpg)
1313
Outline
• Introduction• Algorithm• Applications• Conclusions
![Page 14: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/14.jpg)
1414
Algorithm
• Scale-space extrema detection• Keypoint localization• Orientation assignment• Keypoint descriptor
( )local descriptor
detector
descriptor
![Page 15: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/15.jpg)
1515
Outline
• Introduction• Algorithm
– Scale-space extrema detection– Keypoint localization– Orientation assignment– Keypoint descriptor
• Applications• Conclusions
![Page 16: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/16.jpg)
1616
Detection of Scale-space Extrema
• Detect locations that are invariant to scale change of the image
• SIFT uses DoG filter for scale space because it is efficient and as stable as scale-normalized Laplacian of Gaussian
![Page 17: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/17.jpg)
1717
DoG Filtering
• Convolution with a variable-scale Gaussian
• Convolution with the DoG filter
( , , ) ( , , ) ( , )L x y G x y I x y 2 2 2( ) / 2
2
1( , , )
2x yG x y e
( , , ) ( ( , , ) ( , , )) ( , )
( , , ) ( , , )
D x y G x y k G x y I x y
L x y k L x y
![Page 18: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/18.jpg)
1818
Scale Space
![Page 19: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/19.jpg)
1919
Scale Space: an Example
( , , ) :L x y
( , , ) :D x y
![Page 20: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/20.jpg)
2020
Outline
• Introduction• Algorithm
– Scale-space extrema detection– Keypoint localization– Orientation assignment– Keypoint descriptor
• Applications• Conclusions
![Page 21: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/21.jpg)
2121
Keypoint Localization
• X is selected if it is larger or smaller than all 26 neighbors
![Page 22: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/22.jpg)
2222
Accurate Keypoint Localization
• Reject the points that have low contrast (and are therefore sensitive to noise) or are poorly localized along and edge
233x189 original image
832 keypoints at extrema
729 keypoints after contrast filtering
536 keypoints after curvature filtering
![Page 23: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/23.jpg)
2323
Outline
• Introduction• Algorithm
– Scale-space extrema detection– Keypoint localization– Orientation assignment– Keypoint descriptor
• Applications• Conclusions
![Page 24: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/24.jpg)
24
Orientation Assignment
• By assigning a consistent orientation, the keypoint descriptor can be orientation invariant
Choose a region around the keypoint
Calculate the magnitude of the gradient
Calculate the magnitude of the gradient
Gaussian kernel
Orientation histogram
2 2
1
( , ) ( ( 1, ) ( 1, )) ( ( , 1) ( , 1))
( , ) tan (( ( , 1) ( , 1)) /( ( 1, ) ( 1, )))
m x y L x y L x y L x y L x y
x y L x y L x y L x y L x y
![Page 25: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/25.jpg)
2525
Orientation Assignment: an Example (1)
![Page 26: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/26.jpg)
2626
Orientation Assignment: an Example (2)
![Page 27: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/27.jpg)
2727
Orientation Assignment: an Example (3)
![Page 28: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/28.jpg)
2828
Orientation Assignment: an Example (4)
approximately 25 degrees80% of peak value
![Page 29: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/29.jpg)
2929
Outline
• Introduction• Algorithm
– Scale-space extrema detection– Keypoint localization– Orientation assignment– Keypoint descriptor
• Applications• Conclusions
![Page 30: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/30.jpg)
30
Descriptor
• It computes a descriptor in order to get highly distinctive description
• The descriptors are as invariant as possible to remaining variations
30
![Page 31: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/31.jpg)
3131
Keypoint Descriptor (1)
• The best result: 8 orientations x 4x4 histogram array = 128 dimensions
![Page 32: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/32.jpg)
3232
Keypoint Descriptor (2)
• The 128D vectors are normalized to unit length in order to reduces effect of contrast change
• Because of using gradients, the discriptor is invariant to the changes in illumination
![Page 33: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/33.jpg)
3333
Outline
• Introduction• Algorithm• Applications• Conclusion
![Page 34: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/34.jpg)
3434
Keypoint Matching
• The best candidate match for each keypoint is found by identifying its nearest neighbor in the database
• Using the ratio of distance (closest/next closest) chooses the correct matches– reject the distance ratio that is greater than
0.8
![Page 35: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/35.jpg)
35
Results
![Page 36: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/36.jpg)
3636
Object Recognition (1)
• It creates a Hough transform entry predicting the model location, orientation, and scale
• Solution for affine parameters:
1 2
3 4
x
y
tm mu x
tm mv y
1
2
3
4
0 0 1 0
0 0 0 1
x
y
m
mx yu
mx yv
m
t
t
Ax b
![Page 37: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/37.jpg)
3737
Object Recognition (2)
• The LS solution can be determined by solving the corresponding normal equations
-1T Tx A A A b
![Page 38: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/38.jpg)
3838
Recognition examples (1)
![Page 39: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/39.jpg)
3939
Recognition examples (2)
![Page 40: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/40.jpg)
4040
Outline
• Introduction• Algorithm• Applications• Conclusions
![Page 41: 11 Scale Invariant Feature Transform (SIFT) David G. Lowe University of British Columbia.](https://reader030.fdocuments.net/reader030/viewer/2022012906/56649d5d5503460f94a3cba8/html5/thumbnails/41.jpg)
4141
Conclusions
• The SIFT keypoints are useful due to their distinctiveness
• This approach transforms an image into a large collection of local feature vectors
• It is invariant to image translation, scaling, and rotation, and partially invariant to illumination changes and affine or 3D projection