TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features:...
Transcript of TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features:...
![Page 1: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/1.jpg)
TP12 - Local features:
detection and description
Computer Vision, FCUP, 2018/19
Miguel Coimbra
Slides by Prof. Kristen Grauman
![Page 2: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/2.jpg)
Today
• Local invariant features
– Detection of interest points
• (Harris corner detection)
• Scale invariant blob detection: LoG
– Description of local patches
• SIFT : Histograms of oriented gradients
![Page 3: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/3.jpg)
Local features: main components
1) Detection: Identify the
interest points
2) Description:Extract vector
feature descriptor
surrounding each interest
point.
3) Matching: Determine
correspondence between
descriptors in two views
],,[ )1()1(
11 dxx x
],,[ )2()2(
12 dxx x
Kristen Grauman
![Page 4: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/4.jpg)
Goal: interest operator repeatability
• We want to detect (at least some of) the
same points in both images.
• Yet we have to be able to run the detection
procedure independently per image.
No chance to find true matches!
![Page 5: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/5.jpg)
Goal: descriptor distinctiveness
• We want to be able to reliably determine
which point goes with which.
• Must provide some invariance to geometric
and photometric differences between the two
views.
?
![Page 6: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/6.jpg)
Local features: main components
1) Detection: Identify the
interest points
2) Description:Extract vector
feature descriptor
surrounding each interest
point.
3) Matching: Determine
correspondence between
descriptors in two views
![Page 7: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/7.jpg)
yyyx
yxxx
IIII
IIIIyxwM ),(
x
II x
y
II y
y
I
x
III yx
Recall: Corners as distinctive interest points
2 x 2 matrix of image derivatives (averaged in
neighborhood of a point).
Notation:
![Page 8: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/8.jpg)
Since M is symmetric, we have TXXM
2
1
0
0
iii xMx
The eigenvalues of M reveal the amount of
intensity change in the two principal orthogonal
gradient directions in the window.
Recall: Corners as distinctive interest points
![Page 9: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/9.jpg)
“flat” region
1 and 2 are
small;
“edge”:
1 >> 2
2 >> 1
“corner”:
1 and 2 are large,
1 ~ 2;
One way to score
the cornerness:
Recall: Corners as distinctive interest points
![Page 10: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/10.jpg)
Harris corner detector
1) Compute M matrix for image window surrounding
each pixel to get its cornerness score.
2) Find points with large corner response (f >
threshold)
3) Take the points of local maxima, i.e., perform non-
maximum suppression
![Page 11: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/11.jpg)
Harris Detector: Steps
![Page 12: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/12.jpg)
Harris Detector: Steps
Compute corner response f
![Page 13: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/13.jpg)
Harris Detector: Steps
Find points with large corner response: f > threshold
![Page 14: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/14.jpg)
Harris Detector: Steps
Take only the points of local maxima of f
![Page 15: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/15.jpg)
Harris Detector: Steps
![Page 16: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/16.jpg)
Properties of the Harris corner detector
Rotation invariant?
Scale invariant?
TXXM
2
1
0
0
Yes
![Page 17: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/17.jpg)
Properties of the Harris corner detector
Rotation invariant?
Scale invariant?
All points will be
classified as edgesCorner !
Yes
No
![Page 18: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/18.jpg)
Scale invariant interest points
How can we independently select interest points in
each image, such that the detections are repeatable
across different scales?
![Page 19: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/19.jpg)
Automatic scale selection
Intuition:
• Find scale that gives local maxima of some function
f in both position and scale.
f
region size
Image 1f
region size
Image 2
s1 s2
![Page 20: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/20.jpg)
What can be the “signature” function?
![Page 21: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/21.jpg)
Recall: Edge detection
gdx
df
f
gdx
d
Source: S. Seitz
Edge
Derivative
of Gaussian
Edge = maximum
of derivative
![Page 22: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/22.jpg)
gdx
df
2
2
f
gdx
d2
2
Edge
Second derivative
of Gaussian
(Laplacian)
Edge = zero crossing
of second derivative
Source: S. Seitz
Recall: Edge detection
![Page 23: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/23.jpg)
From edges to blobs
• Edge = ripple
• Blob = superposition of two ripples
Spatial selection: the magnitude of the Laplacian
response will achieve a maximum at the center of
the blob, provided the scale of the Laplacian is
“matched” to the scale of the blob
maximum
Slide credit: Lana Lazebnik
![Page 24: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/24.jpg)
Blob detection in 2D
Laplacian of Gaussian: Circularly symmetric
operator for blob detection in 2D
2
2
2
22
y
g
x
gg
![Page 25: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/25.jpg)
Blob detection in 2D: scale selection
Laplacian-of-Gaussian = “blob” detector2
2
2
22
y
g
x
gg
filter
scale
s
img1 img2 img3Bastian Leibe
![Page 26: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/26.jpg)
Blob detection in 2D
We define the characteristic scale as the scale
that produces peak of Laplacian response
characteristic scale
Slide credit: Lana Lazebnik
![Page 27: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/27.jpg)
Example
Original image
at ¾ the size
Kristen Grauman
![Page 28: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/28.jpg)
Original image
at ¾ the size
Kristen Grauman
![Page 29: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/29.jpg)
Kristen Grauman
![Page 30: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/30.jpg)
Kristen Grauman
![Page 31: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/31.jpg)
Kristen Grauman
![Page 32: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/32.jpg)
Kristen Grauman
![Page 33: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/33.jpg)
Kristen Grauman
![Page 34: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/34.jpg)
)()( yyxx LL
1
2
3
4
5
List of
(x, y, σ)
scale
Scale invariant interest points
Interest points are local maxima in both position
and scale.
Squared filter
response maps
![Page 35: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/35.jpg)
Scale-space blob detector: Example
Image credit: Lana Lazebnik
![Page 36: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/36.jpg)
We can approximate the Laplacian with a
difference of Gaussians; more efficient to
implement.
2 ( , , ) ( , , )xx yyL G x y G x y
( , , ) ( , , )DoG G x y k G x y
(Laplacian)
(Difference of Gaussians)
Technical detail
![Page 37: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/37.jpg)
Local features: main components
1) Detection: Identify the
interest points
2) Description:Extract vector
feature descriptor
surrounding each interest
point.
3) Matching: Determine
correspondence between
descriptors in two views
],,[ )1()1(
11 dxx x
],,[ )2()2(
12 dxx x
![Page 38: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/38.jpg)
Geometric transformations
e.g. scale,
translation,
rotation
![Page 39: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/39.jpg)
Photometric transformations
Figure from T. Tuytelaars ECCV 2006 tutorial
![Page 40: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/40.jpg)
Raw patches as local descriptors
The simplest way to describe the
neighborhood around an interest
point is to write down the list of
intensities to form a feature vector.
But this is very sensitive to even
small shifts, rotations.
![Page 41: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/41.jpg)
SIFT descriptor [Lowe 2004]
• Use histograms to bin pixels within sub-patches
according to their orientation.
0 2p
Why subpatches?
Why does SIFT
have some
illumination
invariance?
![Page 42: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/42.jpg)
CSE 576: Computer Vision
Making descriptor rotation invariant
Image from Matthew Brown
• Rotate patch according to its dominant gradient
orientation
• This puts the patches into a canonical orientation.
![Page 43: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/43.jpg)
• Extraordinarily robust matching technique
• Can handle changes in viewpoint
• Up to about 60 degree out of plane rotation
• Can handle significant changes in illumination
• Sometimes even day vs. night (below)
• Fast and efficient—can run in real time
• Lots of code available• http://people.csail.mit.edu/albert/ladypack/wiki/index.php/Known_implementations_of_SIFT
Steve Seitz
SIFT descriptor [Lowe 2004]
![Page 44: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/44.jpg)
Example
NASA Mars Rover images
![Page 45: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/45.jpg)
NASA Mars Rover images
with SIFT feature matches
Figure by Noah Snavely
Example
![Page 46: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/46.jpg)
SIFT properties
• Invariant to
– Scale
– Rotation
• Partially invariant to
– Illumination changes
– Camera viewpoint
– Occlusion, clutter
![Page 47: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/47.jpg)
Local features: main components
1) Detection: Identify the
interest points
2) Description:Extract vector
feature descriptor
surrounding each interest
point.
3) Matching: Determine
correspondence between
descriptors in two views
![Page 48: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/48.jpg)
Matching local features
Kristen Grauman
![Page 49: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/49.jpg)
Matching local features
?
To generate candidate matches, find patches that have
the most similar appearance (e.g., lowest SSD)
Simplest approach: compare them all, take the closest (or
closest k, or within a thresholded distance)
Image 1 Image 2
Kristen Grauman
![Page 50: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/50.jpg)
Ambiguous matches
At what SSD value do we have a good match?
To add robustness to matching, can consider ratio :
distance to best match / distance to second best match
If low, first match looks good.
If high, could be ambiguous match.
Image 1 Image 2
? ? ? ?
Kristen Grauman
![Page 51: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/51.jpg)
Matching SIFT Descriptors
• Nearest neighbor (Euclidean distance)
• Threshold ratio of nearest to 2nd nearest descriptor
Lowe IJCV 2004
![Page 52: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/52.jpg)
Recap: robust feature-based alignment
Source: L. Lazebnik
![Page 53: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/53.jpg)
Recap: robust feature-based alignment
• Extract features
Source: L. Lazebnik
![Page 54: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/54.jpg)
Recap: robust feature-based alignment
• Extract features
• Compute putative matches
Source: L. Lazebnik
![Page 55: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/55.jpg)
Recap: robust feature-based alignment
• Extract features
• Compute putative matches
• Loop:
• Hypothesize transformation T (small group of putative
matches that are related by T)
Source: L. Lazebnik
![Page 56: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/56.jpg)
Recap: robust feature-based alignment
• Extract features
• Compute putative matches
• Loop:
• Hypothesize transformation T (small group of putative
matches that are related by T)
• Verify transformation (search for other matches consistent
with T)
Source: L. Lazebnik
![Page 57: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/57.jpg)
Recap: robust feature-based alignment
• Extract features
• Compute putative matches
• Loop:
• Hypothesize transformation T (small group of putative
matches that are related by T)
• Verify transformation (search for other matches consistent
with T)
Source: L. Lazebnik
![Page 58: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/58.jpg)
Applications of local
invariant features
• Wide baseline stereo
• Motion tracking
• Panoramas
• Mobile robot navigation
• 3D reconstruction
• Recognition
• …
![Page 59: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/59.jpg)
Automatic mosaicing
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
![Page 60: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/60.jpg)
Wide baseline stereo
[Image from T. Tuytelaars ECCV 2006 tutorial]
![Page 61: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/61.jpg)
Recognition of specific objects, scenes
Rothganger et al. 2003 Lowe 2002
Schmid and Mohr 1997 Sivic and Zisserman, 2003
Kristen Grauman
![Page 62: TP12 - Local features: detection and descriptionmcoimbra/lectures/VC... · TP12 - Local features: detection and description Computer Vision, FCUP, 2018/19 Miguel Coimbra Slides by](https://reader034.fdocuments.net/reader034/viewer/2022050203/5f568dd866982b03a91bb7ac/html5/thumbnails/62.jpg)
Summary
• Interest point detection
– Harris corner detector
– Laplacian of Gaussian, automatic scale selection
• Invariant descriptors
– Rotation according to dominant gradient direction
– Histograms for robustness to small shifts and
translations (SIFT descriptor)