Lecture 7: Finding Features (part 2/2) · PDF file Fei-Fei Li Lecture 7 - SIFT descriptor...

Click here to load reader

  • date post

    22-May-2020
  • Category

    Documents

  • view

    18
  • download

    0

Embed Size (px)

Transcript of Lecture 7: Finding Features (part 2/2) · PDF file Fei-Fei Li Lecture 7 - SIFT descriptor...

  • Lecture 7 - Fei-Fei Li

    Lecture 7: Finding Features (part 2/2)

    Dr. Juan Carlos Niebles Stanford AI Lab

    Professor Fei-Fei Li Stanford Vision Lab

    17-Oct-16 1

  • Lecture 7 - Fei-Fei Li

    What we will learn today?

    •  Local invariant features – MoOvaOon –  Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    17-Oct-16 2

    Previous lecture (#6)

    Some background reading: David Lowe, IJCV 2004

  • Lecture 7 - Fei-Fei Li

    A quick review

    •  Local invariant features – MoOvaOon – Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    17-Oct-16 3

  • Lecture 7 - Fei-Fei Li

    General Approach N

    p ix

    el s

    N pixels

    Similarity measure A

    f

    e.g. color

    Bf

    e.g. color

    B1 B2

    B3 A1

    A2 A3

    Tffd BA

  • Lecture 7 - Fei-Fei Li

    A quick review

    •  Local invariant features – MoOvaOon –  Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    17-Oct-16 5

  • Lecture 7 - Fei-Fei Li

    Quick review: Harris Corner Detector

    17-Oct-16 6

    “edge”: no change along the edge direction

    “corner”: significant change in all directions

    “flat” region: no change in all directions

    Sl id

    e cr

    ed it

    : Al

    yo sh

    a Ef

    ro s

  • Lecture 7 - Fei-Fei Li

    Quick review: Harris Corner Detector

    17-Oct-16 7

    •  Fast approximaOon –  Avoid compuOng the eigenvalues

    –  α: constant (0.04 to 0.06)

    θ = det(M )−α trace(M )2 = λ1λ2 −α(λ1 +λ2 ) 2

    Sl id

    e cr

    ed it

    : Kr

    is te

    n G

    ra um

    an

    “Corner” θ > 0

    “Edge” θ < 0

    “Edge” θ < 0

    “Flat” region

    �1

  • Lecture 7 - Fei-Fei Li

    Quick review: Harris Corner Detector

    17-Oct-16 8

    Sl id

    e ad

    ap te

    d fr

    om D

    ar ya

    F ro

    lo va

    , D

    en is

    S im

    ak ov

  • Lecture 7 - Fei-Fei Li

    Quick review: Harris Corner Detector

    17-Oct-16 9

    •  TranslaOon invariance •  RotaOon invariance •  Scale invariance?

    Not invariant to image scale!

    All points will be classified as edges!

    Corner

    Sl id

    e cr

    ed it

    : Kr

    is te

    n G

    ra um

    an

  • Lecture 7 - Fei-Fei Li

    What we will learn today?

    •  Local invariant features – MoOvaOon –  Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    17-Oct-16 10

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon •  Consider regions (e.g. circles) of different sizes around a point

    •  Regions of corresponding sizes will look the same in both images

    17-Oct-16 11

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon

    •  The problem: how do we choose corresponding circles independently in each image?

    17-Oct-16 12

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon •  SoluOon:

    –  Design a funcOon on the region (circle), which is “scale invariant” (the same for corresponding regions, even if they are at different scales)

    Example: average intensity. For corresponding regions (even of different sizes) it will be the same.

    scale = 1/2

    –  For a point in one image, we can consider it as a funcOon of region size (circle radius)

    f

    region size

    Image 1 f

    region size

    Image 2

    17-Oct-16 13

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon •  Common approach:

    scale = 1/2

    f

    region size

    Image 1 f

    region size

    Image 2

    Take a local maximum of this funcOon

    •  ObservaOon: region size, for which the maximum is achieved, should be co-variant with image scale.

    s1 s2

    Important: this scale invariant region size is found in each image independently!

    17-Oct-16 14

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon

    •  A “good” funcOon for scale detecOon: has one stable sharp peak

    f

    region size

    ba d

    f

    region size

    bad

    f

    region size

    Good !

    •  For usual images: a good funcOon would be one which responds to contrast (sharp local intensity change)

    17-Oct-16 15

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon •  FuncOons for determining scale

    2 2

    21 2 2

    ( , , ) x y

    G x y e σ πσ

    σ +−

    =

    ( )2 ( , , ) ( , , )xx yyL G x y G x yσ σ σ= +

    ( , , ) ( , , )DoG G x y k G x yσ σ= −

    Kernel Imagef = ∗ Kernels:

    where Gaussian

    Note: both kernels are invariant to scale and rotation

    (Laplacian)

    (Difference of Gaussians)

    17-Oct-16 16

  • Lecture 7 - Fei-Fei Li

    Scale Invariant Detectors •  Harris-Laplacian1

    Find local maximum of: –  Harris corner detector in space (image coordinates)

    –  Laplacian in scale

    1 K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001 2 D.Lowe. “DisOncOve Image Features from Scale-Invariant Keypoints”. IJCV 2004

    scale

    x

    y

    � Harris �

    � L

    ap la

    ci an

    •  SIFT (Lowe)2 Find local maximum of: –  Difference of Gaussians in space

    and scale

    scale

    x

    y

    � DoG �

    � D

    oG �

    17-Oct-16 17

  • Lecture 7 - Fei-Fei Li

    Scale Invariant Detectors

    K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001

    •  Experimental evaluaOon of detectors w.r.t. scale change

    Repeatability rate: # correspondences # possible correspondences

    17-Oct-16 18

  • Lecture 7 - Fei-Fei Li

    Scale Invariant DetecOon: Summary

    •  Given: two images of the same scene with a large scale difference between them

    •  Goal: find the same interest points independently in each image

    •  SoluOon: search for maxima of suitable funcOons in scale and in space (over the image)

    Methods:

    1.  Harris-Laplacian [Mikolajczyk, Schmid]: maximize Laplacian over scale, Harris’ measure of corner response over the image

    2.  SIFT [Lowe]: maximize Difference of Gaussians over scale and space

    17-Oct-16 19

  • Lecture 7 - Fei-Fei Li

    What we will learn today?

    •  Local invariant features – MoOvaOon –  Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    17-Oct-16 20

  • Lecture 7 - Fei-Fei Li

    Local Descriptors •  We know how to detect points •  Next quesOon:

    How to describe them for matching?

    ? Point descriptor should be:

    1.   Invariant 2.   Distinctive Sli

    de c

    re di

    t: K

    ri st

    en G

    ra um

    an

    17-Oct-16 21

  • Lecture 7 - Fei-Fei Li

    CVPR 2003 Tutorial

    Recogni5on and Matching Based on Local Invariant Features

    David Lowe Computer Science Department University of BriOsh Columbia

    17-Oct-16 22

  • Lecture 7 -

    Invariant Local Features

    •  Image content is transformed into local feature coordinates that are invariant to translaOon, rotaOon, scale, and other imaging parameters

    17-Oct-16 23

  • Lecture 7 - Fei-Fei Li

    Advantages of invariant local features •  Locality: features are local, so robust to occlusion and clumer (no prior segmentaOon)

    •  Dis5nc5veness: individual features can be matched to a large database of objects

    •  Quan5ty: many features can be generated for even sma