Lecture 6: Finding Features (part 1/2) ... Fei-Fei Li Lecture 6 - Lecture 6: Finding Features (part...

Click here to load reader

  • date post

    23-May-2020
  • Category

    Documents

  • view

    5
  • download

    1

Embed Size (px)

Transcript of Lecture 6: Finding Features (part 1/2) ... Fei-Fei Li Lecture 6 - Lecture 6: Finding Features (part...

  • Lecture 6 - Fei-Fei Li

    Lecture 6: Finding Features (part 1/2)

    Dr. Juan Carlos Niebles Stanford AI Lab

    Professor Fei-Fei Li Stanford Vision Lab

    10-Oct-16 1

  • Lecture 6 - Fei-Fei Li

    What we will learn today?

    •  Local invariant features – MoOvaOon –  Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    10-Oct-16 2

    Next lecture (#7)

  • Lecture 6 - Fei-Fei Li

    What we will learn today?

    •  Local invariant features – MoOvaOon –  Requirements, invariances

    •  Keypoint localizaOon – Harris corner detector

    •  Scale invariant region selecOon – AutomaOc scale selecOon – Difference-of-Gaussian (DoG) detector

    •  SIFT: an image region descriptor

    10-Oct-16 3

    Some background reading: Rick Szeliski, Chapter 4.1.1; David Lowe, IJCV 2004

  • Lecture 6 - Fei-Fei Li

    Image matching: a challenging problem

    10-Oct-16 4

  • Lecture 6 - Fei-Fei Li

    by Diva Sian

    by swashford

    Sl id

    e cr

    ed it

    : St

    ev e

    Se it

    z

    10-Oct-16 5

    Image matching: a challenging problem

  • Lecture 6 - Fei-Fei Li

    Harder Case

    by Diva Sian by scgbt

    Sl id

    e cr

    ed it

    : St

    ev e

    Se it

    z

    10-Oct-16 6

  • Lecture 6 - Fei-Fei Li

    Harder SOll?

    NASA Mars Rover images

    Sl id

    e cr

    ed it

    : St

    ev e

    Se it

    z

    10-Oct-16 7

  • Lecture 6 - Fei-Fei Li

    Answer Below (Look for Ony colored squares)

    NASA Mars Rover images with SIFT feature matches (Figure by Noah Snavely) Sl

    id e

    cr ed

    it :

    St ev

    e Se

    it z

    10-Oct-16 8

  • Lecture 6 - Fei-Fei Li

    MoOvaOon for using local features •  Global representaOons have major limitaOons •  Instead, describe and match only local regions •  Increased robustness to

    –  Occlusions

    –  ArOculaOon

    –  Intra-category variaOons

    θq φ

    dq

    φ

    θ

    d

    10-Oct-16 9

  • Lecture 6 - Fei-Fei Li

    General Approach N

    p ix

    el s

    N pixels

    Similarity measure A

    f

    e.g. color

    Bf

    e.g. color

    B1 B2

    B3 A1

    A2 A3

    Tffd BA

  • Lecture 6 - Fei-Fei Li

    Common Requirements •  Problem 1:

    –  Detect the same point independently in both images

    No chance to match!

    We need a repeatable detector! Slid e

    cr ed

    it :

    D ar

    ya F

    ro lo

    va ,

    D en

    is S

    im ak

    ov

    10-Oct-16 11

  • Lecture 6 - Fei-Fei Li

    Common Requirements •  Problem 1:

    –  Detect the same point independently in both images

    •  Problem 2: –  For each point correctly recognize the corresponding one

    We need a reliable and distinctive descriptor!

    ?

    Sl id

    e cr

    ed it

    : D

    ar ya

    F ro

    lo va

    , D

    en is

    S im

    ak ov

    10-Oct-16 12

  • Lecture 6 - Fei-Fei Li

    Invariance: Geometric TransformaOons

    Sl id

    e cr

    ed it

    : St

    ev e

    Se it

    z

    10-Oct-16 13

  • Lecture 6 - Fei-Fei Li

    Levels of Geometric Invariance

    Sl id

    e cr

    ed it

    : Ba

    st ia

    n Le

    ib e

    10-Oct-16 14

    CS131 CS231a

  • Lecture 6 - Fei-Fei Li

    Invariance: Photometric TransformaOons

    •  Ofen modeled as a linear transformaOon: –  Scaling + Offset

    Sl id

    e cr

    ed it

    : Ti

    nn e

    Tu yt

    el aa

    rs

    10-Oct-16 15

  • Lecture 6 - Fei-Fei Li

    Requirements •  Region extracOon needs to be repeatable and accurate

    –  Invariant to translaOon, rotaOon, scale changes –  Robust or covariant to out-of-plane (�affine) transformaOons –  Robust to lighOng variaOons, noise, blur, quanOzaOon

    •  Locality: Features are local, therefore robust to occlusion and cluier.

    •  QuanOty: We need a sufficient number of regions to cover the object.

    •  DisOncOveness: The regions should contain “interesOng” structure.

    •  Efficiency: Close to real-Ome performance.

    Sl id

    e cr

    ed it

    : Ba

    st ia

    n Le

    ib e

    10-Oct-16 16

  • Lecture 6 - Fei-Fei Li

    Many ExisOng Detectors Available

    •  Hessian & Harris [Beaudet ‘78], [Harris ‘88] •  Laplacian, DoG [Lindeberg ‘98], [Lowe ‘99] •  Harris-/Hessian-Laplace [Mikolajczyk & Schmid ‘01] •  Harris-/Hessian-Affine [Mikolajczyk & Schmid ‘04] •  EBR and IBR [Tuytelaars & Van Gool ‘04] •  MSER [Matas ‘02] •  Salient Regions [Kadir & Brady ‘01] •  Others…

    •  Those detectors have become a basic building block for many recent applica8ons in Computer Vision.

    Sl id

    e cr

    ed it

    : Ba

    st ia

    n Le

    ib e

    10-Oct-16 17

  • Lecture 6 - Fei-Fei Li

    Keypoint LocalizaOon

    •  Goals:

    –  Repeatable detecOon –  Precise localizaOon –  InteresOng content

    � Look for two-dimensional signal changes

    Sl id

    e cr

    ed it

    : Ba

    st ia

    n Le

    ib e

    10-Oct-16 18

  • Lecture 6 - Fei-Fei Li

    Finding Corners

    •  Key property: –  In the region around a corner, image gradient has two or more dominant direcOons

    •  Corners are repeatable and dis8nc8ve

    C.Harris and M.Stephens. "A Combined Corner and Edge Detector.“ Proceedings of the 4th Alvey Vision Conference, 1988.

    Sl id

    e cr

    ed it

    : Sv

    et la

    na L

    az eb

    ni k

    10-Oct-16 19

  • Lecture 6 - Fei-Fei Li

    Corners as DisOncOve Interest Points •  Design criteria

    – We should easily recognize the point by looking through a small window (locality)

    –  Shifing the window in any direc8on should give a large change in intensity (good localiza8on)

    “edge”: no change along the edge direction

    “corner”: significant change in all directions

    “flat” region: no change in all directions Sli

    de c

    re di

    t: A

    ly os

    ha E

    fr os

    10-Oct-16 20

  • Lecture 6 - Fei-Fei Li

    Harris Detector FormulaOon

    •  Change of intensity for the shif [u,v]: E(u,v) = w(x, y) I (x +u, y + v)− I (x, y)"# $%

    2

    x ,y ∑

    Intensity Shifted intensity

    Window function

    or Window function w(x,y) =

    Gaussian 1 in window, 0 outside

    Sl id

    e cr

    ed it

    : Ri

    ck S

    ze lis

    ki

    10-Oct-16 21

  • Lecture 6 - Fei-Fei Li

    Harris Detector FormulaOon •  This measure of change can be approximated by:

    where M is a 2�2 matrix computed from image derivaOves:

    ⎥ ⎦

    ⎤ ⎢ ⎣

    ⎡ ≈

    v u

    MvuvuE ][),(

    M

    Sum over image region – the area we are checking for corner

    Gradient with respect to x, times gradient with respect to y

    2

    2 , ( , ) x x y

    x y x y y

    I I I M w x y

    I I I ⎡ ⎤

    = ⎢ ⎥ ⎢ ⎥⎣ ⎦

    Sl id

    e cr

    ed it

    : Ri

    ck S

    ze lis

    ki

    10-Oct-16 22

  • Lecture 6 - Fei-Fei Li

    Harris Detector FormulaOon

    Ix Image I IxIy Iy

    where M is a 2�2 matrix computed from image derivaOves:

    M

    Sum over image region – the area we are checking for corner

    Gradient with respect to x, times gradient with respect to y

    2

    2 , ( , ) x x y

    x y x y y

    I I I M w x y

    I I I ⎡ ⎤

    = ⎢ ⎥ ⎢ ⎥⎣ ⎦

    Sl id

    e cr

    ed it

    : Ri

    ck S

    ze lis

    ki

    10-Oct-16 23

  • Lecture 6 - Fei-Fei Li

    What Does This Matrix Reveal? •  First, let’s consider an axis-aligned corner:

    •  This means: –  Dominant gradient direcOons align with x or y axis –  If either λ is close to 0, then this is not a corner, so look for locaOons where both are large.

    •  What if we have a corner that is not aligned with the image axes?

    ⎥ ⎦

    ⎤ ⎢ ⎣

    ⎡ =

    ⎥ ⎥ ⎦