CS6320 CV F2012 Rectification

download CS6320 CV F2012 Rectification

of 40

Transcript of CS6320 CV F2012 Rectification

  • 8/18/2019 CS6320 CV F2012 Rectification

    1/40

    Image Rectification (Stereo)

    Guido Gerig

    CS 6320 Spring 2012

  • 8/18/2019 CS6320 CV F2012 Rectification

    2/40

    Example: converging cameras

    courtesy of Andrew Zisserman

  • 8/18/2019 CS6320 CV F2012 Rectification

    3/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    4/40

    Stereo image rectification

    In practice, it is

    convenient if image

    scanlines are the

    epipolar lines.

  • 8/18/2019 CS6320 CV F2012 Rectification

    5/40

    Image Rectification

    http://en.wikipedia.org/wiki/Image_rectification

    If the two cameras are aligned to be coplanar , the search is

    simplified to one dimension - a horizontal line parallel to the

    baseline between the cameras

    http://en.wikipedia.org/wiki/Coplanarhttp://en.wikipedia.org/wiki/Baselinehttp://en.wikipedia.org/wiki/Baselinehttp://en.wikipedia.org/wiki/Coplanar

  • 8/18/2019 CS6320 CV F2012 Rectification

    6/40

    Stereo image rectification

    • Image Reprojection– reproject image planes

    onto commonplane parallel to linebetween optical centers

    – a homography (3x3transform)applied to both inputimages

    – pixel motion is horizontalafter this transformation

    C. Loop and Z. Zhang.Computing RectifyingHomographies for StereoVision. IEEE Conf.Computer Vision andPattern Recognition,1999.

    http://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdf

  • 8/18/2019 CS6320 CV F2012 Rectification

    7/40

    Stereo image rectification:example

    Source: Alyosha Efros

  • 8/18/2019 CS6320 CV F2012 Rectification

    8/40

    l l 

    Image Rectification

     p p

     P 

    O   O

    l    l  p

      p

    ee

    • Common Image Plane

    • Parallel Epipolar Lines

    • Search Correspondences

    on scan line

    • Epipoles → ∞ 

  • 8/18/2019 CS6320 CV F2012 Rectification

    9/40

    All epipolar lines are parallel in the rectified image plane.

    Image Rectification

  • 8/18/2019 CS6320 CV F2012 Rectification

    10/40

    Image Rectification

  • 8/18/2019 CS6320 CV F2012 Rectification

    11/40

    ]R [TE

    T

    IR 

    x

    ]0,0,[   d 0 0 0

    0 0 d

    0 –d   0

    0Epp

    Essential matrix example: parallel cameras

    For the parallel cameras,

    image of any point must lie on

    same horizontal line in each

    image plane.Grauman

  • 8/18/2019 CS6320 CV F2012 Rectification

    12/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    13/40

    Example I: compute the fundamental matrix for a parallel camera stereo rig

    • reduces to y = y/ 

    , i.e. raster correspondence (horizontal scan-lines)

    f

    f

    XY

    Z

  • 8/18/2019 CS6320 CV F2012 Rectification

    14/40

    f

    f

    XY

    Z

    Geometric interpretation ?

  • 8/18/2019 CS6320 CV F2012 Rectification

    15/40

    Image pair rectification

    Goal: Simplify stereo matching by “warping” the

    images

     Apply projective transformation so that epipolar lines

    correspond to horizontal scanlines

    e

    e

    map epipole e to (1,0,0)

    try to minimize image distortion

    problem when epipole in (or close to) the image

    He

    0

    0

    1

  • 8/18/2019 CS6320 CV F2012 Rectification

    16/40

    Planar rectification

    Bring two views

    to standard stereo setup

    (moves epipole to)

    (not possible when in/close to image)

    Image Transformations: Find

    homographies H’ and H sothat after transformations, Fn

    becomes F of parallel cameras:

  • 8/18/2019 CS6320 CV F2012 Rectification

    17/40

    Algorithm Rectification

    Following Trucco & Verri book pp. 159

    • known T and R between cameras

    • Rotate left camera so that epipole el goes to infinity along horizontal axis

    • Apply same rotation to right camera torecover geometry

    • Rotate right camera by R-1

    • Adjust scale

  • 8/18/2019 CS6320 CV F2012 Rectification

    18/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    19/40

    From: Trucco & Verri, Introductory Techniques for 3-D Computer

    Vision, pp. 157-161

  • 8/18/2019 CS6320 CV F2012 Rectification

    20/40

    Rectification ctd.

     before

    after

  • 8/18/2019 CS6320 CV F2012 Rectification

    21/40

    Stereo matching withgeneral camera configuration

  • 8/18/2019 CS6320 CV F2012 Rectification

    22/40

    Image pair rectification

  • 8/18/2019 CS6320 CV F2012 Rectification

    23/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    24/40

    Other Material /Code

    • Epipolar Geometry, Rectification:http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.html 

    Fusiello, Trucco & Verri:Tutorial, Matlabcode etc:http://profs.sci.univr.it/~fusiello/demo/rect/ 

    http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.htmlhttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.htmlhttp://profs.sci.univr.it/~fusiello/demo/rect/http://profs.sci.univr.it/~fusiello/demo/rect/http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.htmlhttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.html

  • 8/18/2019 CS6320 CV F2012 Rectification

    25/4025

    Goal: 3D from Stereo viaDisparity Map

    F&P

    Chapter 11

    image I(x,y) image I´(x´,y´)Disparity map D(x,y)

    (x´,y´)=(x+D(x,y),y)

  • 8/18/2019 CS6320 CV F2012 Rectification

    26/40

    Example: Stereo to Depth Map

  • 8/18/2019 CS6320 CV F2012 Rectification

    27/40

    Run Example

    Demo for stereo reconstruction (out of date):http://mitpress.mit.edu/e- journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.html 

    Updated Webpages:

    http://research.microsoft.com/en-

    us/um/people/zhang/INRIA/softwares.html 

    SFM Example:

    http://research.microsoft.com/en-

    us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.html 

    Software:

    http://research.microsoft.com/en-

    us/um/people/zhang/INRIA/softwares.html 

    http://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.html

  • 8/18/2019 CS6320 CV F2012 Rectification

    28/40

    Example: Zhengyou Zhang 

    Fundamental matrix between thetwo cameras:

    http://www.inria.fr/robotvis/personnel/zzhang/zzhang-eng.htmlhttp://www.inria.fr/robotvis/personnel/zzhang/zzhang-eng.html

  • 8/18/2019 CS6320 CV F2012 Rectification

    29/40

    Points have been

    extracted using

    Harris cornerdetector, point

    matches via

    fundamental

    matrix F and

    search alongepipolar lines.

  • 8/18/2019 CS6320 CV F2012 Rectification

    30/40

    Point matches found by acorrelation technique 

  • 8/18/2019 CS6320 CV F2012 Rectification

    31/40

    3D reconstruction representedby a pseudo stereogram 

    http://research.microsoft.com/en-

    us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.html

  • 8/18/2019 CS6320 CV F2012 Rectification

    32/40

     Additional Materials: Forward

    Translating Cameras

  • 8/18/2019 CS6320 CV F2012 Rectification

    33/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    34/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    35/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    36/40

    f

    f

    X YZ

    first image second image

  • 8/18/2019 CS6320 CV F2012 Rectification

    37/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    38/40

  • 8/18/2019 CS6320 CV F2012 Rectification

    39/40

    e

    e’ 

  • 8/18/2019 CS6320 CV F2012 Rectification

    40/40

    Summary: Properties of the Fundamental matrix