Lecture 10: Multi view geometry - Stanford Computer · PDF fileFei-Fei Li Lecture 10 - Lecture...

Click here to load reader

  • date post

    04-Oct-2018
  • Category

    Documents

  • view

    219
  • download

    0

Embed Size (px)

Transcript of Lecture 10: Multi view geometry - Stanford Computer · PDF fileFei-Fei Li Lecture 10 - Lecture...

  • Lecture 10 -Fei-Fei Li

    Lecture10:Multiviewgeometry

    ProfessorFeiFei LiStanfordVisionLab

    21Oct111

  • Lecture 10 -Fei-Fei Li

    Whatwewilllearntoday?

    Stereovision Correspondenceproblem(ProblemSet2(Q3)) Activestereovisionsystems Structurefrommotion

    21Oct112

    Reading:[HZ] Chapters:4,9,11[FP] Chapters:10

  • Lecture 10 -Fei-Fei Li

    Stereovision Correspondenceproblem(ProblemSet2(Q3)) Activestereovisionsystems Structurefrommotion

    21Oct113

    Whatwewilllearntoday?

    Reading:[HZ] Chapters:4,9,11[FP] Chapters:10

  • Lecture 10 -Fei-Fei Li 21Oct114

    Twoeyeshelp!

    O2 O1

    p2p1

    ?

    R,T

  • Lecture 10 -Fei-Fei Li 21Oct115

    Stereoviewgeometry

    Correspondence:Givenapointinoneimage,howcanIfindthecorrespondingpointxinanotherone?

    Camerageometry:Givencorrespondingpointsintwoimages,findcameramatrices,positionandpose.

    Scenegeometry:Findcoordinatesof3Dpointfromitsprojectioninto2ormultipleimages.

    Previouslecture(#9)

  • Lecture 10 -Fei-Fei Li 21Oct116

    Stereoviewgeometry

    Correspondence:Givenapointinoneimage,howcanIfindthecorrespondingpointxinanotherone?

    Camerageometry:Givencorrespondingpointsintwoimages,findcameramatrices,positionandpose.

    Scenegeometry:Findcoordinatesof3Dpointfromitsprojectioninto2ormultipleimages.

    Thislecture(#10)

  • Lecture 10 -Fei-Fei Li 21Oct117

    StereoviewgeometryPinholeperspectiveprojection

    Subgoals: Solvethecorrespondenceproblem Usecorrespondingobservationstotriangulate(depthestimation)

    p

    P

    p

    O1 O2

  • Lecture 10 -Fei-Fei Li 21Oct118

    DepthestimationPinholeperspectiveprojection

    Assumecamerasarecalibratedandrectified Assumecorrespondenceproblemissolved Goal:estimatedepth(triangulate)

    O O

    P

    e2p pe2

    K Kx

    y

    z

    u

    v

    u

    v

  • Lecture 10 -Fei-Fei Li 21Oct119

    DepthestimationPinholeperspectiveprojection

    f

    p p

    BaselineB

    z

    O O

    P

    f

    zfBpp '

    Note:Disparityisinverselyproportionaltodepth

    =disparity

  • Lecture 10 -Fei-Fei Li 21Oct1110

    Depthestimation

    Pinholeperspectiveprojection

    http://vision.middlebury.edu/stereo/

    Stereopair

    Disparitymap/depthmap Disparitymapwithocclusions

    x x

    zfBpp '

  • Lecture 10 -Fei-Fei Li 21Oct1111

    DepthestimationPinholeperspectiveprojection

    Subgoals: Solvethecorrespondenceproblem Usecorrespondingobservationstotriangulate

    p

    P

    p

    O1 O2

  • Lecture 10 -Fei-Fei Li 21Oct1112

    CorrespondenceproblemPinholeperspectiveprojectionPinholeperspectiveprojection

    p

    P

    p

    O1 O2

    Givenapointin3D,discovercorrespondingobservationsinleftandrightimages[alsocalledbinocularfusionproblem]

  • Lecture 10 -Fei-Fei Li 21Oct1113

    CorrespondenceproblemPinholeperspectiveprojectionPinholeperspectiveprojection

    ACooperativeModel(MarrandPoggio,1976)

    CorrelationMethods(1970)

    MultiScaleEdgeMatching(Marr,PoggioandGrimson,197981)

    [FP] Chapters:11

  • Lecture 10 -Fei-Fei Li 21Oct1114

    CorrelationMethodsPinholeperspectiveprojection

    p

    Pickupawindowaroundp(u,v)

  • Lecture 10 -Fei-Fei Li 21Oct1115

    CorrelationMethodsPinholeperspectiveprojection

    Pickupawindowaroundp(u,v) BuildvectorW Slidethewindowalongvlineinimage2andcomputew Keepslidinguntilwwismaximized.

  • Lecture 10 -Fei-Fei Li 21Oct1116

    CorrelationMethodsPinholeperspectiveprojection

    NormalizedCorrelation;minimize:)ww)(ww()ww)(ww(

  • Lecture 10 -Fei-Fei Li 21Oct1117

    CorrelationMethodsPinholeperspectiveprojectionLeft Right

    scanline

    Norm.corr CreditslideS.Lazebnik

  • Lecture 10 -Fei-Fei Li 21Oct1118

    CorrelationMethodsPinholeperspectiveprojection

    SmallerwindowMoredetailMorenoise

    Largerwindow Smootherdisparitymaps Lesspronetonoise

    Window size = 3 Window size = 20

    CreditslideS.Lazebnik

  • Lecture 10 -Fei-Fei Li 21Oct1119

    IssueswithCorrelationMethodsPinholeperspectiveprojection

    Foreshorteningeffect

    Occlusions

    OO

    O O

    ItisdesirabletohavesmallB/zratio!

  • Lecture 10 -Fei-Fei Li 21Oct1120

    IssueswithCorrelationMethodsPinholeperspectiveprojection

    O O

    smallB/zratio

    Smallerrorinmeasurementsimplieslargeerrorinestimatingdepth

  • Lecture 10 -Fei-Fei Li 21Oct1121

    IssueswithCorrelationMethodsPinholeperspectiveprojection

    Homogeneousregions

    Hardtomatchpixelsintheseregions

  • Lecture 10 -Fei-Fei Li 21Oct1122

    IssueswithCorrelationMethodsPinholeperspectiveprojection

    Repetitivepatterns

  • Lecture 10 -Fei-Fei Li 21Oct1123

    ResultswithwindowsearchPinholeperspectiveprojectionData

    Groundtruth Windowbasedmatching

    CreditslideS.Lazebnik

  • Lecture 10 -Fei-Fei Li 21Oct1124

    Improvingcorrespondence:Nonlocalconstraints

    Uniqueness Foranypointinoneimage,thereshouldbeatmostonematchingpointintheotherimage

  • Lecture 10 -Fei-Fei Li 21Oct1125

    Improvingcorrespondence:Nonlocalconstraints

    Uniqueness Foranypointinoneimage,thereshouldbeatmostonematchingpointintheotherimage

    Ordering Correspondingpointsshouldbeinthesameorderinbothviews

    CourtesyJ.Ponce

  • Lecture 10 -Fei-Fei Li 21Oct1126

    DynamicProgrammingPinholeperspectiveprojection (BakerandBinford,1981)

    Nodes=matchedfeaturepoints(e.g.,edgepoints). Arcs=matchedintervalsalongtheepipolar lines. Arccost=discrepancybetweenintervals.

    Findtheminimumcostpathgoingmonotonicallydownandrightfromthetopleftcornerofthegraphtoitsbottomrightcorner.

    [Usesorderingconstraint]

    CourtesyJ.Ponce

  • Lecture 10 -Fei-Fei Li 21Oct1127

    DynamicProgramming(BakerandBinford,1981)

    (Ohta andKanade,1985)

  • Lecture 10 -Fei-Fei Li 21Oct1128

    Improvingcorrespondence:Nonlocalconstraints

    Uniqueness Foranypointinoneimage,thereshouldbeatmostonematchingpointintheotherimage

    Ordering Correspondingpointsshouldbeinthesameorderinbothviews

    SmoothnessDisparityistypicallyasmoothfunctionofx(exceptinoccludingboundaries)

  • Lecture 10 -Fei-Fei Li 21Oct1129

    StereomatchingasenergyminimizationPinholeperspectiveprojectionI1 I2 D

    Energyfunctionsofthisformcanbeminimizedusinggraphcuts

    Y.Boykov,O.Veksler,andR.Zabih,FastApproximateEnergyMinimizationviaGraphCuts,PAMI01

    W1(i) W2(i+D(i)) D(i)

    )(),,( smooth21data DEDIIEE

    ji

    jDiDE,neighbors

    smooth )()( 221data ))(()( i

    iDiWiWE

  • Lecture 10 -Fei-Fei Li 21Oct1130

    StereomatchingasenergyminimizationPinholeperspectiveprojection

    EnergyminimizationGroundtruth Windowbasedmatching

    Y.Boykov,O.Veksler,andR.Zabih,FastApproximateEnergyMinimizationviaGraphCuts,PAMI01

  • Lecture 10 -Fei-Fei Li 21Oct1131

    Twoframestereocorrespondencealgorithms

    http://www.middlebury.edu/stereo/

    Clickhere

  • Lecture 10 -Fei-Fei Li 21Oct1132

    StereoSDKvisionsoftwaredevelopmentkitA.Criminisi,A.BlakeandD.Robertson

  • Lecture 10 -Fei-Fei Li 21Oct1133

    Foreground/backgroundsegmentationusingstereoV.Kolmogorov,A.Criminisi,A.Blake,G.CrossandC.Rother.Bilayersegmentationofbinocularstereovideo CVPR2005

    http://research.microsoft.com/~antcrim/demos/ACriminisi_Recognition_CowDemo.wmv

  • Lecture 10 -Fei-Fei Li 21Oct1134

    3DUrbanSceneModelingusingastereosystem3DUrbanSceneModelingIntegratingRecognitionandReconstruction,N.Cornelis,B.Leibe,K.Cornelis,L.VanGool,IJCV08.

    http://www.vision.ee.ethz.ch/showroom/index.en.html#

  • Lecture 10 -Fei-Fei Li 21Oct1135

    Stereovision Correspondenceproblem Activestereovisionsystems Structurefrommotion

    Whatwewilllearntoday?

    Reading:[HZ] Chapters:4,9,11[FP] Chapters:10

  • Lecture 10 -Fei-Fei Li 21Oct1136

    Activestereo(point)

    Replaceoneofthetwocamerasbyaprojector Singlecamera ProjectorgeometrycalibratedWhatstheadvantageofhavingtheprojector?

    P

    O2

    p

    Correspondenceproblemsolved!

    projector

  • Lecture 10 -Fei-Fei Li 21Oct1137

    Activestereo(stripe)

    OProjectorandcameraareparallel Correspondenceproblemsolved!

    projector

  • Lecture 10 -Fei-Fei Li 21Oct1138

    Activestereo(stripe)

    Opticaltriangulation Projectasinglestripeoflaserlight Scanitacrossthesurfaceoftheobject Thisisaverypreciseversionofstructuredlightscanning

    Digital Michelangelo Projecthttp://graphics.stanford.edu/projects/mich/

  • Lecture 10 -Fei-Fei Li 21Oct1139

    Activestereo(stripe)

    Digital M

    ichelangelo Project

    http://graphics.stanford.edu/projects/mich/

  • Lecture 10 -Fei-Fei Li 21Oct1140

    Activestereo(shadows)

    Lightsource O

    1camera,1lightsource verycheapsetup calibratedlightsource

    J.Bouguet &P.Perona,99S.Savarese,J.Bouguet &Perona,00

  • Lecture 10 -Fei-Fei Li 21Oct1141

    Activestereo(shadows)J.Bouguet &P.Perona,99S.Savarese,J.Bouguet &Perona,00

  • Lecture 10 -Fei-Fei Li 21Oct1142

    Activestereo(colorcodedstripes)

    Densereconstruction Correspondenceproblemagain Getarounditbyusingcolorcodes

    projectorO

    L.Zhang,B.Curless,andS.M.Seitz2002S.Rusinkiewicz &Levoy 2002

  • Lecture 10 -Fei-Fei Li 21Oct1143

    Activestereo(colorcodedstripes)

    L.Zhang,B.Curless,andS.M.Seitz.RapidShapeAcquisitionUsingColorStructuredLightandMultipassDynamicProgramming.3DPVT 2002

    Rapidshapeacquisition:Projector+stereocameras

  • Lecture