• date post

04-Oct-2018
• Category

Documents

• view

219

0

Embed Size (px)

Transcript of Lecture 10: Multi view geometry - Stanford Computer · PDF fileFei-Fei Li Lecture 10 - Lecture...

• Lecture 10 -Fei-Fei Li

Lecture10:Multiviewgeometry

ProfessorFeiFei LiStanfordVisionLab

21Oct111

• Lecture 10 -Fei-Fei Li

Whatwewilllearntoday?

Stereovision Correspondenceproblem(ProblemSet2(Q3)) Activestereovisionsystems Structurefrommotion

21Oct112

• Lecture 10 -Fei-Fei Li

Stereovision Correspondenceproblem(ProblemSet2(Q3)) Activestereovisionsystems Structurefrommotion

21Oct113

Whatwewilllearntoday?

• Lecture 10 -Fei-Fei Li 21Oct114

Twoeyeshelp!

O2 O1

p2p1

?

R,T

• Lecture 10 -Fei-Fei Li 21Oct115

Stereoviewgeometry

Correspondence:Givenapointinoneimage,howcanIfindthecorrespondingpointxinanotherone?

Camerageometry:Givencorrespondingpointsintwoimages,findcameramatrices,positionandpose.

Scenegeometry:Findcoordinatesof3Dpointfromitsprojectioninto2ormultipleimages.

Previouslecture(#9)

• Lecture 10 -Fei-Fei Li 21Oct116

Stereoviewgeometry

Correspondence:Givenapointinoneimage,howcanIfindthecorrespondingpointxinanotherone?

Camerageometry:Givencorrespondingpointsintwoimages,findcameramatrices,positionandpose.

Scenegeometry:Findcoordinatesof3Dpointfromitsprojectioninto2ormultipleimages.

Thislecture(#10)

• Lecture 10 -Fei-Fei Li 21Oct117

StereoviewgeometryPinholeperspectiveprojection

Subgoals: Solvethecorrespondenceproblem Usecorrespondingobservationstotriangulate(depthestimation)

p

P

p

O1 O2

• Lecture 10 -Fei-Fei Li 21Oct118

DepthestimationPinholeperspectiveprojection

Assumecamerasarecalibratedandrectified Assumecorrespondenceproblemissolved Goal:estimatedepth(triangulate)

O O

P

e2p pe2

K Kx

y

z

u

v

u

v

• Lecture 10 -Fei-Fei Li 21Oct119

DepthestimationPinholeperspectiveprojection

f

p p

BaselineB

z

O O

P

f

zfBpp '

Note:Disparityisinverselyproportionaltodepth

=disparity

• Lecture 10 -Fei-Fei Li 21Oct1110

Depthestimation

Pinholeperspectiveprojection

http://vision.middlebury.edu/stereo/

Stereopair

Disparitymap/depthmap Disparitymapwithocclusions

x x

zfBpp '

• Lecture 10 -Fei-Fei Li 21Oct1111

DepthestimationPinholeperspectiveprojection

Subgoals: Solvethecorrespondenceproblem Usecorrespondingobservationstotriangulate

p

P

p

O1 O2

• Lecture 10 -Fei-Fei Li 21Oct1112

CorrespondenceproblemPinholeperspectiveprojectionPinholeperspectiveprojection

p

P

p

O1 O2

Givenapointin3D,discovercorrespondingobservationsinleftandrightimages[alsocalledbinocularfusionproblem]

• Lecture 10 -Fei-Fei Li 21Oct1113

CorrespondenceproblemPinholeperspectiveprojectionPinholeperspectiveprojection

ACooperativeModel(MarrandPoggio,1976)

CorrelationMethods(1970)

MultiScaleEdgeMatching(Marr,PoggioandGrimson,197981)

[FP] Chapters:11

• Lecture 10 -Fei-Fei Li 21Oct1114

CorrelationMethodsPinholeperspectiveprojection

p

Pickupawindowaroundp(u,v)

• Lecture 10 -Fei-Fei Li 21Oct1115

CorrelationMethodsPinholeperspectiveprojection

Pickupawindowaroundp(u,v) BuildvectorW Slidethewindowalongvlineinimage2andcomputew Keepslidinguntilwwismaximized.

• Lecture 10 -Fei-Fei Li 21Oct1116

CorrelationMethodsPinholeperspectiveprojection

NormalizedCorrelation;minimize:)ww)(ww()ww)(ww(

• Lecture 10 -Fei-Fei Li 21Oct1117

CorrelationMethodsPinholeperspectiveprojectionLeft Right

scanline

Norm.corr CreditslideS.Lazebnik

• Lecture 10 -Fei-Fei Li 21Oct1118

CorrelationMethodsPinholeperspectiveprojection

SmallerwindowMoredetailMorenoise

Largerwindow Smootherdisparitymaps Lesspronetonoise

Window size = 3 Window size = 20

CreditslideS.Lazebnik

• Lecture 10 -Fei-Fei Li 21Oct1119

IssueswithCorrelationMethodsPinholeperspectiveprojection

Foreshorteningeffect

Occlusions

OO

O O

ItisdesirabletohavesmallB/zratio!

• Lecture 10 -Fei-Fei Li 21Oct1120

IssueswithCorrelationMethodsPinholeperspectiveprojection

O O

smallB/zratio

Smallerrorinmeasurementsimplieslargeerrorinestimatingdepth

• Lecture 10 -Fei-Fei Li 21Oct1121

IssueswithCorrelationMethodsPinholeperspectiveprojection

Homogeneousregions

Hardtomatchpixelsintheseregions

• Lecture 10 -Fei-Fei Li 21Oct1122

IssueswithCorrelationMethodsPinholeperspectiveprojection

Repetitivepatterns

• Lecture 10 -Fei-Fei Li 21Oct1123

ResultswithwindowsearchPinholeperspectiveprojectionData

Groundtruth Windowbasedmatching

CreditslideS.Lazebnik

• Lecture 10 -Fei-Fei Li 21Oct1124

Improvingcorrespondence:Nonlocalconstraints

Uniqueness Foranypointinoneimage,thereshouldbeatmostonematchingpointintheotherimage

• Lecture 10 -Fei-Fei Li 21Oct1125

Improvingcorrespondence:Nonlocalconstraints

Uniqueness Foranypointinoneimage,thereshouldbeatmostonematchingpointintheotherimage

Ordering Correspondingpointsshouldbeinthesameorderinbothviews

CourtesyJ.Ponce

• Lecture 10 -Fei-Fei Li 21Oct1126

DynamicProgrammingPinholeperspectiveprojection (BakerandBinford,1981)

Nodes=matchedfeaturepoints(e.g.,edgepoints). Arcs=matchedintervalsalongtheepipolar lines. Arccost=discrepancybetweenintervals.

Findtheminimumcostpathgoingmonotonicallydownandrightfromthetopleftcornerofthegraphtoitsbottomrightcorner.

[Usesorderingconstraint]

CourtesyJ.Ponce

• Lecture 10 -Fei-Fei Li 21Oct1127

DynamicProgramming(BakerandBinford,1981)

• Lecture 10 -Fei-Fei Li 21Oct1128

Improvingcorrespondence:Nonlocalconstraints

Uniqueness Foranypointinoneimage,thereshouldbeatmostonematchingpointintheotherimage

Ordering Correspondingpointsshouldbeinthesameorderinbothviews

SmoothnessDisparityistypicallyasmoothfunctionofx(exceptinoccludingboundaries)

• Lecture 10 -Fei-Fei Li 21Oct1129

StereomatchingasenergyminimizationPinholeperspectiveprojectionI1 I2 D

Energyfunctionsofthisformcanbeminimizedusinggraphcuts

Y.Boykov,O.Veksler,andR.Zabih,FastApproximateEnergyMinimizationviaGraphCuts,PAMI01

W1(i) W2(i+D(i)) D(i)

)(),,( smooth21data DEDIIEE

ji

jDiDE,neighbors

smooth )()( 221data ))(()( i

iDiWiWE

• Lecture 10 -Fei-Fei Li 21Oct1130

StereomatchingasenergyminimizationPinholeperspectiveprojection

EnergyminimizationGroundtruth Windowbasedmatching

Y.Boykov,O.Veksler,andR.Zabih,FastApproximateEnergyMinimizationviaGraphCuts,PAMI01

• Lecture 10 -Fei-Fei Li 21Oct1131

Twoframestereocorrespondencealgorithms

http://www.middlebury.edu/stereo/

Clickhere

• Lecture 10 -Fei-Fei Li 21Oct1132

StereoSDKvisionsoftwaredevelopmentkitA.Criminisi,A.BlakeandD.Robertson

• Lecture 10 -Fei-Fei Li 21Oct1133

Foreground/backgroundsegmentationusingstereoV.Kolmogorov,A.Criminisi,A.Blake,G.CrossandC.Rother.Bilayersegmentationofbinocularstereovideo CVPR2005

http://research.microsoft.com/~antcrim/demos/ACriminisi_Recognition_CowDemo.wmv

• Lecture 10 -Fei-Fei Li 21Oct1134

3DUrbanSceneModelingusingastereosystem3DUrbanSceneModelingIntegratingRecognitionandReconstruction,N.Cornelis,B.Leibe,K.Cornelis,L.VanGool,IJCV08.

http://www.vision.ee.ethz.ch/showroom/index.en.html#

• Lecture 10 -Fei-Fei Li 21Oct1135

Stereovision Correspondenceproblem Activestereovisionsystems Structurefrommotion

Whatwewilllearntoday?

• Lecture 10 -Fei-Fei Li 21Oct1136

Activestereo(point)

P

O2

p

Correspondenceproblemsolved!

projector

• Lecture 10 -Fei-Fei Li 21Oct1137

Activestereo(stripe)

OProjectorandcameraareparallel Correspondenceproblemsolved!

projector

• Lecture 10 -Fei-Fei Li 21Oct1138

Activestereo(stripe)

Opticaltriangulation Projectasinglestripeoflaserlight Scanitacrossthesurfaceoftheobject Thisisaverypreciseversionofstructuredlightscanning

Digital Michelangelo Projecthttp://graphics.stanford.edu/projects/mich/

• Lecture 10 -Fei-Fei Li 21Oct1139

Activestereo(stripe)

Digital M

ichelangelo Project

http://graphics.stanford.edu/projects/mich/

• Lecture 10 -Fei-Fei Li 21Oct1140

Lightsource O

1camera,1lightsource verycheapsetup calibratedlightsource

J.Bouguet &P.Perona,99S.Savarese,J.Bouguet &Perona,00

• Lecture 10 -Fei-Fei Li 21Oct1141

• Lecture 10 -Fei-Fei Li 21Oct1142

Activestereo(colorcodedstripes)

Densereconstruction Correspondenceproblemagain Getarounditbyusingcolorcodes

projectorO

L.Zhang,B.Curless,andS.M.Seitz2002S.Rusinkiewicz &Levoy 2002

• Lecture 10 -Fei-Fei Li 21Oct1143

Activestereo(colorcodedstripes)

L.Zhang,B.Curless,andS.M.Seitz.RapidShapeAcquisitionUsingColorStructuredLightandMultipassDynamicProgramming.3DPVT 2002

Rapidshapeacquisition:Projector+stereocameras

• Lecture