Stanford CS223B Computer Vision, Winter 2006 Lecture 5 Stereo I Professor Sebastian Thrun CAs: Dan...

Stanford CS223B Computer Vision, Winter 2006

Lecture 5 Stereo I

Professor Sebastian ThrunCAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado

StereoStereo

Sebastian Thrun Stanford University CS223B Computer Vision

Homework #1


Vocabulary Quiz

Baseline Epipole Fundamental Matrix Essential Matrix Stereo Rectification


Stereo Vision: Illustration

http://www.well.com/user/jimg/stereo/stereo_list.html


Stereo Example (Stanley Robot)

Disparity map


Stereo Example


Stereo Vision: Outline

Basic Equations Epipolar Geometry Image Rectification Reconstruction Correspondence Dense and Layered Stereo (Active Range Imaging Techniques)


The Two Problems of Stereo

Correspondence (Wed) Reconstruction (Today)


Pinhole Camera Model

Imageplane Focal length f

Center ofprojection



Imageplane

),,( ZYXP

),,( ZYXP

f

Oy

x

z

Z

Z

Y

Y

X

X

ZZ

YY

XX

OPPO



Imageplane

),,( ZYXP

),,( ZYXP

f

Oy

x

z

),(),(),,(Z

Yf

Z

XfyxZYX

YyXxZ

YfY

Z

XfXfZ


Basic Stereo Derivations

),,(1 ZYXP 1Oy

x

z

f

2Oy

x

z

B

BfxxZ ,,, offunction a as for expression Derive 21

1p

2p


Basic Stereo Derivations

),,(1 ZYXP 1Oy

x

z

f

2Oy

x

z

211

11

1

12

1

11 ,

xx

BfZ

Z

Bfx

Z

BXfx

Z

Xfx

B


What If…?

),,(1 ZYXP 1Oy

x

z

f

2Oy

x

z

B

1p

2p

),,(1 ZYXP 1Oy

x

z

1p

f2O

y

x

z

2p


Epipolar Geometry

pl pr

P

Ol Or

Xl

Xr

Pl Pr

fl fr

Zl

Yl

Zr

Yr

Rrotation Tontranslati


Epipolar Geometry

plp

r

P

Ol Orel er

Pl Pr

Epipolar Plane

Epipolar Lines

Epipoles


Epipolar Geometry

Epipolar plane: plane going through point P and the centers of projection (COPs) of the two cameras

Epipoles: The image in one camera of the COP of the other

Epipolar Constraint: Corresponding points must lie on epipolar lines


Essential Matrix

pl pr

P

Ol Orel er

Pl Pr

Coplanarity T, Pl, PlT: 0)()( lT

l PTTP

)( TPRP lr Coordinate Transformation:

0

0

0

xy

xz

yz

TT

TT

TT

S

ll SPPT

0)( lT

rT SPPR

0lT

r RSPP

0)()( lT

rT PTPRResolves to

RSE Essential Matrix 0lT

r EPP


Essential Matrix

pl pr

P

Ol Orel er

Pl Pr

0

0

0

xy

xz

yz

TT

TT

TT

SRSE Essential Matrix

0 lTr Epp0l

Tr EPP

Projective Line: lr Epu


Fundamental Matrix

Same as Essential Matrix in Camera Pixel Coordinates

0lTr pFp

0lTr Epp

Pixel coordinates 11 lT

r EMMF

Intrinsic parameters


Intrinsic Parameters (See Chapter 2)

100

/0

0/

yy

xx

osf

osf

M


Computing F: The Eight-Point Algorithm

Problem: Recover F (3-3 matrix of rank 2) Ides: Get 8 points:

Minimize:

Notice: Argument linear in coefficients of F

0)8()8(

0)1()1(

lT

r

lT

r

pFp

pFp

8

1

2)()(argmin

iFii l

Tr pFp



Run Singular Value Decomposition of A– Appendix A.6, page 322-325– See also G. Strang: Linear algebra and its applications

Least squares solution: column of V corresponding to

the smallest eigenvalue of A

0Ax

SVD viaTUDVA



Idea: Compile points into matrix A

0

0

0

0

0

0

0

0

0

33

32

31

23

22

21

13

12

11

f

f

f

f

f

f

f

f

f

A

0)()( ii lT

r pFp



Decompose A via SVD:

Solution: F is column of V corresponding to the smallest

eigenvector of A

In practice: F will be of rank 3, not 2. Correct by– SVD decomposition of F– Set smallest eigenvalue to 0– Reconstruct F’

TUDVA

TVDUF ''' 0)2(')1(''' DDD

TVDUF ''''



Input: n point correspondences ( n >= 8)– Construct homogeneous system Ax= 0 from

• x = (f11,f12, ,f13, f21,f22,f23 f31,f32, f33) : entries in F• Each correspondence give one equation• A is a nx9 matrix

– Obtain estimate F^ by SVD of A:• x (up to a scale) is column of V corresponding to the least

singular value– Enforce singularity constraint: since Rank (F) = 2

• Compute SVD of F:• Set the smallest singular value to 0: D -> D’• Correct estimate of F :

Output: the estimate of the fundamental matrix F’ Similarly we can compute E given intrinsic

parameters

0lTr pFp

TUDVA

TUDVF ˆ

TVUDF' '


Recitification

Idea: Align Epipolar Lines with Scan Lines.

Question: What type transformation?


Locating the Epipoles

pl pr

P

Ol Orel er

Pl Pr

Input: Fundamental Matrix F– Find the SVD of F– The epipole el is the column of V corresponding to the

null singular value (as shown above)– The epipole er is the column of U corresponding to the

null singular value (similar treatment as for el) Output: Epipole el and er

TUDVF

el lies on all the epipolar lines of the left image

0lTr pFp

0lTr eFp

0leF


Stereo Rectification (see Trucco)

Stereo System with Parallel Optical AxesEpipoles are at infinity

Horizontal epipolar lines

pl

pr

P

Ol Or

Xl

Xr

Pl Pr

Zl

Yl

Zr

Yr

T


pl

pr

P

Ol Or

Pl Pr

Reconstruction (3-D): Idealized


pl

pr

P

Ol Or

Pl Pr

Reconstruction (3-D): Real

See Trucco/Verri, pages 161-171


Summary Stereo Vision (Class 1)

Epipolar Geometry: Corresponding points lie on epipolar

line

Essential/Fundamental matrix: Defines this line

Eight-Point Algorithm: Recovers Fundamental matrix

Rectification: Epipolar lines parallel to scanlines

Reconstruction: Minimize quadratic distance

Stanford CS223B Computer Vision, Winter 2006 Lecture 5 Stereo I Professor Sebastian Thrun CAs: Dan...

Documents

Transcript of Stanford CS223B Computer Vision, Winter 2006 Lecture 5 Stereo I Professor Sebastian Thrun CAs: Dan...