Stereo Course web page: vision.cis.udel.edu/~cv April 11, 2003 Lecture 21.
-
Upload
ariel-melton -
Category
Documents
-
view
217 -
download
1
Transcript of Stereo Course web page: vision.cis.udel.edu/~cv April 11, 2003 Lecture 21.
Announcements
• No class on Monday• Read Forsyth & Ponce, Chapter 11-
11.1 and Hartley & Zisserman, Chapter 10-10.2 on calculating the fundamental matrix for next Wednesday
• HW4 will be assigned next Wednesday and due Monday, April 28
Two-View Geometry
• The relationship of two views of a scene taken from different camera positions to one another
• Interpretations– “Stereo vision” generally means two
synchronized cameras or eyes capturing images
– Could also be two sequential views from the same camera in motion
• Assuming a static scene
Mapping Points between Images
• What is the relationship between the images x, x’ of the scene point X in two views?
• Intuitively, it depends on:– The rigid transformation between cameras
(derivable from the camera matrices P, P’)
– The scene structure (i.e., the depth of X)• Parallax: Closer points appear to move more
Example: Two-View Geometry
courtesy of F. Dellaert
x1x’1
x2x’2
x3 x’3
Is there a transformation relating the points xi to x’i ?
Epipolar Geometry
• Baseline: Line joining camera centers C, C’• Epipolar plane ¦: Defined by baseline and
scene point X
from Hartley& Zisserman
baseline
Epipolar Lines
• Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes
• Epipoles e, e’: Where baseline intersects image planes– Equivalently, the image in one view of the other camera center.
C C’
from Hartley& Zisserman
Epipolar Pencil
• As position of X varies, epipolar planes “rotate” about the baseline– This set of planes is called the epipolar pencil
• Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines
from Hartley& Zisserman
Epipolar Constraint• Camera center C and image point define ray in 3-D space
that projects to epipolar line l’ in other view (since it’s on the epipolar plane)
• 3-D point X on this ray, so image of X in other view x’ must be on l’
• In other words, the epipolar geometry defines a mapping x ! l’ of points in one image to lines in the other
from Hartley& Zisserman
C C’
x’
Example: Epipolar Lines for Converging Cameras
from Hartley & ZissermanLeft view Right view
Intersection of epipolar lines = Epipole ! Indicates location of other camera center
Special Case: Translation Parallel to Image
Plane
Note that epipolar lines are parallel and corresponding points lie on correspond-ing epipolar lines (the latter is true for all kinds of camera motions)
Special Case: Translation along Optical Axis
• Epipoles coincide at focus of expansion
• Not the same (in general) as vanishing point of scene lines
from Hartley & Zisserman
The Fundamental Matrix F• Mapping of point in one image to epipolar
line in other image x ! l’ is expressed algebraically by the fundamental matrix F
• Write this as l’ = F x• Remember that the point-on-line
relationship is given by l’ ¢ Fx = 0, which can also be written as l’T
Fx = (Fx)T l’ =
0•F is 3 x 3, rank 2 (not invertible, in contrast
to homographies)– 7 DOF (homogeneity and rank constraint take
away 2 DOF)
line point
The Fundamental Matrix F• Since x’ is on l’, by the point-on-line definition we
know that x’T l’ = 0
• Combined with l’ = F x, we can thus relate corres-ponding points in the camera pair (P, P’) to each other with the following:
x’T F x = 0
• The fundamental matrix of (P’, P) is the transpose FT
from Hartley& Zisserman
x’
Deducing Epipoles from F• For any x (besides e), l’ = F x
contains e’, so e’T (F x) = 0. Thus, e’T
F = 0 ! FT e’ = 0 – Similarly, Fe = 0
• Solve systems Fe = 0, FTe’ = 0– Same as last step of DLT (recall that there we
were trying to solve Ap = 0)– E.g., to solve Fe = 0 in Matlab:
• Take SVD of F with [U,S,V] = svd(F)• Last column of V is the answer e
x’
from Hartley & Zisserman
Cross Products & Skew-Symmetric Matrices
• If a = (a1, a2, a3), then define
• With this we have the following identity:
Building F from KnownCamera Matrices P, P’
• P, P’ determine unique fundamental matrix by
F = [e’]£P’P+
– Pseudoinverse A+ (pinv in Matlab): Like inverse for non-square matrices (i.e., AA+A = A, etc.)
• Defined as A+ = (ATA)-1AT when inverse exists
• Can also get camera matrices from F– Canonical form: P = K[Id j 0], P’ = K’[R j t]– However, there is an ambiguity up to a projective
transformation H such that we actually get PH, P’H
The Essential Matrix E• Fundamental matrix when calibration
matrices K, K’ of the two cameras are known is called the essential matrix E:
• Allows computation of camera matrices P, P’ up to a scale ambiguity
Extracting Structure
• The key aspect of epipolar geometry is its linear constraint on where a point in one image can be in the other
• By correlation-matching pixels (or features) along epipolar lines and measuring the disparity between them, we can construct a depth map (scene point depth is inversely proportional to disparity)
View 1 View 2 Computed depth mapcourtesy of P. Debevec