Multiple View Geometry

THE GEOMETRY OF MULTIPLE VIEWS

Reading: Chapter 10.

• Epipolar Geometry

• The Essential Matrix

• The Fundamental Matrix

• The Trifocal Tensor

• The Quadrifocal Tensor

Epipolar Geometry

• Epipolar Plane

• Epipoles

• Epipolar Lines

• Baseline

Epipolar Constraint

• Potential matches for p have to lie on the corresponding epipolar line l’.

• Potential matches for p’ have to lie on the corresponding epipolar line l.

Epipolar Constraint: Calibrated Case

Essential Matrix(Longuet-Higgins, 1981)

Properties of the Essential Matrix

• E p’ is the epipolar line associated with p’.

• ETp is the epipolar line associated with p.

• E e’=0 and ETe=0.

• E is singular.

• E has two equal non-zero singular values (Huang and Faugeras, 1989).

T

T

Epipolar Constraint: Small MotionsTo First-Order:

Pure translation:Focus of Expansion

Epipolar Constraint: Uncalibrated Case

Fundamental Matrix(Faugeras and Luong, 1992)

Properties of the Fundamental Matrix

• F p’ is the epipolar line associated with p’.

• FT p is the epipolar line associated with p.

• F e’=0 and FT e=0.

• F is singular.

T

T

The Eight-Point Algorithm (Longuet-Higgins, 1981)

|F | =1.

Minimize:

under the constraint

2

Non-Linear Least-Squares Approach (Luong et al., 1993)

Minimize

with respect to the coefficients of F , using an appropriate rank-2 parameterization.

Problem with eight-point algorithm

linear least-squares: unit norm vector F yielding smallest residual

What happens when there is noise?

The Normalized Eight-Point Algorithm (Hartley, 1995)

• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is sqrt(2) pixels: q = T p , q’ = T’ p’.

• Use the eight-point algorithm to compute F from thepoints q and q’ .

• Enforce the rank-2 constraint.

• Output T F T’.T

i i i i

i i

Weak-calibration Experiments

Epipolar geometry example

Example: converging cameras

courtesy of Andrew Zisserman

Trinocular Epipolar Constraints

These constraints are not independent!

Trinocular Epipolar Constraints: Transfer

Given p and p , p can be computed

as the solution of linear equations.

1 2 3

problem for epipolar transfer in trifocal plane!

image from Hartley and Zisserman

Trinocular Epipolar Constraints: Transfer

There must be more to trifocal geometry…

Trifocal Constraints

Trifocal Constraints

All 3x3 minorsmust be zero!

Calibrated Case

Trifocal Tensor

Trifocal Constraints

Uncalibrated Case

Trifocal Tensor

Trifocal Constraints: 3 Points

Pick any two lines l and l through p and p .

Do it again.2 3 2 3

T( p , p , p )=01 2 3

Properties of the Trifocal Tensor

Estimating the Trifocal Tensor

• Ignore the non-linear constraints and use linear least-squares

• Impose the constraints a posteriori.

• For any matching epipolar lines, l G l = 0.

• The matrices G are singular.

• They satisfy 8 independent constraints in theuncalibrated case (Faugeras and Mourrain, 1995).

2 1 3T i

1i

For any matching epipolar lines, l G l = 0. 2 1 3T i

The backprojections of the two lines do not define a line!

108 putative matches 18 outliers

88 inliers 95 final inliers

(26 samples)

(0.43)(0.23)

(0.19)

TrifocalTensor

Example

courtesy of Andrew Zisserman

additional line matches

TrifocalTensor

Example

images courtesy of Andrew Zisserman

Transfer: trifocal transfer

doesn’t work if l’=epipolar line

image courtesy of Hartley and Zisserman

(using tensor notation)

Image warping using T(1,2,N)(Avidan and Shashua `97)

Multiple Views (Faugeras and Mourrain, 1995)

Two Views

Epipolar Constraint

Three Views

Trifocal Constraint

Four Views

Quadrifocal Constraint(Triggs, 1995)

Geometrically, the four rays must intersect in P..

Quadrifocal Tensorand Lines

Quadrifocal tensor

determinant is multilinear

thus linear in coefficients of lines !

There must exist a tensor with 81 coefficients containing all possible combination of x,y,w coefficients for all 4

images: the quadrifocal tensor

wi

wi

yi

yi

xi

xii

Ti MlMlMl Ml

Twi

yi

xi lll ][

s

r

q

p

pqrs

MMMM

Q

4

3

2

1

det