Triangulation and Multi-View Geometry

Class 9

Read notes Section 3.3, 4.3-4.4, 5.1(if interested, read Triggs’s paper on MVG using tensor notation, see

http://www.unc.edu/courses/2004fall/comp/290/089/papers/Triggs-ijcv95.pdf

)

Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do

Step 3.1 select minimal sample (i.e. 7 matches)

Step 3.2 compute solution(s) for F

Step 3.3 determine inliers

until (#inliers,#samples)<95%

samples#7)1(1

matches#inliers#

#inliers 90%

80%

70% 60%

50%

#samples

5 13 35 106 382

Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches

(generate hypothesis)

(verify hypothesis)

Automatic computation of F

Abort verification early

Given n samples and an expected proportion of inliers p, how likely is it that I have observed less than T inliers?abort if P<0.02 (initial sample most probably contained outliers)

(inspired from Chum and Matas BMVC2002)

OOOOOIOOIOOOOOIOOOOOOOIOOOOOIOIOOOOOOOOOIOIIIIOIIIOIOIIIIOOIOIIIIOIOIOIIIIIIII

(use normal approximation to binomial)

To avoid problems this requires to also verify at random!

(but we already have a random sampler anyway)

restrict search range to neighborhood of epipolar line (e.g. 1.5 pixels)

relax disparity restriction (along epipolar line)

Finding more matches

• Degenerate cases• Planar scene• Pure rotation

• No unique solution• Remaining DOF filled by noise• Use simpler model (e.g. homography)

• Solution 1: Model selection (Torr et al., ICCV´98, Kanatani, Akaike)

• Compare H and F according to expected residual error (compensate for model complexity)

• Solution 2: RANSAC• Compare H and F according to inlier count

(see next slide)

Degenerate cases:

RANSAC for (quasi-)degenerate cases

• Full model (8pts, 1D solution)

Sample for out of plane points among outliers

closest rank-6 of Anx9 for all plane inliers

(accept inliers to solution F)

(accept inliers to solution F1,F2&F3)

• Planar model (6pts, 3D solution)

Accept if large number of remaining inliers• Plane+parallax model (plane+2pts)

80% in plane 2% out plane18% outlier

• Absence of sufficient features (no texture)• Repeated structure ambiguity

(Schaffalitzky and Zisserman, BMVC‘98)

• Robust matcher also finds Robust matcher also finds support for wrong hypothesissupport for wrong hypothesis• solution: detect repetition solution: detect repetition

More problems:

RANSAC for ambiguous matching

• Include multiple candidate matches in set of potential matches

• Select according to matching probability (~ matching score)

• Helps for repeated structures or scenes with similar features as it avoids an early commitment, but also useful in general

(Tordoff and Murray ECCV02)

geometric relations between two views is fully

described by recovered 3x3 matrix F

two-view geometry

Triangulation (finally!)

C1x1

L1

x2

L2

X

C2

Triangulation

- calibration

- correspondences

Triangulation• Backprojection

• Triangulation

Iterative least-squares

• Maximum Likelihood Triangulation

C1 x1L1

x2

L2

X

C2

Optimal 3D point in epipolar plane

• Given an epipolar plane, find best 3D point for (m1,m2)

m1

m2

l1 l2l1m1

m2l2

m1´

m2´

Select closest points (m1´,m2´) on epipolar lines

Obtain 3D point through exact triangulationGuarantees minimal reprojection error (given this epipolar plane)

Non-iterative optimal solution

• Reconstruct matches in projective frame by minimizing the reprojection error

• Non-iterative methodDetermine the epipolar plane for reconstruction

Reconstruct optimal point from selected epipolar plane Note: only works for two views

2222

11 ,, MPmMPm DD

(Hartley and Sturm, CVIU´97)

2222

11 ,, lmlm DD (polynomial of degree 6)

m1

m2

l1 l2

3DOF

1DOF

Backprojection

• Represent point as intersection of row and column

Useful presentation for deriving and understanding multiple view geometry(notice 3D planes are linear in 2D point coordinates)

• Condition for solution?

Multi-view geometry

(intersection constraint)

(multi-linearity of determinants)

(= epipolar constraint!)

(counting argument: 11x2-15=7)

Multi-view geometry

(multi-linearity of determinants)

(= trifocal constraint!)

(3x3x3=27 coefficients)

(counting argument: 11x3-15=18)

Multi-view geometry

(multi-linearity of determinants)

(= quadrifocal constraint!)

(3x3x3x3=81 coefficients)

(counting argument: 11x4-15=29)

Next class: rectification and stereo

image I(x,y) image I´(x´,y´)Disparity map D(x,y)

(x´,y´)=(x+D(x,y),y)