Epipolar Geometry class 11
description
Transcript of Epipolar Geometry class 11
![Page 1: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/1.jpg)
Epipolar Geometryclass 11
Multiple View GeometryComp 290-089Marc Pollefeys
![Page 2: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/2.jpg)
Multiple View Geometry course schedule(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16
(no class) Projective 2D Geometry
Jan. 21, 23
Projective 3D Geometry (no class)
Jan. 28, 30
Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13
Camera Calibration Single View Geometry
Feb. 18, 20
Epipolar Geometry 3D reconstruction
Feb. 25, 27
Fund. Matrix Comp. Structure Comp.
Mar. 4, 6 Planes & Homographies Trifocal Tensor
Mar. 18, 20
Three View Reconstruction
Multiple View Geometry
Mar. 25, 27
MultipleView Reconstruction
Bundle adjustment
Apr. 1, 3 Auto-Calibration Papers
Apr. 8, 10
Dynamic SfM Papers
Apr. 15, 17
Cheirality Papers
Apr. 22, 24
Duality Project Demos
![Page 3: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/3.jpg)
More Single-View Geometry
• Projective cameras and planes, lines, conics and quadrics.
• Camera calibration and vanishing points, calibrating conic and the IAC
** CPPQ T
coneQCPP T
lPT
![Page 4: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/4.jpg)
Two-view geometry
Epipolar geometry
3D reconstruction
F-matrix comp.
Structure comp.
![Page 5: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/5.jpg)
(i) Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the
corresponding point x’ in the second image?
(ii) Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i}, i=1,…,n, what are the cameras P and P’ for the two views?
(iii) Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of (their pre-image) X in space?
Three questions:
![Page 6: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/6.jpg)
The epipolar geometry
C,C’,x,x’ and X are coplanar
![Page 7: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/7.jpg)
The epipolar geometry
What if only C,C’,x are known?
![Page 8: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/8.jpg)
The epipolar geometry
All points on project on l and l’
![Page 9: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/9.jpg)
The epipolar geometry
Family of planes and lines l and l’ Intersection in e and e’
![Page 10: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/10.jpg)
The epipolar geometry
epipoles e,e’= intersection of baseline with image plane = projection of projection center in other image= vanishing point of camera motion direction
an epipolar plane = plane containing baseline (1-D family)
an epipolar line = intersection of epipolar plane with image(always come in corresponding pairs)
![Page 11: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/11.jpg)
Example: converging cameras
![Page 12: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/12.jpg)
Example: motion parallel with image plane
![Page 13: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/13.jpg)
Example: forward motion
e
e’
![Page 14: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/14.jpg)
The fundamental matrix F
algebraic representation of epipolar geometry
l'x
we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matrix F
![Page 15: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/15.jpg)
The fundamental matrix F
geometric derivation
xHx' π
x'e'l' FxxHe' π
mapping from 2-D to 1-D family (rank 2)
![Page 16: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/16.jpg)
The fundamental matrix F
algebraic derivation
λCxPλX IPP
PP'e'F
xPP'CP'l
(note: doesn’t work for C=C’ F=0)
xP
λX
![Page 17: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/17.jpg)
The fundamental matrix F
correspondence condition
0Fxx'T
The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’ in the two images 0l'x'T
![Page 18: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/18.jpg)
The fundamental matrix F
F is the unique 3x3 rank 2 matrix that satisfies x’TFx=0 for all x↔x’
(i) Transpose: if F is fundamental matrix for (P,P’), then FT is fundamental matrix for (P’,P)
(ii) Epipolar lines: l’=Fx & l=FTx’(iii) Epipoles: on all epipolar lines, thus e’TFx=0, x
e’TF=0, similarly Fe=0(iv) F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2)(v) F is a correlation, projective mapping from a point x to
a line l’=Fx (not a proper correlation, i.e. not invertible)
![Page 19: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/19.jpg)
The epipolar line geometry
l,l’ epipolar lines, k line not through e l’=F[k]xl and symmetrically l=FT[k’]xl’
lke
kl lFk
e'
(pick k=e, since eTe≠0)
leFl' l'e'Fl T
![Page 20: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/20.jpg)
Fundamental matrix for pure translation
![Page 21: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/21.jpg)
Fundamental matrix for pure translation
![Page 22: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/22.jpg)
Fundamental matrix for pure translation
e'He'F RKKH 1
0101-00000
F T1,0,0e'
example:
y'y 0Fxx'T
0]X|K[IPXx
Z
xKt]|K[IXP'x'-1
ZKt/xx'
ZX,Y,Z x/K)( -1T
motion starts at x and moves towards e, faster depending on Z
pure translation: F only 2 d.o.f., xT[e]xx=0 auto-epipolar
![Page 23: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/23.jpg)
General motion
Zt/K'xRKK'x' -1
0Hxe''x T
0x̂e''x T
![Page 24: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/24.jpg)
Geometric representation of F
2/FFF TS 2/FFF T
A AS FFF
0FxxT xx 0xFx AT
0xFx ST
Fs: Steiner conic, 5 d.o.f.Fa=[xa]x: pole of line ee’ w.r.t. Fs, 2 d.o.f.
![Page 25: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/25.jpg)
Geometric representation of F
![Page 26: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/26.jpg)
Pure planar motion
Steiner conic Fs is degenerate (two lines)
![Page 27: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/27.jpg)
Projective transformation and invariance
-1-T FHH'F̂ x'H''x̂ Hx,x̂
Derivation based purely on projective concepts
X̂P̂XHPHPXx -1
F invariant to transformations of projective 3-space
X̂'P̂XHHP'XP'x' -1
FP'P,
P'P,F
unique
not unique
canonical form
m]|[MP'0]|[IP
MmF
![Page 28: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/28.jpg)
Projective ambiguity of cameras given Fprevious slide: at least projective ambiguitythis slide: not more!
Show that if F is same for (P,P’) and (P,P’), there exists a projective transformation H so that P=HP and P’=HP’
~ ~
~ ~
]a~|A~
['P~ 0]|[IP
~ a]|[AP' 0]|[IP
A~
a~AaF
T1 avAA~
kaa~ klemma:
kaa~Fa~0AaaaF2rank
TavA-A~
k0A-A~
kaA~
a~Aa
kkIkH T1
1
v0
'P~
]a|av-A[v
0a]|[AHP' T1T1
1
kk
kkIk
(22-15=7, ok)
![Page 29: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/29.jpg)
Canonical cameras given F
F matrix corresponds to P,P’ iff P’TFP is skew-symmetric
X0,FPXP'X TT
F matrix, S skew-symmetric matrix
]e'|[SFP' 0]|[IP
000FSF
0Fe'0FSF0]|F[I]e'|[SF
TT
T
TTT
(fund.matrix=F)
Possible choice:
]e'|F][[e'P' 0]|[IP
Canonical representation:
]λe'|ve'F][[e'P' 0]|[IP T
![Page 30: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/30.jpg)
The essential matrix
~fundamental matrix for calibrated cameras (remove K)
t]R[RRtE T
0x̂E'x̂ T
FKK'E T
x'K'x̂ x;Kx̂ -1-1
5 d.o.f. (3 for R; 2 for t up to scale)
E is essential matrix if and only iftwo singularvalues are equal (and third=0)
T0)VUdiag(1,1,E
![Page 31: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/31.jpg)
Four possible reconstructions from E
(only one solution where points is in front of both cameras)
![Page 32: Epipolar Geometry class 11](https://reader036.fdocuments.net/reader036/viewer/2022062409/56814f68550346895dbd1e00/html5/thumbnails/32.jpg)
Next class: 3D reconstruction