Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective...
-
Upload
camron-perry -
Category
Documents
-
view
220 -
download
1
Transcript of Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective...
![Page 1: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/1.jpg)
Two-views geometryOutline
Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry:
Homography Epipolar geometry, the essential matrix Camera calibration, the fundamental matrix
3D reconstruction (Stereo algorithms) next week.
Many of the slides are courtesy of Prof. Ronen Basri
![Page 2: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/2.jpg)
3-D Scene
u
u’
What can 2 images tell us about ….Faugeras et. al. ECCV 92
Objective
![Page 3: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/3.jpg)
3-D Scene
u
u’
Study the mathematical relations between corresponding image points.
“Corresponding” means originated from the same 3D point.
Objective
![Page 4: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/4.jpg)
World Cup 66: England-Germany
![Page 5: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/5.jpg)
World Cup 66: Second View
![Page 6: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/6.jpg)
World Cup 66: England-Germany
Conclusion: no goal (missing 3 inches)
(Reid and Zisserman, “Goal-directed video metrology”)
![Page 7: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/7.jpg)
Camera Obscura
"Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January 24, 1544, and later he used this illustration of the event in his book De Radio Astronomica et Geometrica, 1545. It is thought to be the first published illustration of a camera obscura..." Hammond, John H., The Camera Obscura, A Chronicle
![Page 8: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/8.jpg)
A few words about Cameras
Camera obscura dates from 15th century First photograph on record shown in the book - 1822 Basic abstraction is the pinhole camera Current cameras contain a lens and a recording device
(film, CCD, CMOS) The human eye functions very much like a camera
![Page 9: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/9.jpg)
Ideal LensesLens acts as a pinhole (for 3D points at the focal depth).
![Page 10: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/10.jpg)
Regular LensesE.g., the cameras in our lab.
To learn more on lens-distortion see Hartley & Zisserman Sec. 7.4 p.189.Not part of this class.
![Page 11: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/11.jpg)
Pinhole Camera
![Page 12: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/12.jpg)
Single View Geometry
f
X
P Y
Z
x
p y
f
∏x
p y
f
![Page 13: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/13.jpg)
Notation
O – Focal center π – Image plane Z – Optical axis f – Focal length
![Page 14: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/14.jpg)
Projection
x y f
X Y Z
f
x
y
Z
X
Y
![Page 15: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/15.jpg)
Perspective Projection
f Xx
Zf Y
yZ
Origin (0,0,0) is the Focal center X,Y (x,y) axis are along the image axis (height / width). Z is depth = distance along the Optical axis f – Focal length
![Page 16: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/16.jpg)
Orthographic Projection
•Projection rays are parallel•Image plane is fronto-parallel(orthogonal to rays)
•Focal center at infinity
x X
y Y
![Page 17: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/17.jpg)
Scaled Orthographic ProjectionAlso called “weak perspective”
x sX
y sY
0
fs
Z
![Page 18: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/18.jpg)
Pros and Cons of Projection Models Weak perspective has simpler math.
Accurate when object is small and distant. Useful for object recognition.
Pinhole perspective much more accurate. Used in structure from motion.
When accuracy really matters (SFM), we must model the real camera (exact imaging processes): Perspective projection, calibration parameters (later), and
all other issues (radial distortion).
![Page 19: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/19.jpg)
Two-views geometryOutline
Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry:
Homography Epipolar geometry, the essential matrix Camera calibration, the fundamental matrix
3D reconstruction from two views (Stereo algorithms)
Hartley & Zisserman: Sec. 2 Proj. Geom. of 2D.Sec. 3 Proj. Geom. of 3D.
![Page 20: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/20.jpg)
Reading
Hartley & Zisserman:
Sec. 2 Proj. Geo. of 2D:• 2.1- 2.2.3 point lines in 2D• 2.3 -2.4 transformations • 2.7 line at infinity
Sec. 3 Proj. Geo. of 3D. • 3.1 – 3.2 point planes & lines. • 3.4 transformations
![Page 21: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/21.jpg)
Euclidean Geometry is good for
questions like:
what objects have the same shape (= congruent)
Same shapes are related by rotation and translation
Why projective Geometry (Motivation)
![Page 22: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/22.jpg)
Why Projective Geometry (Motivation) Answers the question what appearances
(projections) represent the same shape
Same shapes are related by a projective transformation
![Page 23: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/23.jpg)
Where do parallel lines meet?
Parallel lines meet at the horizon (“vanishing line”)
Why Projective Geometry (Motivation)
![Page 24: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/24.jpg)
Coordinates in Euclidean Space
0 1 2 3 ∞
Not in space
![Page 25: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/25.jpg)
Coordinates in Projective Line P1
-1 0 1 2 ∞
k(0,1)
k(1,0)
k(2,1)k(1,1)k(-1,1)
Points on a line P1 are represented as rays from origin in 2D,Origin is excluded from space
“Ideal point”
![Page 26: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/26.jpg)
Coordinates in Projective Plane P2
k(0,0,1)
k(x,y,0)
k(1,1,1)
k(1,0,1)
k(0,1,1)
“Ideal point”
Take R3 –{0,0,0} and look at scale equivalence class (rays/lines trough the origin).
z
y
x
z
y
x
![Page 27: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/27.jpg)
Projective Line vs. the Real Line
-1 0 1 2 ∞
k(0,1)
k(1,0)
k(2,1)k(1,1)k(-1,1)
“Ideal point”
Symbol R P1
Space The real line R^2 – {0,0}
Objects (points) points Equivalence classes (2D “rays”)
Realization Intersection with line y=1
![Page 28: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/28.jpg)
Projective Plane vs Euclidian plane
k(0,0,1)
k(x,y,0)
k(1,1,1)
k(1,0,1)
k(0,1,1)“Ideal line”
Symbol R2 P2
Space The real plane R3 – {0,0,0}
Objects (points) point Equivalence classes (3D rays)
Realization Intersection with plane z=1
![Page 29: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/29.jpg)
2D Projective Geometry: Basics A point:
A line:
we denote a line with a 3-vector
Line coordinates are homogenous
Points and lines are dual: p is on l if
Intersection of two lines/points
2 2( , , ) ( , )T Tx yx y z P
z z
0 ( ) ( ) 0x y
ax by cz a b cz z
0Tl p
1 2 ,l l 1 2p p
( , , )Ta b c
ll
![Page 30: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/30.jpg)
Cross Product
1 2 1 2 1 2
1 2 1 2 1 2
1 2 1 2 1 2
x x y z z y
y y z x x z
z z x y y x
0T Tw u v w u w v
Every entry is a determinant of the two other entries
w Area of parallelogram bounded by u and v
Hartley & Zisserman p. 581
![Page 31: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/31.jpg)
Cross Product in matrix notation [ ]x
0
0
0
xy
xz
yz
x
tt
tt
tt
t1 2 1 2 1 2
1 2 1 2 1 2
1 2 1 2 1 2
x x y z z y
y y z x x z
z z x y y x
0
0
0
x y z z y
y z x z x
z x y y x
t x t z t y t t x
t y t x t z t t y
t z t y t x t t z
Hartley & Zisserman p. 581
ptpt x
![Page 32: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/32.jpg)
Example: Intersection of parallel lines
00
)(
0
)(
)(
2122
21
21 a
b
a
b
cccca
ccb
ll
1 2 1 2 1 2
1 2 1 2 1 2
1 2 1 2 1 2
x x y z z y
y y z x x z
z z x y y x
Q: How many ideal points are there in P2?A: 1 degree of freedom family – the line at infinity
),,( ),,( 2211 cbalcbal
![Page 33: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/33.jpg)
Projective Transformations
u
u’
![Page 34: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/34.jpg)
Transformations of the projective line
dycx
byax
y
xG dc
ba
:
1/
//
dc
dbda
dc
ba
11'
''
1 xc
bxax G
Given a 2D linear transformation G:R2 R2 Study the induced transformation on the Equivalents classes.
1'
''
xc
bxax G
On the realization y=1 we get
![Page 35: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/35.jpg)
Properties:1'
''
xc
bxax T
dc
baT
1. Invertible (T-1 exists) 2. Composable (To G is a projective transformation)3. Closed under composition
• Has 4 parameters • 3 degrees of freedom • Defined by 3 points
TT Every point defines 1 constraint
![Page 36: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/36.jpg)
Transformations of the projective line
1P
Pencil of raysPerspective mapping
A perspective mapping is a projective transformation T:P1 P1
Perceptivity is a special projective mapping. Hartley & Zisserman p. 632Lines connecting corresponding points are “concurrent”
![Page 37: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/37.jpg)
Ideal points and projective transformations
Projective transformation can map ∞ to a real point
![Page 38: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/38.jpg)
Plane Perspective
2P
![Page 39: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/39.jpg)
2D Projective Transformation
Projectivity: An invertible mapping h:P2 P2
S.T:
Homography. A 3x3 (non singular) invertible matrix acting on homogenous 3-vectores.
Collineation A transformations that map lines to lines
Hartley & Zisserman p. 32
line aon lie )(),(),( line aon lie ,, 321321 xhxhxhxxx
4 names 3 definitions
![Page 40: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/40.jpg)
2D Projective Transformation
H is defined up to scale
9 parameters 8 degrees of freedom Determined by 4 corresponding points
how does H operate on lines?
0
1: 0 ( )( ) 0T T Tl H l l p l H Hp
Hartley & Zisserman p. 32
HH
![Page 41: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/41.jpg)
Plane Perspective
2P
This mapping clearly maps lines to lines
![Page 42: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/42.jpg)
Plane Perspective acting on conics
2P
Hartley & Zisserman p. 30 & 36Not part of this class
![Page 43: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/43.jpg)
Rotation:Translation:
Hierarchy of Transformations
Rigid (Isometry)
Similarity
Affine
Projective
Scale
Hartley & Zisserman p. Sec. 2.4
![Page 44: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/44.jpg)
cos sin, , det 1
sin cosTR R R I R
Rotation:
Translation:x
y
tt
t
2 2, 1, (2)a b
R a b R SOb a
Euclidean Transformations (Isometries)
q Rp t
![Page 45: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/45.jpg)
Hierarchy of Transformations
Isometry (Euclidean),
Similarity,
Affine, general linear
Projective,
0 1
R t
,0 1
a bsR tsR
b a
, (2)0 1
A tA GL
(3) : , 0H GL q Hp
![Page 46: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/46.jpg)
Invariants
Length Area Angles Parallelism
Isometry √ √ √ √
Similarity ××
(Scale)√ √
Affine × × × √
Projective × × × ×
![Page 47: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/47.jpg)
Two-views geometryOutline
Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry:
Homography Epipolar geometry, the essential matrix Camera calibration, the fundamental matrix
3D reconstruction from two views (Stereo algorithms)
![Page 48: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/48.jpg)
Two View Geometry When a camera changes position and
orientation, the scene moves rigidly relative to the camera
3-D Scene
u
u’
X
Y
Z
d
p
Rotation + translation
![Page 49: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/49.jpg)
3-D Scene
Rotation + translation
u
u’
X
Y
Z
d
p
Objective:
find formulas that links corresponding points
![Page 50: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/50.jpg)
Two View Geometry (simple cases) In two cases this results in homography:
1. Camera rotates around its focal point
2. The scene is planar
Then: Point correspondence forms 1:1mapping depth cannot be recovered
![Page 51: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/51.jpg)
Camera Rotation
' , 0
( )
'' ' ( ' ')
' ( ' )'
P RP t
Zp P P p
f
Zp P P p
f
Zp Rp p Rp
Z
(R is 3x3 non-singular)
![Page 52: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/52.jpg)
Planar Scenes
IntuitivelyA sequence of two perspectivities
Algebraically
Need to show:
( )
1'
1, '
' ,'
T
TT
T
n P d aX bY cZ d
n PP RP t RP t R tn P
d d
H R tn P HPd
Zp Hp
Z
Scene
Camera 1
Camera 2
Hpp '
![Page 53: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/53.jpg)
Summary: Two Views Related by HomographyTwo images are related by homography:
One to one mapping from p to p’ H contains 8 degrees of freedom Given correspondences, each point determines
2 equations 4 points are required to recover H Depth cannot be recovered
' ,'
Zp Hp
Z
![Page 54: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/54.jpg)
The General Case: Epipolar Lines
epipolar lineepipolar line
![Page 55: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/55.jpg)
Epipolar Plane
epipolar plane
epipolar lineepipolar lineepipolar lineepipolar line
BaselineBaseline
PP
OO O’O’
![Page 56: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/56.jpg)
Epipole Every plane through the baseline is an epipolar
plane It determines a pair of epipolar lines (one in each image)
Two systems of epipolar lines are obtained Each system intersects in a point, the epipole The epipole is the projection of the center of the
other camera
epipolar planeepipolar linesepipolar linesepipolar linesepipolar lines
BaselineBaselineOO O’O’
![Page 57: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/57.jpg)
Example
![Page 58: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/58.jpg)
Epipolar Lines
epipolar plane
epipolar lineepipolar lineepipolar lineepipolar line
BaselineBaseline
PP
OO O’O’
To define an epipolar plane, we define the plane through the two camera centers O and O’ and some point P. This can be written algebraically (in some worldcoordinates as follows:
' ' 0T
OP OO O P
![Page 59: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/59.jpg)
Essential Matrix (algebraic constraint between corresponding image points) Set world coordinates around the first camera
What to do with O’P? Every rotation changes the observed coordinate in the second image
We need to de-rotate to make the second image plane parallel to the first
Replacing by image points
' ' 0T
OP OO O P
' 0TP t RP
, 'P OP t OO
' 0Tp t Rp Other derivations Hartley & Zisserman p. 241
![Page 60: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/60.jpg)
Essential Matrix (cont.)
Denote this by:
Then
Define
E is called the “essential matrix”
t p t p
' ' 0T Tp t Rp p t Rp
E t R
' 0Tp Ep
' 0Tp t Rp
![Page 61: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/61.jpg)
Properties of the Essential Matrix E is homogeneous Its (right and left) null spaces are the two epipoles 9 parameters Is linear E, E can be recovered up to scale using 8 points. Has rank 2.
The constraint detE=0 7 points suffices In fact, there are only 5 degrees of freedom in E,
3 for rotation 2 for translation (up to scale), determined by epipole
0 ': l plpE t
' 0Tp Ep
e) trough lines ( : : 12 all PPEThus
![Page 62: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/62.jpg)
BackgroundThe lens optical axis does not coincide with
the sensor
We model this using a 3x3 matrix the Calibration matrix
Camera Internal Parameters or Calibration matrix
![Page 63: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/63.jpg)
Camera Calibration matrix
The difference between ideal sensor ant the real one is modeled by a 3x3 matrix K:
(cx,cy) camera center, (ax,ay) pixel dimensions, b skew
We end with
0
0 0 1
x x
y y
a b c
K a c
q Kp
![Page 64: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/64.jpg)
Fundamental Matrix
F, is the fundamental matrix.
1 1
1
1
' 0 ( ) ( ') 0
( ) ' 0
T T
T T
T
p Ep K q E K q
q K EK q
F K EK
![Page 65: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/65.jpg)
Properties of the Fundamental Matrix F is homogeneous Its (right and left) null spaces are the two epipoles 9 parameters Is linear F, F can be recovered up to scale using 8 points. Has rank 2.
The constraint detF=0 7 points suffices
e) trough lines ( : 12 all PPF
0'Fpp t
![Page 66: Two-views geometry Outline Background: Camera, Projection Necessary tools: A taste of projective geometry Two view geometry: Homography Epipolar geometry,](https://reader031.fdocuments.net/reader031/viewer/2022032200/56649f555503460f94c78f10/html5/thumbnails/66.jpg)
Homography Epipolar
Form
Shape One-to-one map Concentric epipolar lines
D.o.f. 8 8/5 F/E
Eqs/pnt 2 1
Minimal configuration
4 5+ (8, linear)
Depth No Yes, up to scale
Scene Planar
(or no translation)
3D scene
Two-views geometry Summary:
0'Fpp tHpp '