3dphoto02_projectiveeewGeometry
-
Upload
akhirul-hajri -
Category
Documents
-
view
230 -
download
0
description
Transcript of 3dphoto02_projectiveeewGeometry
-
3D photography
Kevin Kser, Marc Pollefeys
Spring 2012
http://www.cvg.ethz.ch/teaching/2012spring/3dphoto/
-
Feb 20 Introduction
Feb 27 Lecture: Geometry, Camera Model, Calibration
Mar 5 Lecture: Features, Tracking/Matching
Mar 12 Project Proposals by Students
Mar 19 Lecture: Epipolar Geometry
Mar 26 Short lecture Stereo
+ 2 papers (2 teams of 2 students)
Apr 2 Short lecture Active Ranging, Structured Light + 2 papers
Apr 9 Easter
Apr 16 Project Updates (Sechseluten in afternoon)
Apr 23 Short lecture Volumetric Modeling +2 papers
Apr 30 Short lecture Single View + 2 papers
May 7 Short lecture SfM from Photo Collections + 2 papers
May 14 Short lecture Silhouettes + 2 papers
May 21 Short lecture + 2 papers
May 28 Pentecost / White Monday
May 29-June 1 Final Demos
Schedule (tentative)
-
Projective Geometry and Camera model
Class 2
points, lines, planes conics and quadrics
transformations camera model
See http://www.cs.unc.edu/~marc/tutorial/
Book: Hartley/Zisserman: Multiple View Geometry in Computer Vision
-
Homogeneous coordinates
0=++ cbyax ( ) ( ) 0=1x,y,a,b,c T
( ) ( ) 0,~ ka,b,cka,b,c TT
Homogeneous representation of 2D points and lines
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
( ) ( ) 0,1,,~1,, kyxkyx TT
The point x lies on the line l if and only if
Homogeneous coordinates
Inhomogeneous coordinates ( )Tyx,( )T
321 ,, xxx but only 2DOF
Note that scale is unimportant for incidence relation
0=xlT
-
Points from lines and vice-versa
l'l=x
Intersections of lines
The intersection of two lines and is l l'
Line joining two points
The line through two points and is x'x=lx x'
Example
1=x
1=y
1
)1-,1,0( y
x
1
)1-,0,1( y
x
Note:
[ ] 'xx='xx
with
0-
0-
-0
x
xy
xz
yz
-
Ideal points and the line at infinity
T0,,l'l ab
Intersections of parallel lines
( ) ( )TT and ',,=l' ,,=l cbacba
Example
1=x 2=x
Ideal points ( )T0,, 21 xxLine at infinity ( )T1,0,0=l
22 l= RP
tangent vector
normal direction
-ab,( )ba,
Note that in P2 there is no distinction
between ideal points and others
-
3D points and planes
Homogeneous representation of 3D points and planes
The point X lies on the plane if and only if
0=+++ 44332211 XXXX
0=XT
The plane goes through the point X if and only if
0=XT
-
Planes from points
0
X
X
X
3
2
1
T
T
T
0=X 0=X 0,=X 321TTT
andfromSolve
(solve as right nullspace of )
T
T
T
3
2
1
X
X
X
-
Points from planes
0X
3
2
1
T
T
T
x=X M 321 XXXM
0= MT
0=X 0=X 0,=X X 321TTT
andfromSolve
(solve as right nullspace of ) X
T
T
T
3
2
1
Representing a plane by its span
-
Lines
T
T
B
AW BA
22
** 0=WW=WWTT
0001
1000W
0010
0100W*
Example: X-axis
(4dof)
Representing a line by its span
T
T
Q
PW* QP
Dual representation (planes)
(Alternative: Plcker representation, details see e.g. H&Z)
-
Points, lines and planes
TX
WM 0=M
T
W*M 0X M
W
X
*W
Common plane for point X and line W
Intersection line W and plane
-
Plcker coordinates
Elegant representation for 3D lines
jijiij ABBAl -
TL 344223141312 ,,,,, llllll5P
0B,AB,A,det| LL( ) 0=|LL (Plcker internal constraint)
(two lines intersect)
( )123413421423231442133412+++++=| llllllllllllLL
(for more details see e.g. H&Z)
(with A and B points)
-
Conics
Curve described by 2nd-degree equation in the plane
0=+++++ 22 feydxcybxyax
0=+++++ 2332312
221
2
1 fxxexxdxcxxbxax
3
2
3
1 ,x
xy
xx
x or homogenized
0=xx CT
or in matrix form
fed
ecb
dba
2/2/
2/2/
2/2/
Cwith
{ }fedcba :::::5DOF:
-
Five points define a conic
For each point the conic passes through
0=+++++ 22 feydxcyybxax iiiiii
or
( ) 0=.1,,,,, 22 ciiiiii yxyyxx ( )Tfedcba ,,,,,=c
0
1
1
1
1
1
55
2
555
2
5
44
2
444
2
4
33
2
333
2
3
22
2
222
2
2
11
2
111
2
1
c
yxyyxx
yxyyxx
yxyyxx
yxyyxx
yxyyxx
stacking constraints yields
-
Tangent lines to conics
The line l tangent to C at point x on C is given by l=Cx
l x
C
-
Dual conics
0=ll *CTA line tangent to the conic C satisfies
Dual conics = line conics = conic envelopes
1-* CC In general (C full rank):
-
Degenerate conics
A conic is degenerate if matrix C is not of full rank
TT ml+lm=C
e.g. two lines (rank 2)
e.g. repeated line (rank 1)
Tll=C
l
l
m
Degenerate line conics: 2 points (rank 2), double point (rank1)
( ) CC **Note that for degenerate conics
-
Quadrics and dual quadrics
(Q : 4x4 symmetric matrix) 0=QXXT
9 d.o.f.
in general 9 points define quadric
det Q=0 degenerate quadric
tangent plane
Q
QX=
0=Q *T
-1* Q=Q relation to quadric (non-degenerate)
-
2D projective transformations
A projectivity is an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Definition:
A mapping h:P2P2 is a projectivity if and only if there
exist a non-singular 3x3 matrix H such that for any point
in P2 reprented by a vector x it is true that h(x)=Hx
Theorem:
Definition: Projective transformation
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
x=x' Hor
8DOF
projectivity=collineation=projective transformation=homography
-
Transformation of 2D points, lines and conics
Transformation for lines
l=l' -TH
Transformation for conics
-1-TCHHC ='
Transformation for dual conics
THHCC
** ='
x=x' HFor a point transformation
-
Fixed points and lines
e=e H
(eigenvectors H =fixed points)
l=l TH
(eigenvectors H-T =fixed lines)
(1=2 pointwise fixed line)
-
Hierarchy of 2D transformations
100
2221
1211
y
x
taa
taa
100
2221
1211
y
x
tsrsr
tsrsr
333231
232221
131211
hhh
hhh
hhh
100
2221
1211
y
x
trr
trr
Projective
8dof
Affine
6dof
Similarity
4dof
Euclidean
3dof
Concurrency, collinearity,
order of contact (intersection,
tangency, inflection, etc.),
cross ratio
Parallellism, ratio of areas,
ratio of lengths on parallel
lines (e.g midpoints), linear
combinations of vectors
(centroids).
The line at infinity l
Ratios of lengths, angles.
The circular points I,J
lengths, areas.
invariants transformed
squares
-
The line at infinity
3
2
1
1
-
1t-
0ll
l
l
l
AA
AH
T-T
The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity
110
t2
1
x
x
xx AA
H
10
t111- AAHA
An affinity maps points
Lines map with an inverse, transposed H:
1-
l
1
0
0
1t-
0ll
A
AH
T-T
A
The line at infinity thus is mapped as:
-
Affine properties from images
projection rectification
APA
lll
HH
321
010
001
0,l 3321 llllT
-
Affine rectification v1 v2
l1
l2 l4
l3
l
21 vvl
211 llv
432 ll=v
-
The circular points
0
1
I i
0
1
J i
I
0
1
0
1
100
cossin
sincos
II
iseitss
tssi
y
x
S
H
The circular points I, J are fixed points under the projective transformation H iff H is a similarity
-
The circular points
circular points
0=++++ 2332312
2
2
1 fxxexxdxxx 0=+ 222
1 xx
l
T
T
0,-,1J
0,,1I
i
i
( ) ( )TT 0,1,0+0,0,1=I i
Algebraically, encodes orthogonal directions
03 x
-
Conic dual to the circular points
TT JIIJ* C
000
010
001*
C
T
SS HCHC*
*
The dual conic is fixed conic under the
projective transformation H iff H is a similarity
*
C
l
I
J
-
Angles
( )( )22
2
1
2
2
2
1
2211
++
+=cos
mmll
mlml
( )T321 ,,=l lll ( )
T
321 ,,=m mmm
Euclidean:
Projective: mmll
mlcos
**
*
CC
C
TT
T
0=ml *CT
(orthogonal)
-
Transformation of 3D points, planes and quadrics
Transformation for lines
( )l=l' -TH
Transformation for conics
( )-1-TCHHC ='
Transformation for dual conics
( )THHCC ** ='
( )x=x' H
For a point transformation
X=X' H
=' -TH
-1-TQHH=Q'
THHQ=Q' **
(cfr. 2D equivalent)
-
Hierarchy of 3D transformations
vTv
tAProjective 15dof
Affine
12dof
Similarity
7dof
Euclidean
6dof
Intersection and tangency
Parallellism of planes,
Volume ratios, centroids,
The plane at infinity
Angles, ratios of length
The absolute conic
Volume
10
tAT
10
tRT
s
10
tRT
-
The plane at infinity
1
0
0
0
1t-
0
A
AH
TT
A
The plane at infinity is a fixed plane under a projective transformation H iff H is an affinity
1. canonical position
2. contains directions
3. two planes are parallel line of intersection in 4. line // line (or plane) point of intersection in
( )T1,0,0,0=( )T0,,,=D 321 XXX
-
The absolute conic
The absolute conic is a fixed conic under the projective transformation H iff H is a similarity
04
2
3
2
2
2
1
X
XXX
The absolute conic is a (point) conic on . In a metric frame:
T321321 ,,I,, XXXXXXor conic for directions: (with no real points)
1. is only fixed as a set 2. Circle intersect in two circular points 3. Spheres intersect in
-
The absolute dual quadric
00
0I*
T
The absolute dual quadric * is a fixed conic under the projective transformation H iff H is a similarity
1. plane at infinity is the nullvector of 2. Angles:
( )( )2
*
21*
1
2
*
1
=cos
TT
T
-
Camera model
Relation between pixels and rays in space
?
-
Pinhole camera
-
Pinhole camera model
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
linear projection in homogeneous coordinates!
-
Pinhole camera model
101
0
0
Z
Y
X
f
f
Z
fY
fX
101
01
01
1Z
Y
X
f
f
Z
fY
fX
PXx
0|I)1,,(diagP ff
-
Principal point offset
T
yx
T pZfYpZfXZYX )+/,+/(),,(
principal point T
yx pp ),(
101
0
0
1
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
Z
Y
X
y
x
x
x
-
Principal point offset
101
0
0
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
y
x
x
x
camX0|IKx
1
y
x
pf
pf
K calibration matrix
-
Camera rotation and translation
( )C~-X~R=X~ cam
X10
RC-R
1
10
C~
R-RXcam
Z
Y
X
camX0|IKx XC~
-|IKRx
t|RKP C~R-t PX=x
~
-
CCD camera
1
yy
xx
p
p
K
11
y
x
y
x
pf
pf
m
m
K
-
General projective camera
1
yx
xx
p
ps
K
1
yx
xx
p
p
K
C~|IKRP non-singular
11 dof (5+3+3)
t|RKP
intrinsic camera parameters
extrinsic camera parameters
-
Radial distortion
Due to spherical lenses (cheap)
Model:
R
y
xyxKyxKyx ...))()(1(),( 442
22
1R
http://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpg straight lines are not straight anymore
-
Camera model
Relation between pixels and rays in space
?
-
Projector model
Relation between pixels and rays in space
(dual of camera)
(main geometric difference is vertical principal point offset
to reduce keystone effect)
?
-
Affine cameras
-
Action of projective camera on points
and lines
forward projection of line
( ) b+a=PB+PA=B)+P(A=X
back-projection of line
lP= T
PXlX TT ( )PX= x0;=xlT
PX=x
projection of point
-
Action of projective camera on conics
and quadrics
back-projection to cone
CPPQ Tco 0=CPXPX=CxxTTT
( )PX=x
projection of quadric
TPPQC ** 0lPPQlQT*T*T
( )lP= T
-
Image of absolute conic
-
Practical Camera Calibration
Unknown: constant camera intrinsics K
(varying) camera pose R,t
Known: 3D coordinates of chessboard corners
=> Define to be the z=0 plane ( X=[X1 X2 0 1] )
Point is mapped as x = K (r1 r2 r3 t) X x = K (r1 r2 t) [X1 X2 1]
1
yy
xx
pf
pf
K
Homography H between image and chess coordinates,
estimate from known Xi and measured xi
Method and Pictures from Zhang (ICCV99): Flexible Camera Calibration By Viewing a
Plane From Unknown Orientations
-
Direct Linear Transformation (DLT)
ii Hxx 0Hxx ii
i
i
i
i
xh
xh
xh
Hx3
2
1
T
T
T
iiii
iiii
iiii
ii
yx
xw
wy
xhxh
xhxh
xhxh
Hxx12
31
23
TT
TT
TT
0
h
h
h
0xx
x0x
xx0
3
2
1
TTT
TTT
TTT
iiii
iiii
iiii
xy
xw
yw
Tiiii wyx ,,x
0hA i
-
Direct Linear Transformation (DLT)
Equations are linear in h
0
h
h
h
0xx
x0x
xx0
3
2
1
TTT
TTT
TTT
iiii
iiii
iiii
xy
xw
yw
0AAA 321 iiiiii wyx
0hA i Only 2 out of 3 are linearly independent
(indeed, 2 eq/pt)
0
h
h
h
x0x
xx0
3
2
1
TTT
TTT
iiii
iiii
xw
yw
(only drop third row if wi0)
Holds for any homogeneous
representation, e.g. (xi,yi,1)
-
Direct Linear Transformation (DLT)
Solving for H
0Ah 0h
A
A
A
A
4
3
2
1
size A is 8x9 or 12x9, but rank 8
Trivial solution is h=09T is not interesting
1-D null-space yields solution of interest
pick for example the one with 1h
-
Direct Linear Transformation (DLT)
Over-determined solution
No exact solution because of inexact measurement
i.e. noise
0Ah 0h
A
A
A
n
2
1
Find approximate solution
- Additional constraint needed to avoid 0, e.g.
- not possible, so minimize
1h Ah0Ah
-
DLT algorithm
Objective
Given n4 2D to 2D point correspondences {xixi}, determine the 2D homography matrix H such that xi=Hxi
Algorithm
(i) For each correspondence xi xi compute Ai. Usually only two first rows needed.
(ii) Assemble n 2x9 matrices Ai into a single 2nx9 matrix A
(iii) Obtain SVD of A. Solution for h is last column of V
(iv) Determine H from h
-
Importance of normalization
0
h
h
h
0001
1000
3
2
1
iiiiiii
iiiiiii
xyxxxyx
yyyxyyx
~102 ~102 ~102 ~102 ~104 ~104 ~102 1 1
orders of magnitude difference!
Monte Carlo simulation
for identity computation based on 5 points
(not normalized normalized)
-
Normalized DLT algorithm
Objective
Given n4 2D to 2D point correspondences {xixi}, determine the 2D homography matrix H such that xi=Hxi
Algorithm
(i) Normalize points
(ii) Apply DLT algorithm to
(iii) Denormalize solution
,x~x~ ii inormiinormi xTx
~,xTx~
norm
-1
norm TH~
TH
1
norm
100
2/0
2/0
T
hhw
whw
-
Geometric distance measured coordinates
estimated coordinates
true coordinates
xxx
2H
xH,xargminH iii
d Error in one image
e.g. calibration pattern
221-H
Hx,xxH,xargminH iiiii
dd
Symmetric transfer error
d(.,.) Euclidean distance (in image)
ii
iiii
i
ii ddii
xHx subject to
x,xx,xargminx,x,H22
x,xH,
Reprojection error
-
Reprojection error
221- Hx,xxHx, dd
22 x,xxx, dd
-
Statistical cost function and Maximum Likelihood Estimation
Optimal cost function related to noise model
Assume zero-mean isotropic Gaussian noise (assume outliers removed)
22 2/xx,
22
1xPr de
22
i 2xH,x /2i
2
1H|xPr
idei
constantxH,xH|xPrlog2
i2i2
1id
Error in one image
Maximum Likelihood Estimate
2
i xH,x id
-
Statistical cost function and Maximum Likelihood Estimation
Optimal cost function related to noise model
Assume zero-mean isotropic Gaussian noise (assume outliers removed)
22 2/xx,
22
1xPr de
22i2i 2xH,xx,x /
2i2
1H|xPr
iidd
ei
Error in both images
Maximum Likelihood Estimate
2i2
i x,xx,x ii dd
-
Gold Standard algorithm
Objective
Given n4 2D to 2D point correspondences {xixi}, determine the Maximum Likelyhood Estimation of H
(this also implies computing optimal xi=Hxi)
Algorithm
(i) Initialization: compute an initial estimate using
normalized DLT or RANSAC
(ii) Geometric minimization of reprojection error:
Minimize using Levenberg-Marquardt over 9 entries of h
or Gold Standard error:
compute initial estimate for optimal {xi}
minimize cost over {H,x1,x2,,xn}
if many points, use sparse method
2i2
i x,xx,x ii dd
-
Practical Camera Calibration
Unknown: constant camera intrinsics K
(varying) camera pose R,t
Known: 3D coordinates of chessboard corners
=> Define to be the z=0 plane ( X=[X1 X2 0 1] )
Point is mapped as x = K (r1 r2 r3 t) X x = K (r1 r2 t) [X1 X2 1]
1
yy
xx
pf
pf
K
Homography H between image and chess coordinates,
estimate from known Xi and measured xi
Method and Pictures from Zhang (ICCV99): Flexible Camera Calibration By Viewing a
Plane From Unknown Orientations
-
Practical Camera Calibration
1
yy
xx
pf
pf
K
H has the shape:
H = K(r1 r2 t) K1H = (r1 r2 t)
With rotation matrix constraints
r1Tr2 = 0
r1Tr1=r2
Tr2=1
Squaring yields 2 constraints on IAC (KKT)-1
h1T KTK1h2 = 0
hT1KTK1h1 = h
T2K
TK1h2
n images provide 2n constraints on IAC
More measurements => better estimate of IAC !
-
Practical Camera Calibration
1
yy
xx
pf
pf
K
Given an estimate for the IAC, factorize (KKT)-1
Factorize
[Obtain Poses]
Clamp
Optimize
-
Reminder
Project presentation in 2 weeks
Find team mate / decide topic soon
Limited number of kinects: first come, first served !