December 12 th, 2001C. Geyer/K. Daniilidis GRASP Laboratory Slide 1 Structure and Motion from...
-
Upload
randell-casey -
Category
Documents
-
view
216 -
download
0
Transcript of December 12 th, 2001C. Geyer/K. Daniilidis GRASP Laboratory Slide 1 Structure and Motion from...
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 1
Structure and Motion from Uncalibrated Catadioptric Views
Structure and Motion from Uncalibrated Catadioptric Views
Christopher Geyerand
Kostas Daniilidis
Christopher Geyerand
Kostas Daniilidis
of the
GRASP Laboratory,University of Pennsylvania
of the
GRASP Laboratory,University of Pennsylvania
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 2
The big picture:The big picture:
Two views obtained from a parabolic catadioptric camera1 are sufficient to perform a Euclidean reconstruction from point correspondences
NO PROJECTIVE AMBIGUITY!NO PROJECTIVE AMBIGUITY!Fine print: 1. with non-varying intrinsics; if intrinsics vary, three frames are necessary; it is assumed that aspect ratio is 1 and there is no skew.
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 3
AssumptionsAssumptions
• Parabolic mirror• Aspect ratio known• Skew known• Camera and mirror aligned• Orthographic projection
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 4
OutlineOutline
• Review– Related work– What’s a catadioptric sensor?– The projection model
• Theory– Define representation of image features: circle space– Define the catadioptric fundamental matrix
• Algorithm– Computation of the fundamental matrix– Reconstruction
• Experiment• Conclusion & future work
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 5
Related workRelated work
• Nayar, Baker– theory of catadioptric image formation [ICCV
`98]
• Geyer, Daniilidis– theory of projective geometry induced in
catadioptric images[see December `01 IJCV, ECCV `00]
• Svoboda, Pajdla– description of epipolar geometry
[ECCV `98]
• Kang– non-linear self-calibration [CVPR `00]
generalcata-
dioptric
2,3 n-view
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 6
What’s a catadioptric camera?What’s a catadioptric camera?
Catadioptric camera combines a mirror and lens. We investigate cameras with a parabolic mirror and orthographically projecting lens
Mirror
CCD
Lens
sample image
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 8
Projection modelProjection model (Review)
F
P
s
R
Q
• The image of P is the orthographic projection of the intersection of the line through F and P with the parabola.
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 9
JuvN=ikjjjjjjjjcx + 2 fx
- z+"
x2+y2+z2
cy+ 2 fy
- z+"
x2+y2+z2
y{zzzzzzzz
Projection modelProjection model
• The formula for this projection is:
(where (x,y,z) is the space point and (u,v) is the image point)
Uh-oh! It’s not linear! How do we apply SFM algorithms?
(Review)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 10
Projection modelProjection model
• But wait! Neither is the perspective projection formula:JuvN=ikjjjjjcx + fx
z
cy+ fyz
y{zzzzzThat’s why we use homo-geneous coordinates invented by Möbius and Feuerbach in 1827.
FeuerbachFeuerbachMöbiusMöbius
(Review)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 11
Projection modelProjection model
• Using homogeneous coordinates we can write:
Embedding image points in a higher dimensional space linearizes the projection equation
ikjjjjjuv1y{zzzzz=l P
ikjjjjjjjjxyz1
y{zzzzzzzz
U=l PX
(Review)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 12
Projection modelProjection model
Can we perform a similar trick for Can we perform a similar trick for the parabolic projection?the parabolic projection?
Yes, represent points in a higher Yes, represent points in a higher dimensional space, lying on a dimensional space, lying on a paraboloid.paraboloid.
But first we review properties of But first we review properties of line images.line images.
(Review)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 14
A circle! But what kind?A circle! But what kind?Must have same dof as line imagesMust have same dof as line images
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 15
One thatintersects..One thatintersects..
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 16
reduces 3 dof to 2 dofreduces 3 dof to 2 dof
One that the calibratingintersects.. conic antipodallyOne that the calibratingintersects.. conic antipodally
Calibrating conic
we call ω΄
Calibrating conic
we call ω΄
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 17
• Equation of calibrating conic:
• Equation of image of absolute conic
• Same radii; ratio of radii = i; both encode the intrinsics
Projection modelProjection model
Hcx - uL2+Hcy - vL2 =H2 fL2Hcx - uL2+Hcy - vL2 =Hi 2 fL2
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 18
Circle SpaceCircle Space
Is there a cleverrepresentationof the space ofcircles?
Yes! And it usesthe paraboloid!
See Dan Pedoe’s Geometry
(Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 19
Choose a point PChoose a point P
NOT NECESSARILY THE SAME PARABOLOID
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 20
Find the cone with vertex Ptangent to the paraboloid
Find the cone with vertex Ptangent to the paraboloid
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 21
Find the plane through the inters-ection of the cone and the paraboloid
Find the plane through the inters-ection of the cone and the paraboloid
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 22
Orthographically project the intersection:a circle centered at the shadow of P
Orthographically project the intersection:a circle centered at the shadow of P
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 23
What is the result of varying P?
What is the result of varying P?
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 24
SummarySummary
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 25
Lifting of image pointsLifting of image points
p
p (lifting)~
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 26
In other words…In other words…
• We add a fourth coordinate:
p=ikjjjjjuv1y{zzzzzpŽ=
ikjjjjjjjjj
uv
u2+v2+c1
y{zzzzzzzzz
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 27
SummarySummary
• In this “circle space” we can represent line images and image points
• How do we represent circles intersecting the calibrating conic antipodally?
(Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 28
Collinear point representationscorrespond to coaxal circles
Collinear point representationscorrespond to coaxal circles
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 29
Coplanar point representations map to circles intersecting a circle antipodallyCoplanar point representations map to circles intersecting a circle antipodally
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 30
How does the plane changewhen the image plane translates?
How does the plane changewhen the image plane translates?
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 31
Circle spaceCircle space
• What is the significance of this plane?
• As it turns out it is the polar plane of the point representation
(Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 32
Absolute conicAbsolute conic (Theory)
Image of the absolute conic
Calibratingconic
Plane of antipodallyintersecting circles
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 33
Projections of space pointsProjections of space points
spacepoint
it’simage
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 34
Projections of space pointsProjections of space points
image of the absolute conic
spacepoint
it’simage
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 35
But uncalibratedBut uncalibrated
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 36
Transformations of Circle SpaceTransformations of Circle Space
Lineartransfor-mation
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 37
Step to linearizationStep to linearization (Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 38
LinearizationLinearization
• There is a 3x4 K such that
(the perspective projection)
in particular
KpŽ=
ikjjjjjjjj
xzyz1
y{zzzzzzzz
K =
ikjjjjjjjjj
1 0 0 - cx0 1 0 - cy
- cx2f
-cy2f
14f
1- 16f2+4cx2+4cy2
16f
y{zzzzzzzzz
(Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 39
LinearizationLinearization
The image of the absoluteconic is in the kernel of K
KwŽ=0
(Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 40
Two viewsTwo views
P
p
q
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 41
On to the epipolar constraintOn to the epipolar constraint
• If p and q are projections of the same point in two catadioptric images associated with K1 and K2 then
and
are the perspective projections.
q¢=K2 qŽ p¢=K1 pŽ
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 42
The epipolar constraint The epipolar constraint
The perspective projections satisfy
for some essential matrix E. Therefore
this is the epipolar constraint for parabolic catadioptric cameras.
p¢T Eq¢=0
pŽT K1T EK2 qŽ=0
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 43
Fundamental matrixFundamental matrix
• The 4x4 matrix has rank 2 and since
it satisfies
F=K1T EK2
FwŽ2=0 and FT wŽ1 =0
K2wŽ2=0 and K1wŽ1 =0
(Theory)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 45
Reconstruction algorithmReconstruction algorithm
Non-varying intrinsics:1. Compute F from >= 16 point matches
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 46
Reconstruction algorithmReconstruction algorithm
Non-varying intrinsics:1. Compute F from > 15 point matches2. Ensure F is rank 2: project to rank 2 manifold (using SVD)
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 47
Reconstruction algorithmReconstruction algorithm
Non-varying intrinsics:1. Compute F from > 15 point matches2. Ensure F is rank 2: project to rank 2 manifold (using SVD)3. Find ω by intersecting left and right nullspaces of F
~
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 48
Reconstruction algorithmReconstruction algorithm
Non-varying intrinsics:1. Compute F from > 15 point matches2. Ensure F is rank 2: project to rank 2 manifold (using SVD)3. Find ω by intersecting left and right nullspaces of F4. Compute E = K-TFK-1 (K found from ω)
~
~
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 49
Reconstruction algorithmReconstruction algorithm
Non-varying intrinsics:1. Compute F from > 15 point matches2. Ensure F is rank 2: project to rank 2 manifold (using SVD)3. Find ω by intersecting left and right nullspaces of F4. Compute E = K-TFK-1 (K found from ω)5. Project E to manifold of essential matrices (average its singular values)
~
~
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 50
Reconstruction algorithmReconstruction algorithm
Non-varying intrinsics:1. Compute F from > 15 point matches2. Ensure F is rank 2: project to rank 2 manifold (using SVD)3. Find ω by intersecting left and right nullspaces of F4. Compute E = K-TFK-1 (K found from ω)5. Project E to manifold of essential matrices (average its singular values)6. Do Euclidean reconstruction with E
~
~
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 51
Varying intrinsicsVarying intrinsics
Three views w/absolute conics ω1, ω2, ω3 compute fundamental matrices between pairs of views: F12, F23, F31
Intersect nullspaces of F12T and F31
to find ω1
Similarly for ω2 and ω3
~ ~ ~
~ ~
~
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 52
ExperimentExperiment
First viewFirst view
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 53
ExperimentExperiment
Second view same intrinsicsSecond view same intrinsics
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 54
ExperimentExperiment
Generate point correspondencesmanually or automatically
Generate point correspondencesmanually or automatically
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 56
catadioptric fundamental matrix
ConclusionConclusion
• We have defined a represent-ation of image features in a parabolic catadioptric image
• We have shown that the catadioptric fundamental matrixcatadioptric fundamental matrix yields a bilinear constraint on lifted image points
• Its left and right nullspaces encode the image of the absolute conic.
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 57
Conclusion (cont’d)Conclusion (cont’d)
• Non-varying intrinsics: in two views create Euclidean reconstruction
• Varying intrinsics: three views are sufficient to compute Euclidean structure
• Though aspect ratio 1 and skew 0 is assumed, this still beats the perspective case with same assumption where there is a projective ambiguity
December 12th, 2001 C. Geyer/K. DaniilidisGRASP Laboratory
Slide 58
Future WorkFuture Work
• Necessary and sufficient conditions on matrix F
• Critical motions• Ambiguous surfaces• Relaxing aspect ratio and skew
assumption• Hyperbolic mirrors• Thanks!