Robotics - Projective Geometry and Camera modelhome.deib.polimi.it/gini/robot/docs/vision1.pdf ·...
Transcript of Robotics - Projective Geometry and Camera modelhome.deib.polimi.it/gini/robot/docs/vision1.pdf ·...
Robotics - Projective Geometry and Camera model
Marcello Restelli
Dipartimento di Elettronica, Informazione e BioingegneriaPolitecnico di Milano
May 2013
Inspired from Matteo Pirotta’s slides (Robotics @ Como 2013)
Camera Geometry Pin Hole Model
Outline
1 Camera Geometry
2 Pin Hole Model
2/26
Camera Geometry Pin Hole Model
Outline
1 Camera Geometry
2 Pin Hole Model
3/26
Camera Geometry Pin Hole Model
What is an image
Image
Two-dimensional brightness array: I
3× two-dimensional array: IR , IG , IB
RGB: Red, Green, Blue
others: YUV, HSV, HSL, · · ·
Ideal: I : Ω ⊂ R2 → R+
Discrete: I : Ω ⊂ N2 → R∗+e.g., Ω = [0, 639]× [0, 479] ⊂ N2
e.g., Ω = [1, 1024]× [1, 768] ⊂ N2
e.g., R∗+ = [0, 255] ⊂ N
e.g., R∗+ = [0, 1] ⊂ R
I(x , y) is the intensity
I result of 3D→ 2D projection: flat
we lose: angles, distances (lengths)
4/26
Camera Geometry Pin Hole Model
Camera
Optical system
Set of lenses to direct light
change in the direction of propagation
CCD sensor
integrate energy both
in time (exposure time)
in space (pixel size)
5/26
Camera Geometry Pin Hole Model
Thin lenses model
Thin lenses
Mathematical model
Optical axis (z)
Focal plane πf (⊥ z)
Optical center o
Parameters
f distance o, πf
Property
Parallel rays converge πf
Rays through o undeflected
z
y
o
πf
f
6/26
Camera Geometry Pin Hole Model
Rays from scene
Image from a scene point P
P = (Z ,Y )
Ray through o undeflected
Ray parallel to z cross in (−f , 0)
Similarities
Blue triangles:h
Y=
r − f
f
Green triangles:h
Y=
r
Z
Fresnel’s Law
1
Z+
1
r=
1
f
Note: Z →∞⇒ r → f
z
y
P
p
o
f
r
h
Z
Y
7/26
Camera Geometry Pin Hole Model
The image plane
Image plane πI
Plane ⊥ z at distance d
Blur Circle
If d 6= r
image of P is a circle C
Diameter of C:
φ(C) =a(d − r)
ra is the aperture
z
y
P
p
o
C
a
πI πf
f
r
d
Z
Focused image
φ(C) < pixel size
Depth of field : range [Z1,Z2] : φ(C) < pixel size
set of values that satisfies Fresnel’s law
8/26
Camera Geometry Pin Hole Model
Depth of field - Example 1
9/26
Camera Geometry Pin Hole Model
Depth of field - Example 2
The same scene - different aperture
10/26
Camera Geometry Pin Hole Model
Aperture controls depth of field
Why not make the aperture as small as possible?
Less light gets through;
Diffraction effects . . .
11/26
Camera Geometry Pin Hole Model
Aperture controls depth of field
Changing the aperture size affects depth of field
A smaller aperture increases the range in which the object is approximately in
focus;
But small aperture reduces amount of light need to increase exposure.
12/26
Camera Geometry Pin Hole Model
Outline
1 Camera Geometry
2 Pin Hole Model
13/26
Camera Geometry Pin Hole Model
Pinhole model - Definition
Hypothesis
Z a
Z f → r ∼ f
Image of P
lPo : line that join P and o
p = πI ∩ lPo
Notes
p is the image of ∀Pi ∈ lPo
lPo : interpretation line of p
Pin-Hole Model
Y
X
camera
centre
Z
principal axis
P
O
-f
p
image plane
O(I)
x
y
z
Frontal Pin-Hole Model
xp
Y
image plane
camera
centre
Z
principal axis
P
O
O(I)
f
y
X
14/26
Camera Geometry Pin Hole Model
Pinhole model - Geometry
camera centre: centre of projection or optical centre (O);
principal axis: line from the camera centre perpendicular to the image plane (z);
principal point: intersection between image plane an principal axis (O(I ));
principal plane: plane through the camera centre, parallel to the image plane.
xp
Y
image plane
camera
centre
Z
principal axis
P
O
O(I)
f
y
X
15/26
Camera Geometry Pin Hole Model
Pinhole model - Geometry (Cont’d)
Given
P(O) =[X ,Y ,Z , 1
]TP′
(O)=[X ′,Y ′,Z ′, 1
]Tp(I) =
[x , y , 1
]TProjection
y = fY
Z
x = fX
Zlook at the triangles
Note
λP(O) projects on p(I)[sX , sY , sZ , 1
]Tprojects on p(I),
∀s 6= 0
Z ′ = −f , X ′ = −fX
Z, Y ′ = −f
Y
Z
x = −X ′, y = −Y ′
z (O)
y (O)
P(O)
p(I)
O
y (I)
I
−f
y
Z
Y
16/26
Camera Geometry Pin Hole Model
Pinhole model - Frontal Camera Model
The pinhole camera model is inconvenience that the focal length f is negative. Thecoordinate system:
We will use the pinhole model as an approximation;
Put the camera centre (O) at the origin;
Put the image plane in front of the camera centre;
The camera looks up the positive z axis.
Projection equations
Compute intersection with IP of ray from P to O;
Derived using similar triangles
x = fX
Z, y = f
Y
Z
Ignoring the final image coordinate: (X ,Y ,Z)T → (f XZ, f Y
Z)T
17/26
Camera Geometry Pin Hole Model
Pinhole model - MatrixProjection equations
y = fY
Z
x = fX
ZIn matrix formx ′
y ′
w ′
=
f 0 00 f 00 0 1
1 0 0 00 1 0 00 0 1 0
XYZW
=
f 0 0 00 f 0 00 0 1 0
XYZW
p(I) = π P(O)
Define
K =
f 0 00 f 00 0 1
:
intrinsic parameters
π =[K 0
]: projection matrix
18/26
Camera Geometry Pin Hole Model
Distortions from Physical Lenses
Pinhole camera model assumes:
Perfect pinhole camera lenses;
Image x = I (p) ∈ R2 be measured in infinite accuracy;
Principal point is at the center of the image.
Physical camera lenses give us:
Distorted imaging projections;
Finite resolutions dened by the sensing devices in digital cameras;
Offset between image center and optical center.
x ′
y ′I′
x
y
I
p′
p
Ox
y
z
19/26
Camera Geometry Pin Hole Model
Pinhole model - Image coordinates - 1
Reference system on image
I: origin centered on z (O) ∩ πI
I′: origin centered top-left image
c(I′) =[cx , cy
]T: position of I in I′
Metric
I metric
I′ in pixel
c(I′) in pixel
Definition[0, 0]T (I)
≡[cx , cy
]T (I′)
: principal point
Image of the optical center (o) or z (O) ∩ πI
x ′
y ′
I′
x
y
I
20/26
Camera Geometry Pin Hole Model
Pinhole model - Image coordinates - 2
The origin of coordinates in image planes is not
at the principal point[cx , cy
]T (I′)
:
(X ,Y ,Z)T → (fX
Z+ cx , f
Y
Z+ cy )T
x ′
y ′
w ′
=
f 0 cx 0
0 f cy 0
0 0 1 0
X
Y
Z
W
p(I′) = π P(O)
x ′
y ′
I′
x
y
c(I ′)
I
21/26
Camera Geometry Pin Hole Model
Pinhole model - Image coordinates - 3 (CCD Camera)
The pinhole camera model assumes that image coordinate Euclidean (equal scales
in both axial direction);
In CCD there is the possibility of non-square pixels;
Image coordinates measured in pixel introduces unequal scale factors in each
direction.
If the number of pixels per unit distance coordinates are mx and my in the x and
y direction:
π =
mx 0 0
0 my 0
0 0 1
f 0 cx 0
0 f cy 0
0 0 1 0
where (cx , cy ) is the principal point in terms of pixel dimensions.
22/26
Camera Geometry Pin Hole Model
Pinhole model - Image coordinates - 4
Meters to pixels
Consider I′′: origin on I, in pixel
Scale meters to pixels
p(I′′)x = sxp
(I)x
p(I′′)y = syp
(I)y
sx = 1dx
, dx : width of a pixel [m]
sy = 1dy
, dy : height of a pixel [m]
sx = sy : square pixel
p(I′′) =
sx 0 0
0 sy 0
0 0 1
p(I)
Translation
p(I′) =
1 0 cx
0 1 cy
0 0 1
p(I′′)
x ′
y ′
I′
x
y
x
y
c(I ′)
I ≡ I′′
23/26
Camera Geometry Pin Hole Model
Pinhole model - Intrinsic camera matrix
Consider
p(I) =
f 0 0 0
0 f 0 0
0 0 1 0
P(O)
p(I′′) =
sx 0 0
0 sy 0
0 0 1
p(I)
p(I′) =
1 0 cx
0 1 cy
0 0 1
p(I′′)
In one step
p(I′) =
sx f 0 cx 0
0 sy f cy 0
0 0 1 0
P(O)
p(I′) = πP(O) = K[I∣∣0]P(O)
The intrinsic camera matrix
or calibration matrix
K =
fx s cx
0 fy cy
0 0 1
fx , fy : focal lenght (in pixels)
fy/fx = sy/sx = a: aspect ratio
s: skew factor
pixel not orthogonal
usually 0 in modern cameras
cx , cy : principal point (in pixel)
usually 6= half image size due to
misalignment of CCD
24/26
Camera Geometry Pin Hole Model
Camera rotation and translation
Consider
p(I′) = πP(O)
point is expressed in terms of world
coordinate frame: P(W ). W and O are
related via a rotation and a translation.
P(O) = T(O)OWP(W ) = R(O)
OW
(P(W ) −O(W )
)extrinsic camera matrix
One step
p(I′) =[K 0
] [R −RC0 1
]P(W )
π =[KR −KRC
]complete projection matrix
Note
R is R(O)OW , C is O(O)
OW (camera centre)
i.e., the position and orientation of W in O
X
Z
Y
R, t
Y
X
O Z
W
W
O
T(O)OW
p(W ) p(O)
25/26
Camera Geometry Pin Hole Model
Camera rotation and translation - 2
General pinhole camera
p(I′) =[KR −KRC
]P(W )
π = KR[I −C
]C is the camera centre in world reference
9 (11) dof: 3 (5) for K, 3 for R and 3 for C
parameters in K are called internal
parameters in R and C are called external
Other formulation
do not make the camera center explicit
p(I′) =[K 0
] [R t
0 1
]P(W )
π =[KR Kt
]t = −RC
The intrinsic parameters
K =
fx s cx
0 fy cy
0 0 1
The extrinsic parameters
R = R (φ, θ, ρ)
C = [c1, c2, c3]T
26/26