Camera - Πανεπιστήμιο ΑιγαίουOrthographic projection A Modern Approach Slides by...
23
13/5/2009 1 Camera Calibration Camera and Photography 1000s: – 1030: Alhazan made the first working model of Camera Obscura. 1600s : – Robert Boyle observed that silver chloride changed color on exposure to “air”. 1700s: – 1727: Johann Heinrich Schulze discovered some liquids changed color on exposure to light. 1800s: – Thomas Wedgwood captured images but the silhouette wont stay. – 1827: Niépce produced the first photo which required an exposure time of 8 hours!. Satya Prakash Mallick
Transcript of Camera - Πανεπιστήμιο ΑιγαίουOrthographic projection A Modern Approach Slides by...
13/5/2009
1
Camera Calibration
Camera and Photography1000s:
– 1030: Alhazan made the first working model of Camera gObscura.
1600s :– Robert Boyle observed that silver chloride changed color on
exposure to “air”. 1700s:
– 1727: Johann Heinrich Schulze discovered some liquids changed color on exposure to light.g p g
1800s: – Thomas Wedgwood captured images but the silhouette wont
stay.– 1827: Niépce produced the first photo which required an
exposure time of 8 hours!. Satya Prakash Mallick
13/5/2009
2
Camera and Photography
1800s:– 1839: Daguerre reduced the time of exposure to 30 mins. – August 19, 1839: French Government bought Daguerreotype and made it public.
– The term photography was coined in the year 1839 by Sir John Herschel.
– William Henry Talbot came up with Calotype. This introduced the concept of “negatives” in photographyM h B B d Ph hi d f A i Ci il W– Mathew B. Brady: Photographic documents of American Civil War.
– George Eastman formed a company called Kodak which brought cameras to the domain of a common man.
Satya Prakash Mallick
Cameras and Photography
1900s:1900: Kodak introduced Brownie cameras which cost $1 ( no I didn’t– 1900: Kodak introduced Brownie cameras which cost $1 ( no I didn’t miss any zeros! ) each.
– 1907: Lumiere Brothers introduced a new 35mm standard in color photography.
– 1924: Introduction of Leica; the photojournalistic standard camera. – 1947: Edwin Herbert Land introduced Polaroid photography. – 1963: D. Gregg, made a crude form of digital camera.– Nov. 1972: Image processing research community got it’s most used test image: Ms Lena Soderberg (“They” haven’t sued us yet!)test image: Ms. Lena Soderberg. ( They haven t sued us yet!)
– 1986: Kodak created a sensor that could record 1.4 Mega pixels. – 1990s : First commercial digital cameras started appearing in the market.
Satya Prakash Mallick
13/5/2009
3
Coordinate SystemsWorld Coordinate System: It’s a known reference coordinate system with respect toreference coordinate system with respect to which we calibrate the camera. Camera Coordinate System: It’s a coordinate system with its origin at the optical center of the camera. Pixel Coordinate SystemPixel Coordinate System
Satya Prakash Mallick
Coordinate Systems
13/5/2009
4
Camera Calibration
What is the goal of camera calibration?– Estimate the extrinsic and intrinsic camera parameters.
How?– By establishing a set of correspondences between point features in the world (Xw, Yw, Zw) and theirpoint features in the world (Xw, Yw, Zw) and their projections on the image (xim, yim)
Calibration pattern: an exampleTwo orthogonal grids.Equally spaced black squaresEqually spaced black squares.Assume that the world reference frame is centered at the lower left corner of the left grid, with axes parallel to the three directions identified by e ee i e io i e i ie ythe calibration pattern.
13/5/2009
5
Calibration pattern: example
Given the size of the planes, the number of squares etc (i e all known by construction) the coordinatesetc. (i.e., all known by construction), the coordinates of each vertex can be computed in the world reference frame using trigonometry.The projection of the vertices on the image can be found by intersecting the edge lines of the corresponding square sides (or through corner detection).
Pinhole camerasAbstract camera model ‐ box with a small hole in it
A Modern ApproachSlides by D.A. Forsyth
13/5/2009
6
Distant objects are smaller
A Modern ApproachSlides by D.A. Forsyth
The equation of projection
A Modern ApproachSlides by D.A. Forsyth
13/5/2009
7
Camera ModelsPerspective:
xfx ′=' yfy ′'Affine:
1. Weak Perspective:
2 Orthographic:
zfx =
zfy =
ozxfx ′='
ozyfy ′='
2. Orthographic:
Note: We will deal only with perspective projection xx =' yy ='
Satya Prakash Mallick
Weak perspective
A Modern ApproachSlides by D.A. Forsyth
13/5/2009
8
Orthographic projection
A Modern ApproachSlides by D.A. Forsyth
Methods(1) Direct parameter calibration.
Di t f th i t i i d t i i tDirect recovery of the intrinsic and extrinsic camera parameters.
(2) Camera parameters through the projection matrix(2.1) Estimate the elements of the projection matrix.(2.2) Compute the intrinsic/extrinsic as closed‐form functions of the entries of the projection matrix.
Prof. Bebis
13/5/2009
9
Review of basic equations
From world coordinates to camera coordinates
For simplicity, we will replace ‐T’ with T
Prof. Bebis
Review of basic equations
From camera coordinates to pixel coordinates:
Relating world coordinates to pixel coordinates:
Prof. Bebis
13/5/2009
10
Method 1: Direct Parameter Calibration
Independent intrinsic parametersThe fi e i t i i a a ete f o a d o a e ot– The five intrinsic parameters f, sx, sy, ox, and oy are not independent
– We can define the following four independent parameters:
Prof. Bebis
Method 1: Direct Parameter Calibration Main steps
(1) Assuming that ox and oy are known, estimate all the remaining parameters.
(2) Estimate ox and oy
Prof. Bebis
13/5/2009
11
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Problem statementProblem statement
Assuming that ox and oy are known, compute f x , α , R, and T from N corresponding pairs of points
and (xi , yi ), i = 1, . . . , N.
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T (cont’d)
To simplify notation, set (xim‐ ox , yim‐ oy) = (x, y)o si p i y otatio , set (xim ox , yim oy) (x, y)
Since the above two equations have the same denominator, we get the following equation:
Prof. Bebis
13/5/2009
12
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Derive a system of equationsEa h ai of o e o di oi t lead to a– Each pair of corresponding points leads to an equation:
– Dividing by f y, we ca re‐write the above equations as follows:
where
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
– N corresponding points lead to a homogeneous system of N equations with 8homogeneous system of N equations with 8 unknowns:
Prof. Bebis
13/5/2009
13
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Solving the system
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Determine α and | γ |
Prof. Bebis
13/5/2009
14
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Determine r21, r22, r23, r11, r12, r13, Ty, TxWe a dete i e the abo e a a ete u to a– We can determine the above parameters, up to an unknown common sign.
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Determine r31, r32, r3331 32 33– Can be estimated as the cross product of R1 and R2:
Th f R l d f d ( h f R– The sign of R3 is already fixed (the entries of R3remain unchanged if the signs of all the entries of R1 and R2 are reversed).
Prof. Bebis
13/5/2009
15
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Ensuring the orthogonality of Rg g y– The computation of R does not take into account explicitly
the orthogonality constraints.– The estimate of R cannot be expected to be orthogonal:– We can ʺenforceʺ the orthogonality on using SVD:
l h I– Replace D with I:
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Determine the sign of γg γ– Consider the following equations again:
Prof. Bebis
13/5/2009
16
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Determine Tz and fxz fx– Consider the equation:
– Let’s rewrite it in the form:
or xTz+fx(r11Xw+r12Yw+r13Zw+Tx) = -x(r31Xw+r32Yw+r33Zw)
Prof. Bebis
13/5/2009
17
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
– We can obtain Tz and fx by solving a system of equations like the above, written for N points:
Using SVD, the (least-squares) solution is:Prof. Bebis
Method 1: Direct Parameter CalibrationStep 1: estimate f x , α, R, and T
Determine fy:
Prof. Bebis
13/5/2009
18
Method 1: Direct Parameter CalibrationStep 2: estimate ox and oy
The computation of o and oy is based on the followingThe computation of ox and oy is based on the following theorem:
Orthocenter Theorem: Let T be the triangle on the image plane defined by the three vanishing points of three mutually orthogonal sets of parallel lines in space. Then (o o ) is the orthocenter of TThen, (ox , oy) is the orthocenter of T.
Prof. Bebis
Method 1: Direct Parameter CalibrationStep 2: estimate ox and oy
We can use the same calibration pattern to compute th i hi i t ( th i f ll lthree vanishing points (use three pairs of parallel lines defined by the sides of the planes).
- It is important that the calibration pattern is imaged from a viewpoint guaranteeing that none of the three mutually orthogonal directions will be
ll l t th i l !near parallel to the image plane!
- To improve the accuracy of the (ox , oy) computation, it is a good idea to estimate the center using severalviews of the calibration pattern and average the results.
Prof. Bebis
13/5/2009
19
Method 2: Camera parameters through the projection matrix
Review of basic equations
Prof. Bebis
Method 2: Camera parameters through the projection matrix: Step 1: solve for mij’s
The matrix M has 11 independent entries (e.g., divide t b )every entry by m11).
We would need at least N=6 world‐image point correspondences to solve for the entries of M.
Prof. Bebis
13/5/2009
20
Method 2: Camera parameters through the projection matrix: Step 1: solve for mij’s
These equations will lead to a homogeneous system of equations:system of equations:
N x 12 matrix
Prof. Bebis
Method 2: Camera parameters through the projection matrix: Step 1: solve for mij’s
Prof. Bebis
13/5/2009
21
Method 2: Camera parameters through the projection matrix: Step 2: find the
intrinsic/extrinsic parameters using mij’s
The full expression for M is as follows:
Let’s define the following vectors:
Prof. Bebis
Method 2: Camera parameters through the projection matrix: Step 2: find the
intrinsic/extrinsic parameters using mij’s
Th l i f ll ( b k fThe solutions are as follows (see book for details):
The rest parameters are easily computed ....
Prof. Bebis
13/5/2009
22
Comments on camera calibration
The precision of calibration depends on how accurately the world and image points are locatedaccurately the world and image points are located.– Studying how localization errors ʺpropagateʺ to the
estimates of the camera parameters is very important.
Prof. Bebis
Comments on camera calibration (cont’d)
Although the two methods presented should d th lt t l tproduce the same results ‐ at least
theoretically‐we usually obtain different solutions due to different error propagations.Method 2 is simpler and should be preferred if we do not need to compute the intrinsic/extrinsic camera parametersintrinsic/extrinsic camera parameters explicitly.
Prof. Bebis
13/5/2009
23
How would you estimate the accuracy of a calibration algorithm?
Project known 3D points on the imageC h h hCompare their projections with the corresponding pixel coordinates of the points!Repeat for many points and estimate “re‐projection” error!
Prof. Bebis
