Plane-based external camera calibration with accuracy measured by relative deflection angle Chunhui...

Plane-based external camera calibration with accuracy measured by relative deflection angleChunhui Cui ， KingNgiNgan

Journal Image Communication Volume 25 Issue 3, March, 2010 Elsevier Science Inc. New York, NY, USA

South Taiwan University ESLab

Professor ： Tsai, Lian-Jou

Student ： Tsai, Yu-Ming

PPT Production rate ： 100%

Date ： 2011/09/28

Outline• Introduction

• Basic equations

• Proposed pair-wise pose estimation

Homography estimation

(R, t) estimation

• Accuracy measure based on relative

deflection angle

• Experimental results

• Conclusion 1South Taiwan University ESLab

• Camera calibration has always been an essential component

of photogrammetric measurement.• Traditional Photogrammetric Calibration employs the

reference grids and determines the intrinsic matrix K using

images of a known object point array.

• External camera calibration ： estimate the poses of different

cameras and register them to get her in a world coordinate

system.• The separation of the internal and external calibration benefits

both the accuracy and efficiency of parameter estimation.

2South Taiwan University ESLab

Introduction (1/2)


Introduction (2/2)• In this paper, we present an efficient plane-based

external calibration method to estimate the relative

pose between two neighboring cameras.• Based on homography, three different pair-wise

estimation algorithms are proposed to recover the

rotation and translation between cameras.• It is convenient to take the RDA measure since it

does not require any prior knowledge of the true 3D

coordinates of control points.


Camera model

• Pinhole camera model, the relationship between a 3D point M and its image projection m.


Homography (1/3)• Suppose two cameras capture a planar pattern

3D Point ： M

Camera2 ： C2

Camera1 ： C1

•choose C1 as the world coordinate system.


Homography (2/3)

•m1 is the projection of M onto camera1

•m2 is the projection of M onto camera2

• where λ is an unknown arbitrary scalar.


Homography (3/3)


pair-wise pose estimation (1/6)• We apply Zhang’s method to do the intrinsic calibration for each individual camera.• Only the external calibration is necessary every time the cameras are moved and refocused to capture new 3D video.• We only use the planar pattern to estimate the relative relationship between two neighboring cameras.• It is argued that the chaining procedure is prone to errors.• Accurate and stable pair-wise calibration can benefit the accuracy of transform chaining.


pair-wise pose estimation (2/6)• Using this information, (R, t) between two cameras can be determined through an arbitrarily chosen model plane.• using these estimates may result in very unstable calibration results, since the accuracy of corner detection highly depends on the plane location and orientation.


pair-wise pose estimation (3/6)Homography estimation • Each image pair, one view from camera1 and the other from camera2, leads to a homography H. • Suppose there are totally P image pairs and then we can estimate P homographies Hi (i=1,2,…,P) induced by different planes.


pair-wise pose estimation (4/6)

Calculation of n and λ• P different stereo views lead to P different normals ni.• The plane position and orientation (R1, t1) relative to C1, where the plane is now assumed on Z=0 of the world coordinate system.



• The collineation matrix G has median singular value equal to one.

• That matrix K1, K2 and H are known,so we can compute λ and then recover the matrix G.


(R, t) estimation• A linear to solve the rotation R and translation t.

• As each model plane introduces a normal ni, a homography Hi and thus a matrix Gi, by stacking P equations.


Implicit orthogonal constraint (1/3)

• The three row vectors form an orthonormal basis of R^3.



• One of the necessary conditions to guarantee the orthogonality of R.


Two-step method• We obtain the uniform ratio o fthe three elements of t vector：

• t vector is reduced to a single scale s as ：

• While keeps the problem linear. (R, t) estimated by this method not only conform to the homography geometry, but also satisfy the orthogonal constraint R^T R=1 much better.‧


Three-step method (1/2)• In three dimensions a rotation can be defined by a single angle of rotation θ, and the direction of a unit vector ：

• v is the corresponding eigenvector ； Rv = v

(a)

(c)


Three-step method (2/2)• Estimate cosθ, sinθ and vector t together by stacking P such equations：

• Based on the above description, the Three-step method is outlined as follows：(1) Use Two-step method to compute the initial estimation of t. (2) Estimate vector v based on (a) with t fixed.(3) Estimate parameters cosθ, sinθ and refine t together based on (b) with v fixed. cosθ, sinθ and v already computed, the rotation matrix R can be recovered by (c).

(b)


Nonlinear method• We should impose the constraint R^T R=1 explicitly in the ‧ maximum likelihood estimation and it turns out to be a constrained nonlinear minimization problem.


relative deflection angle (1/4)• It can be achieved with good measures by: (1)Using lens with larger focal length so that the array of pixels focus on a smaller area of the scene.(2) Reducing the distance between the scene and the cameras to take a close look. (3) Directly increasing the image resolution.

• But the focal length and resolution of the digital camera will not benefit the measurement.• We propose to measure the calibration accuracy based on the relative deflection angle(RDA).


• Therefore the deflection angle θerr integrates the errors from both internal and external calibration.•The mean value E[θerr] over all the control points can be reasonably utilized to reveal the inaccuracy of stereo camera calibration.

relative deflection angle (2/4)


relative deflection angle (3/4)• Consider a pixel rectangle of size du*dv.• Uniform digitization noise in the rectangle has a standard deviation ： • This digitization error corresponds to the uncertainty of the projection of a 3D point onto the image plane, which can be measured as the angle of the deflected projection rays.


relative deflection angle (4/4)• θsys is associated with the image resolution and the camera focal length.


Experimental resultsSimulation on (R, t) estimation• Two cameras capture a planar pattern with9*12 50mm square corners. • Total 30 different plane poses are used for calibration.• The second camera is translated and rotated by；

• Add zero mean uncorrelated Gaussian noise to the image projections of the control points (square corners on the plane) from a standard deviation of 0.1 pixels up to 0.9 pixels.•The relative estimation are used to measure the calibration accuracy,The results of 50 simulations are summarized in：


Test on RDA metric (1/4)

• For all three experiments, the nonlinear method initialized by 3-step is applied to estimate the external parameters.

• For comparison, we apply another error metric to measure the calibration accuracy.• The mean square distance MSD defined in following will be used as the second error metric to measure the calibration results.


Test on RDA metric (2/4)• In the first experiment, the baseline of the two cameras is kept 1m fixed. We change the distance of the pattern from the cameras as well as the pattern orientations.


Test on RDA metric (3/4)• In the second experiment, the camera setup is varied in terms of the baseline length.• the distance between the pattern and the baseline midpoint is slightly adjusted such that the distances from the pattern to the two cameras are the same and kept unchanged


Test on RDA metric (4/4)• we change the camera focal length while keeping the baseline length and the viewing distance fixed.


Multi-camera calibration (1/2)• The intrinsic parameters for the five cameras are estimated individually beforehand using Zhang’s method.• A checkerboard pattern with 9*12 50 mm square corners is used for providing control points.• Two different system setups are tested as sketched in Fig. 7.


Multi-camera calibration (2/2)• We apply the proposed RDA metric to measure the calibration errors.• To investigate the error introduced by the chaining of pair-wise (R, t) estimation.


Conclusion• we present a convenient and efficient method to calibrate a typical multi-camera system.• Three different algorithms are proposed based on homography.• As the orthogonal constraint is fully imposed in the nonlinear method, it achieves the best calibration results in terms of the robustness to data noise. • The Three-step and Nonlinear methods is more flexible and universal than the traditional multi-camera calibration methods, where all the cameras are required to capture the same control points.• The RDA can be employed to provide a fair and meaningful evaluation on the calibration results for different cameras under different working conditions.

Thanks for your listening!! Any questions?

South Taiwan University ESLab

Plane-based external camera calibration with accuracy measured by relative deflection angle Chunhui...

Documents

Transcript of Plane-based external camera calibration with accuracy measured by relative deflection angle Chunhui...