Lecture 5.3 Camera calibration - Universitetet i oslo · Camera calibration • Camera calibration...

Lecture 5.3 Camera calibration

Thomas Opsahl

Introduction

2

• For finite projective cameras, the correspondence between points in the world and points in the image can be described by the simple model

𝑃 =𝑓𝑢 𝑠 𝑐𝑢0 𝑓𝑣 𝑐𝑣0 0 1

𝑅 𝒕

• This camera model is typically not good enough for accurate geometrical computations based on

images

𝑥𝐶 𝑦𝐶

𝑧𝐶 𝐶

𝑧𝐶 = 1

𝑊 𝑿 𝑢

𝑣

𝒙 𝒖

The image

World frame

Introduction

• Since light enters the camera through a lens instead of a pinhole, most cameras suffer from some kind of distortion

• This kind of distortion can be modeled and compensated for

• A radial distortion model can look like this 𝑥� = 𝑥 1 + 𝜅1 𝑥2 + 𝑦2 + 𝜅2 𝑥2 + 𝑦2 2 𝑦� = 𝑦 1 + 𝜅1 𝑥2 + 𝑦2 + 𝜅2 𝑥2 + 𝑦2 2

where 𝜅1 and 𝜅2 are the radial distortion parameters

X Y

Z

x

y

No radial distortion

X Y

Z

x

y

Barrel distortion

X Y

Z

x

y

Pincushion distortion

3

Introduction

• When we calibrate a camera we typically estimate the camera calibration matrix 𝐾 together with distortion parameters

• A model 𝐾 𝑅 𝒕 , using such an estimated 𝐾, does not in general describe the correspondence between points in the world and points in the image

• Instead it describes the correspondence between points in the world and points in a undistorted image – an image where the distortion effects have been removed

Does not satisfy 𝒖� = 𝐾 𝑅 𝑡 𝑿�

Satisfies 𝒖� = 𝐾 𝑅 𝑡 𝑿�

Image from a non-projective camera

Equivalent projective-camera image

Images: http://www.robots.ox.ac.uk/~vgg/hzbook/ 4

http://www.robots.ox.ac.uk/~vgg/hzbook/

Undistortion

• So earlier when we estimated homographies between overlapping images, we should really have been working with undistorted images!

• How to undistort? – Matlab

[undist_img,newOrigin] = undistortImage(img,cameraParams); undistortedPoints = undistortPoints(points,cameraParams);

– OpenCV cv::undistort(img, undist_img, P, distCoeffs); cv::undistortPoints(pts,undist_pts,P,distCoeffs);

• The effect of undistortion is that we get an image or a set of points that satisfy the perspective camera

model 𝒖� = 𝑃𝑿� much better than the original image or points – So we can continue working with the simple model

5

Camera calibration

• Camera calibration is the process of estimating the matrix 𝐾 together with any distortion parameter that we use to describe radial/tangential distortion

• We have seen how 𝐾 can be found directly from the camera matrix 𝑃

• The estimation of distortion parameters can be baked into this

• One of the most common calibration algorithms was proposed by Zhegyou Zhang in the paper ’’A Flexible New Technique for Camera Calibration’’ in 2000

– OpenCV: calibrateCamera – Matlab: Camera calibration app

• This calibration algorithm makes use of multiple images of a asymmetric chessboard

6

Zhang’s method in short

• Zhang’s method requires that the calibration object is planar

• Then the 3D-2D relationship is described by a homography

[ ] [ ]1 2 3 01 1 1

1

Xu X X

Yv K K Y H Y

é ùé ù é ù é ùê úê ú ê ú ê úê ú= = =ê ú ê ú ê úê úê ú ê ú ê úê úë û ë û ë û

ë û

1 2r r r t r r t

7

𝑋𝑊

𝑌𝑊

𝑍𝑊

𝑥𝐶 𝑦𝐶

𝑧𝐶 𝐶

𝑊


• This observation puts 2 constraints on the intrinsic parameters due to the fact that 𝑅 is orthonormal

• Where 𝐻 = 𝒉1

𝑇 𝒉2𝑇 𝒉3

𝑇 and 𝐾−𝑇 = 𝐾𝑇 −1 = 𝐾−1 𝑇

• So for each 3D-2D correspondence we get 2 constraints on the matrix 𝐵 = 𝐾−𝑇𝐾−1

• Next step is to isolate the unknown parameters in order to compute 𝐵

11 2 1 2

1 11 1 2 2 1 1 2 2

0 0T T T

T T T T T T

K KK K K K

- -

- - - -

ü ì= =Þý í

= =þ î

r r h hr r r r h h h h

8


• 𝐵 can be computed directly from 𝐾

• We see that 𝐵 is symmetric, so that we can represent it by the parameter vector

𝒃 = 𝑏11 𝑏12 𝑏22 𝑏13 𝑏23 𝑏33𝑇

( )

( ) ( )

0 02 2 2

11 12 13 20 01 0

21 22 23 2 2 2 2 2 2 2

31 32 33 22

0 0 0 00 0 0 02 2 2 2 2 2 2

1

1

1

y

u u v u v

yT

u v u v v u v v

y yy

u v u v v u v v

v s u fsf f f f f

b b b s v s u f vs sB K K b b bf f f f f f f f

b b bs v s u f v s u fv s u f v v

f f f f f f f f

- -

é ù--ê ú

ê úé ù ê ú-ê ú ê ú= = = - + -ê ú ê úê ú ê úë û

ê ú- --ê ú- + +ê úë û

9


• If we denote

• Thus we have

1 1

1 2 2 1

2 2

3 1 1 3

3 2 2 3

3 3

then

i j

i j i j

i j T Tij i j ij

i j i j

i j i j

i j

h hh h h h

h hB

h h h hh h h h

h h

é ùê ú+ê úê ú

= =ê ú+ê úê ú+ê úê úë û

v h h v b

1 2 1 2 12

1 1 2 2 1 1 2 2 11 22

00 00

T T T

T T T T T T

BB B

ü ì ü é ù= = é ùÞ Û =ý í ý ê ú ê ú= = - ë ûþ î þ ë û

r r h h vb

r r r r h h h h v v

10


• Given 𝑁 images of the planar calibration object we stack the equations to get a homogeneous system of linear equations which can be solved by SVD when 𝑁 ≥ 3

• From the estimated 𝒃 we can recover all the intrinsic parameters

• The distortion coefficients are then estimated solving a linear least-squares problem

• Finally all parameters are refined iteratively

• More details in Zhang’s paper http://research.microsoft.com/en-us/um/people/zhang/Papers/TR98-71.pdf

11

http://research.microsoft.com/en-us/um/people/zhang/Papers/TR98-71.pdf

Camera calibration in practice

• OpenCV – Camera calibration tutorial – We’ll test it out in the lab

• Matlab

– App: Camera Calibrator

– This opens a new window

12


• Add Images and specify the size of the chessboard squares

– Chessboard detection starts

13




• Inspect the correctness of detection and choose what to estimate

– 2/3 radial distortion coeffs? – Tangential distortion? – Skew?

• Calibrate

14






• Calibrate

• Export to a Matlab variable

15






• Calibrate

• Export to a Matlab variable

16

Summary

• Calibration – Zhangs method using planar 3D-points – Matlab app – DLT method: Est 𝑃, decompose into 𝐾 𝑅 𝒕

• Undistortion

– For geometrical computations we work on undistorted images/feature points

• Additional reading – Szeliski: 6.3

• Optional reading

– A flexible new technique for camera calibration, by Z. Zhang

17

Lecture 5.3 Camera calibration - Universitetet i oslo · Camera calibration • Camera calibration...

Documents

Transcript of Lecture 5.3 Camera calibration - Universitetet i oslo · Camera calibration • Camera calibration...