Camera parameters

• Extrinisic parameters define location and orientation of camera reference frame with respect to world frame

• Intrinsic parameters define pixel coordinates of image point with respect to coordinates in camera reference frame

Homogenous coordinates

Add an extra coordinate and define equivalenceRelationship

(X,Y,Z) -> (wX, wY, wZ, w)

(x,y) -> (kx, ky, k)

Makes it possible to write the Perspective projection as a linear Transformation (matrix)(from projective space to projective plane)

Central projection

Z

Yfy

Z

Xfx

10100

000

000

Z

Y

X

f

f

w

v

u

w

vw

u

y

x

w

v

u

HC NHC

Scaled orthographic projection

fYyfXx

11000

000

000

Z

Y

X

f

f

w

v

u

w

vw

u

y

x

w

v

u

HC NHC

In a simpler notation

T describes the position of the origin of camera frame with respect to world frame

R describes the rotation which aligns the camera frame with the world frame

Pc = R(Pw – T)

(here –RT = BOA)

ABAB

AB OPRP

Translation and Rotation

)( TPRP wc

110001w

w

w

c

c

c

Z

Y

X

RTR

Z

Y

X

Intrinsic parameters

Z

Yfy

Z

Xfx

x

y

xpix

ypix

yky

xkx

ypix

xpix

Scaling

Intrinsic parameters

Z

Yfy

Z

Xfx

x

y

xpix

ypix

0

0

yyky

xxkx

ypix

xpix

Intrinsic parameters

Z

Yfy

Z

Xfx

x

y

xpix

ypix

0

0

sin

cot

yyθ

ky

xykxkx

ypix

xxpix

The internal calibration parameters

shear theis sin

sfk

ffkf yyxx

10100

00

0

0

0

c

c

c

y

x

Z

Y

X

yf

xsf

w

v

u

w

vy

w

ux pixpix

1

intw

w

w

ext Z

Y

X

MM

w

v

u

with

100

0int oy

ox

yf

xsf

M

TRrrr

TRrrr

TRrrr

MT

T

T

ext

2333231

2232221

1131211

MPp

MPp

TIMTIRMp |'| 33int

Properties of matrix M

• M has 11 degrees of freedom (5 internal 3 rotation, 3 translation parameters) , 3x4 matrix defined up to scale

•The 3x3 submatrix M’=MintR is non-singular (Mint is upper triangular, R is orthogonal -> essential QR decomposition)

Radial distortion

from lens distortion (pin cushioning effect)

Straight lines are not imaged straight

222

42

21 ....1)(

)()(

dd

dd

yxr

rkrkrL

rLyyrLxx

(significant error for cheap optics and short focal length)

x and xd

measured fromimage center

Radial calibration

2,

2,2,1 )()()( idi

iidi yyxxkkfMinimize

Using lines to be straight(x’,y’) is radial projection of (xd, yd) on straight line

),( yx

),( dd yx

Calibration Procedure

• Calibration target : 2 planes at right angle with checkerboard (Tsai grid)

• We know positions of corners of grid with respect to a coordinate system of the target

• Obtain from images the corners• Using the equations (relating pixel coordinates to

world coordinates) we obtain the camera parameters (the internal parameters and the external (pose) as a side effect)

Image Processing

• Canny edge detection

• Straight line fitting to detect long edges

• Intersection of lines to detect image corners

• Matching image corners and 3D checkerboard corners

Estimation procedure

• First estimate M from corresponding image points and scene points (solving homogeneous equation)

• Second decompose M into internal and external parameters

• Use estimated parameters as starting point to solve calibration parameters non-linearly.

1

intw

w

w

ext Z

Y

X

MM

w

v

u

PMp

.

.

.

..

..

..

3

2

1

T

T

T

m

m

m

M

Pm

Pm

w

vy

Pm

Pm

w

ux

3

2

3

1

0 PMpor

(homogeneous equation)

nnnnnnnnnn

nnnnnnnnnn

yZyZyXyZYX

xZxYxXxZYX

yZyYyXyZYX

xZxYxXxZYX

A

10000

00001

10000

00001

1111111111

1111111111

0Am

34

33

12

11

.

.

m

m

m

m

m

Solving A m = 0

Linear homogeneous system

Have at least 5 times as many equations as unknowns (28 points)

Minimize ||Am||2 with the constraint ||m||2=1

M is the unit singular value of A correspondingto the smallest singular value (the last column ofV, where A = UDVT is the SVD of A),or the eigenvector (corresponding to smallest eigenvalue ) of ATA

Finding camera translation

T~Let be the homogeneous representation of T

T~ is the null vector of M: 0

~ TM

0~

|3int TTIRM

Null vector is found using SVD( is the unit singular vector corresponding to the smallest singular value of M)T~

(position of camera center)

Finding camera orientation and internal parameters

• Left 3x3 submatrix of M is of the form M’= Mint R Mint upper triangular R orthogonal• Any nonsingular matrix can be decomposed into the product of an upper triangular and an orthogonal matrix (RQ factorization—here R refers to upper triangular and Q to orthogonal) (Similar to QR factorization)

RQ factorization of M’• Givens rotations

100

0''''

0''''

'0'

010

'0'

,

0

0

001

cs

sc

R

cs

sc

R

cs

scR zyx

To set M’32 to zero, solve equation

Thus:

03,32,3 msmc

2/123,3

22,3

2,3

2/123,3

22,3

3,3

)()( mm

ms

mm

mc

Multiply M’ by Rx ( such that term (3,2) is 0), then by, Ry (choosing c’, s’ such that term (3,1) is zero), then by Rz (with c’’, s’’ such that term (2,1) is zero)

RMRRRMMMRRRM Tx

Ty

Tzzyx intintint ''

Improving solution with nonlinear optimization

Find m using the linear constraint

Use as initialization for nonlinear optimization

||,||i

ii MPp

(Levenberg-Marquardt iterative minimization)

Algorithm described inMultiple View Geometry in Computer Vision(Hartley, Zisserman)