1

Old summary of camera modelling

• 3 coordinate frame• projection matrix• decomposition• intrinsic/extrinsic param

2

World coordinate frame: extrinsic parameters

11

1w

w

w

c

c

c

ZYX

ZYX

0tR

Finally, we should count properly ...

13343 0

tR0IKC

3

‘abstract’ camera: projection from P3 to P2

This is the most general camera model without considering optical distortion

As lines are preserved so that it is a linear transformation and can be represented by a 3*4 matrix

23: PP C

c33

34333231

24232221

14131211

CC

cccccccccccc

Math: central proj. Physics: pin-hole

‘new’ way of looking at ‘old’ modeling

4

• 11 d.o.f.• Rank(P) = ?• ker(P)=c• row vectors, planes• column vectors, directions• principal plane: w=0• calibration, 6 pts• decomposition by QR, • K intrinsic (5). R, t, extrinsic (6)• geometric interpretation of K, R, t (backward from u/x=v/y=f/z to P)• internal parameters and absolute conic

Properties of the 3*4 matrix P

5

It is the image of the absolute conic, prove it first!

1T KK

TKK

Point conic:

The dual conic:

What is the calibration matrix K?

6

01cos))((2)()( 002

20

2

201T

vuvu

T

avvuuvvuu

uKKu

2002

20

2

20 1cos))((2)()( i

avvuuvvuu

vuvu

),( 00 vu

vu

7

Don’t forget: when the world is planar …

10

34333231

24232221

14131211

zyx

cccccccccccc

wvu

1343231

242221

141211 yx

ccccccccc

wvu

A general plane homography!

8

Camera calibration

ii Xu Given

• Estimate C• decompose C into intrinsic/extrinsic

from image processing or by hand

9

Calibration set-up:

3D calibration object

10

),,(),( ii iiiii zyxvu Xu

34333231

24232221

34333231

14131211

czcycxcczcycxcv

czcycxcczcycxcu

iii

iiii

iii

iiii

The remaining pb is how to solve this ‘trivial’ system of equations!

11

Review of some basic numerical algorithms

• linear algebra: how to solve Ax=b?• (non-linear optimisation)• (statistics)

12

Linear algebra review

• Gaussian elimination • LU decomposition

• orthogonal decomposition • QR (Gram-Schmidt)• SVD (the high(est)light of linear algebra!)

13

Solving (full rank) square matrix linear sys Ax =b = elimination = LU factorization

1. factor A into LU2. solve Lc = b (lower triangular, forward substitution)3. solve Ux=c (upper tri., backward substitution)

14

Solving for Least squares solution for Ax=b, min||Ax-b|| = pseudo-inverse x = (A^TA)-1(A^T A)b (theoretically, but not numerically)

Numerically, QR does it well: as A^TA= R^TR,

Orthogonal bases and Gram-Schmidt A = QR

15

Solving for homogeneous system Ax=o subject to ||x||=1,

It is equivalent to min||Ax||, i.e. x^T A^T A x,

the solution is the eigenvector of A^TA associated with the smallest eigenvalue

Triangular systems not bad, but diagonal system is better!

Diagonalization = eigen vectors => doable for symmtric matrices

16

SVD gives orthogonal bases for all subspaces

TVUA nm

• row space: first Vs• null space: last Vs• col space: first Us• null space of the trans : last Us

A x = b, pseudo-inverse, x = A+ b for both square system and least squares sol.Even better with homogeneous sys: A x =0, x = v_n !

You get everything with svd:

17

Linear methods of computing P

• p34=1• ||p||=1• ||p3||=1

Geometric interpretation of these constraints

18

Decomposition

• analytical by equating K(R,t)=P• (QR (more exactly it is RQ))

19

zT

zyvTT

v

zxuTT

u

ttvtvtutu

3

0302

0301

43

rrrrr

C

1. Renormalise by c32. tz = c343. r3 = c34. u0 = c1^T c35. v0 = c2^T c36. alpha u7. alpha v8. …

20

Linear, but non-optimal,but we want optima, but non-linear,

methods of computing P

21

))()((min 2

34333231

242322212

34333231

14131211

czcycxcczcycxcv

czcycxcczcycxcu

iii

iiii

iii

iiii

How to solve this non-linear system of equations?

22

(Non-linear iterative optimisation)

• J d = r from vector F(x+d)=F(x)+J d• minimize the square of y-F(x+d)=y-F(x)-J d = r – J d • normal equation is J^T J d = J^T r (Gauss-Newton)• (H+lambda I) d = J^T r (LM)

Note: F is a vector of functions, i.e. min f=(y-F)^T(y-F)

23

Using a planar pattern

Cf. the paper by Zhengyou Zhang (ICCV99), Sturm and Maybank (CVPR99)

(Homework: read these papers.)

Why? it is more convenient to have a planar calibration pattern than a 3D calibration object, so it’s very popular now for amateurs.

24

10

34333231

24232221

14131211

zyx

cccccccccccc

wvu

1343231

242221

141211

yx

ccccccccc

wvu

• first estimate the plane homogrphies Hi from u and x, 1. How to estimate H? 2. Why one may not be sufficient?

• extract parameters from the plane homographies

25

z

y

xu

tttrur

cccccccccccc

********13011

34333231

24232221

14131211 H

Relationship between H and parameters:

How to extract intrinsic parameters?

26

(How to extract intrinsic parameters?)

1TKK

0)(1T

xHKKHx TT

The absolute conic in image

The (transformed) absolute conic in the plane:

The circular points of the Euclidean plane (i,1,0) and (-i,1,0) go thru this conic: two equations on K!

27

It turns the camera into an spherical one, or angular/direction sensor!

Direction vector:

Angle between two rays ...

uKd -1

What does the calibration give us?

uKx -1Normalised coordinates:

28

Summary of calibration

1. Get image-space points2. Solve the linear system 3. Optimal sol. by non-linear method4. Decomposition by RQ