Multi View Geometry (Spring '08) · 2018. 1. 30. · 3 Camera Calibration 5 Multi View Geometry...

1

Camera Calibration 1

2008-1

Multi View Geometry (Spring '08)

Prof. Kyoung Mu LeeSoEECS, Seoul National University

Computation of the Camera Matrix P


Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Computation of the Camera Matrix - DLT

• Problem:Given point correspondences , how to find a camera matrix P, such that ?

• DLT:

ii xX ↔ii PXx =

i

ii

i

ii

i

i

i

i

i

i vyuxZYX

vu

ωωω

==

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡, ,

1

P

0PPP

X0XXX0

0PPP

X0XXX0

0PPP

0XXX0X

XX0

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡

−−

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡

−−

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−

3

2

1

3

2

1

3

2

1

Tii

TTi

Tii

Ti

T

Tii

TTii

Tii

Tii

T

TTii

Tii

Tii

TTii

Tii

Tii

T

xy

uv

uvu

v

ωω

ωω

0pA =i

0PXx =× ii

2



Computation of the Camera Matrix - DLT

• Using n correspondences, we have

Minimal solutionqP has 11 dof, 2 independent eq./pointsqThus, 6 correspondences are needed

Over-determined solutionqMore than 6 correspondencesqThis can be solved by minimizing the algebraic error (DLT)

with constraintsTppp ),(ˆ where,1ˆor 1 33,3231

33 === ppp

0pA =×122n

Apmin3p̂

=P



Degenerate Configurations

• More complicate situations than 2D case

When the camera and points are on a twisted cubic

When points lie on plane or single line passing through the projection center

3



DLT- Data Normalization

• Normalization of 3D data

Simple as before in 2D

Anistropic scaling

32

2D 3D



DLT - Lines

• Extend DLT to Line correspondencesAssume an imaged line l and two points X0 and X1 on the 3D lineSince the plane back-projected from l is PTl, the two points must satisfy

Two linear equations on the entries of P for a line

Thus for n correspondences, we have

0pA =×122n

l

X0

X1PTl

C

why?

4



Geometric error

• Assume Xi are known

• The geometric error in the image:

• MLE of P:

points estimated:ˆpoints measured:

ii

i

PXxx=



Gold Standard Algorithm

ObjectiveGiven n≥6 3D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelihood Estimation of P

Algorithm(i) Linear solution:

(a) Normalization: (b) DLT:

(ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:

(iii) Denormalization:

ii UXX =~

ii Txx =~

UPTP ~1-=

( )2~~,~∑i

iid XPx

5



Calibration Example

• Use Canny edge detector• Detect straight lines from the edgels• Find corners by intersecting the lines • Subpixel accuracy < 1/10 • # of constraints > 5 x # of unknowns (11) => more than

28 points are required



Errors in the world points

• Errors in the world

• Errors in the image and in the world

ii XPx)

=

6



Geometric Interpretation of Algebraic Error

• Normalized DLT minimizes

• Note invariance to 2D and 3D similarity transform given proper normalization

( )2)ˆ,(ˆ∑i

iiid xxω

( ) iTiii yx PX=1,ˆ,ˆω̂ );depth(ˆˆ 3 PXp±=iω

)ˆ,(~)ˆ,(ˆ iiiii fdd XXxxω

then1ˆ if therefore, 3 =p

Tiiii

Tiiii

yx

ZYX

)1,,(

)1,,,(

ω=

=

x

X



Estimation of an Affine Camera Matrix

• Affine camera matrix: • For

• By stacking n correspondences, and using pseudo inverse

• Algebraic error: (=geometric error, in this case )

)1,0,0,0( ,),,( 3321 == TTA PPPPPT

iiiiT

iiii yxZYX )1,,( and )1,,,( ω== xX

ii

i

iTi

T

TTi

i

iTT

i

Ti

T

yx

xy

bpAPP

X00X

0PP

0XX0

=⇒

⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛⎥⎦

⎤⎢⎣

⎡⇒

=⎟⎟⎠

⎞⎜⎜⎝

⎛−

+⎟⎟⎠

⎞⎜⎜⎝

⎛⎥⎦

⎤⎢⎣

⎡ −

8

2

1

2

1

bApbpA +=⇒= 8888

( ) ( ) ( )∑∑ =−+−=−i

iii

iT

iiT

i dyx222212 ˆ, xxXPXPApb

7



Gold Standard Algorithm - Affine Camera

ObjectiveGiven n≥4 3D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelihood Estimation of P(remember P3T=(0,0,0,1))

Algorithm(i) Normalization:(ii) For each correspondence

(iii) solution is and

(iv) Denormalization:

bpA =88bAp += 88

ii UXX =~

ii Txx =~

UPTP ~1-=

⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛⎥⎦

⎤⎢⎣

⎡

i

iTi

T

TTi

yx~~

~~

~~

2

1

PP

X00X

)1,0,0,0(~3 =TP



Decompose P into K, R, and t

• We note that M=KR is the product of an upper triangular and rotation matrix

Factor M into KR using the QR matrix decomposition. This determines K and R.Then

• Note that this produces a matrix with nonzero skew s

with , and θ is the angle between the image axes

Tppp ),,( 3424141−= Kt

θtan=s

8



Restricted Camera Estimation

• Camera estimation with some known parameter information:

The skew is zero s = 0The pixels are square The principal point (x0, y0) is knownThe internal parameter matrix K is known

• Minimizing geometric error:impose constraint through parameterizationSuppose , then2D Image error case: LM minimization 3D and 2D error case: LM minimization

yx αα =

nf 29 RR: →nnf 593 RR: →+

yxs αα == ,0




• Minimizing algebraic error:Assume map from param q → , i.e. p=g(q)Minimize

• Reduced measurement matrixReplace with such that

By using SVD

By using QR factorization

Thus,

found be toparameters theis where,)( qqAAp g=

12 x 2nA 12 x 12Â

AAAAPAAPAPAPpAAp ˆˆˆˆˆ TTTTTT =⇒=⇒=

AADV(VDUDV(VDUAA

DVAUDVAˆˆ))())(

then,ˆ and Let TTTTT

TT

===

==

AAAQQA(AA

AQAˆˆ)ˆ)(ˆ

then,ˆLet TTTT ==

=

129 RR:)(ˆ: →⇒→ fgf qAq

]~|[ CRRKP −=

9




• InitializationFind initial P using DLTDecompose P and clamp fixed parameters to their desired values e.g. s=0, αx= αy (hard constraint).Set other variables to their values obtained by the new decompositionIt can sometimes cause big jump in error

• Alternative initializationUse general DLTImpose soft constraints

gradually increase weights



Exterior Orientation

• If the camera is calibrated (internal parameters are known), then the position and orientation of the camera should be determined

→ Pose estimation• 6 DOF ⇒ 3 points minimal (4 solutions in general)

10



Experimental Evaluation



Covariance Estimation

• ML residual error

• Example:

37.0 ,365.0 ,197 === σε resn

σεσε

↔−=

res

res nd2/1)2/1(

11



Covariance for Estimated Camera

• Compute Jacobian at ML solution, then

variance per parameter can be found on diagonal

• Confidence ellipsoid for the camera center

( )+−= JΣJΣ xP 1T

2χ(chi-square distribution=distribution of sum of squares)

cumulative-1

)(12 α−= nFk



Covariance for Estimated Camera

Camera center covariance ellipsoids

12



Radial Distortion

center opticalcenter distortion ),( ≈cc yx

),( yx )ˆ,ˆ( yxr),( cc yx

long focal lengthshort focal length



Radial Distortion

• Correction of distortion

• Distortion function: Taylor series

• Distortion parameters can be determined by some constraints or cost function:

Straight line should be straightDeviation from the linear mapping

222 )()( cc yyxxr −+−=

L++++= 332

211)( rrrrL κκκsufficient is 1)( 2rrL κ+=

13



Radial Distortion



Radial Distortion

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles false /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ]>> setdistillerparams> setpagedevice

Multi View Geometry (Spring '08) · 2018. 1. 30. · 3 Camera Calibration 5 Multi View Geometry...

Documents

Transcript of Multi View Geometry (Spring '08) · 2018. 1. 30. · 3 Camera Calibration 5 Multi View Geometry...