# Geometric Models & Camera Calibration

date post

08-Jan-2016Category

## Documents

view

37download

2

Embed Size (px)

description

### Transcript of Geometric Models & Camera Calibration

Geometric Models & Camera CalibrationReading: Chap. 2 & 3

CS4243: Geometric Models

IntroductionWe have seen that a camera captures both geometric and photometric information of a 3D scene.Geometric: shape, e.g. lines, angles, curvesPhotometric: color, intensityWhat is the geometric and photometric relationship between a 3D scene and its 2D image?We will understand these in terms of models.

CS4243: Geometric Models

ModelsModels are approximations of reality.Reality is often too complex, or computationally intractable, to handle.Examples:Newtons Laws of Motion vs. Einsteins Theory of RelativityLight: waves or particles?No model is perfect.Need to understand its strengths & limitations.

CS4243: Geometric Models

Camera Projection Models3D scenes project to 2D imagesMost common model: pinhole camera model

From this we may derive several types of projections:orthographicweak-perspective para-perspectiveperspective

CS4243: Geometric Models

Pinhole Camera ModelffYZobjectvirtual image planereal image planeimage is not invertedimage is invertedoptical centre

CS4243: Geometric Models

Real vs. Virtual Image PlaneThe image is physically formed on the real image plane (retina).The image is vertically and laterally inverted. We can imagine a virtual image plane at a distance of f in front of the camera optical center, where f is the focal length.The image on this virtual image plane is not inverted.This is actually more convenient.Henceforth, when we say image plane we will mean the virtual image plane.

CS4243: Geometric Models

Perspective Projectionfu

CS4243: Geometric Models

Perspective ProjectionThe imaging process is a many-to-one mapping all points on the 3D ray map to a single image point.

Therefore, depth information is lost

From similar triangles, can write down the perspective equation:

CS4243: Geometric Models

Perspective ProjectionThis assumes coordinate axes is at the pinhole.

CS4243: Geometric Models

World coordinatesystemCamera coordinatesystem3D Scene point Pt : World origin w.r.t. camera coordinate axesP : 3D position of scene point w.r.t world coordinate axesRotation RR : Rotation matrix to align world coordinate axes to camera axes

CS4243: Geometric Models

Perspective Projection, : scaling in image u, v axes, respectively : skew angle, that is, angle between u, v axesu0, v0 : origin offsetNote: =kf, =lf where k, l are the magnification factors

CS4243: Geometric Models

Intrinsic vs. Extrinsic parametersIntrinsic: internal camera parameters.6 of them (5 if you dont care about focal length)Focal length, horiz. & vert. magnification, horiz. & vert. offset, skew angle

Extrinsic: external parameters6 of them3 rotation angles, 3 translation param

Imposing assumptions will reduce # params.Estimating params is called camera calibration.

CS4243: Geometric Models

Representing Rotations Euler angles

pitch: rotation about x axis :

yaw: rotation about y axis:

roll: rotation about z axis:

CS4243: Geometric Models

Rotation MatrixTwo properties of rotation matrix:

R is orthogonal: RTR = Idet(R) = 1

CS4243: Geometric Models

Orthographic ProjectionImage planeProjection rays are parallel

CS4243: Geometric Models

Orthographic Projection Equations: first two rows of R: first two elements of tThis projection has 5 degrees of freedom.

CS4243: Geometric Models

Weak-perspective Projection

CS4243: Geometric Models

Weak-perspective ProjectionThis projection has 7 degrees of freedom.

CS4243: Geometric Models

Para-perspective Projection

CS4243: Geometric Models

Para-perspective ProjectionCamera matrix given in textbook Table 2.1, page 36.Note: Written a bit differently.This has 9 d.o.f.

CS4243: Geometric Models

Real camerasReal cameras use lensesLens distortion causes distortion in imageLines may project into curvesSee Chap 1.2, 3.3 for details Change of focal length (zooming) scales the imagenot true if assuming orthographic projection)Color distortions too

CS4243: Geometric Models

Color Aberration Bad White Balance

CS4243: Geometric Models

Color AberrationPurple Fringing

CS4243: Geometric Models

Color Aberration Vignetting

CS4243: Geometric Models

Geometric Camera Calibration

CS4243: Geometric Models

RecapA general projective matrix:

This has 11 d.o.f. , i.e. 11 parameters5 intrinsic, 6 extrinsic

How to estimate all of these?Geometric camera calibration

CS4243: Geometric Models

Key IdeaCapture image of a known 3D object (calibration object).Establish correspondences between 3D points and their 2D image projections.Estimate MEstimate K, R, and t from MEstimation can be done using linear or non-linear methods.We will study linear methods first.

CS4243: Geometric Models

Setting things upCalibration object:3D object, or2D planar object captured at different locations

CS4243: Geometric Models

Setting it upSuppose we have N point correspondences:P1, P2, PN are 3D scene points u1, u2, uN are corresponding image points

Let mT1, mT2, mT3 be the 3 rows of MLet (ui, vi) be the (non-homogeneous) coords of the i th image point.Let Pi be the homogeneous coords of i th scene point.

CS4243: Geometric Models

MathFor the i th point

Each point gives 2 equations.

Using all N points gives 2N equations:

CS4243: Geometric Models

MathA m = 0A : 2N x 12 matrix, rank is 11m : 12 x 1 column vectorNote that m has only 11 d.o.f.

CS4243: Geometric Models

MathSolution?m is in the Nullspace of A !

Use SVD: A = U VT

m = last column of V m is only up to an unknown scaleIn this case, SVD solution makes || m || = 1

CS4243: Geometric Models

Estimating K, R, tNow that we have M (3x4 matrix)

c is the 3D coords of camera center wrt World axesLet be homogeneous coords of c It can be shown that M = 0So is in the Nullspace of MAnd t is computed from Rc, once R is known.

CS4243: Geometric Models

Estimating K, R, tB, the left 3x3 submatrix of M is KRPerform RQ decomposition on B.The Q is the required rotation matrix RAnd the upper triangular R is our K matrix !Impose the condition that diagonals of K are positive

We are done!

CS4243: Geometric Models

RQ factorizationAny n x n matrix A can be factored into A = RQWhere both R, Q are n x nR is upper triangular, Q is orthogonalNot the same as QR factorizationTrick: post multiply A by Givens rotation matrices: Qx ,Qy, Qz

CS4243: Geometric Models

RQ factorizationc, s chosen to make a particular entry of A zeroFor example, to make a21 = 0, we solve

Choose Qx ,Qy, Qz such that

CS4243: Geometric Models

DegeneracyCaution: The 3D scene points should not all lie on a plane.Otherwise no solutionIn practice, choose N >= 6 points in general position

Note: the above assumes no lens distortion.See Chap 3.3 for how to deal with lens distortion.

CS4243: Geometric Models

SummaryPresented pinhole camera modelFrom this we get hierarchy of projection typesPerspective, Para-perspective, Weak-perspective, Orthographic

Showed how to calibrate camera to estimate intrinsic + extrinsic parameters.

Unlike human vision, cameras can not adapt their spectral responses to cope with different lighting conditions; as a result, the acquired image may have an undesirable shift in the entire color range. Digital cameras have an function called white balancing. If improperly white balanced, the color and tone of the whole picture will be wrong.

Over-exposure at the time.When camera lens is wide open and there's high contrast in the scene, what normally should be a white line could appear purple. That is called "purple fringing" or a "halo" effect. The purple fringe around the edges is mainly caused by color interpolation combined with chromatic aberration. There are little patches of light purple discoloration all around the background and a heavy purple fringing artifact appeared along the leaning branch.we can notice that the corners of the print are getting darker. Darkening of an image around the edges and corners is called vignetting. Its mainly caused by two reasons: One is the design of the camera or lens itself. All lenses falloff toward the corners, and the physical lens barrier would prevent some light from entering the lens. Therefore, the central part of an image would be brighter than that of the parts at the corner or along the edges. The other cause may be light hitting the film or CCD sensor surface at an oblique angle. Its more often occurring when using wide-angle lenses. In flash photography a similar effect can be caused by the coverage of the flash being insufficient to illuminate the field of view.

*View more*