Geometric Models & Camera Calibration

Click here to load reader

download Geometric Models &  Camera Calibration

of 38

  • date post

    08-Jan-2016
  • Category

    Documents

  • view

    37
  • download

    2

Embed Size (px)

description

Geometric Models & Camera Calibration. Reading: Chap. 2 & 3. Introduction. We have seen that a camera captures both geometric and photometric information of a 3D scene. Geometric: shape, e.g. lines, angles, curves Photometric: color, intensity - PowerPoint PPT Presentation

Transcript of Geometric Models & Camera Calibration

  • Geometric Models & Camera CalibrationReading: Chap. 2 & 3

    CS4243: Geometric Models

    IntroductionWe have seen that a camera captures both geometric and photometric information of a 3D scene.Geometric: shape, e.g. lines, angles, curvesPhotometric: color, intensityWhat is the geometric and photometric relationship between a 3D scene and its 2D image?We will understand these in terms of models.

    CS4243: Geometric Models

    ModelsModels are approximations of reality.Reality is often too complex, or computationally intractable, to handle.Examples:Newtons Laws of Motion vs. Einsteins Theory of RelativityLight: waves or particles?No model is perfect.Need to understand its strengths & limitations.

    CS4243: Geometric Models

    Camera Projection Models3D scenes project to 2D imagesMost common model: pinhole camera model

    From this we may derive several types of projections:orthographicweak-perspective para-perspectiveperspective

    CS4243: Geometric Models

    Pinhole Camera ModelffYZobjectvirtual image planereal image planeimage is not invertedimage is invertedoptical centre

    CS4243: Geometric Models

    Real vs. Virtual Image PlaneThe image is physically formed on the real image plane (retina).The image is vertically and laterally inverted. We can imagine a virtual image plane at a distance of f in front of the camera optical center, where f is the focal length.The image on this virtual image plane is not inverted.This is actually more convenient.Henceforth, when we say image plane we will mean the virtual image plane.

    CS4243: Geometric Models

    Perspective Projectionfu

    CS4243: Geometric Models

    Perspective ProjectionThe imaging process is a many-to-one mapping all points on the 3D ray map to a single image point.

    Therefore, depth information is lost

    From similar triangles, can write down the perspective equation:

    CS4243: Geometric Models

    Perspective ProjectionThis assumes coordinate axes is at the pinhole.

    CS4243: Geometric Models

    World coordinatesystemCamera coordinatesystem3D Scene point Pt : World origin w.r.t. camera coordinate axesP : 3D position of scene point w.r.t world coordinate axesRotation RR : Rotation matrix to align world coordinate axes to camera axes

    CS4243: Geometric Models

    Perspective Projection, : scaling in image u, v axes, respectively : skew angle, that is, angle between u, v axesu0, v0 : origin offsetNote: =kf, =lf where k, l are the magnification factors

    CS4243: Geometric Models

    Intrinsic vs. Extrinsic parametersIntrinsic: internal camera parameters.6 of them (5 if you dont care about focal length)Focal length, horiz. & vert. magnification, horiz. & vert. offset, skew angle

    Extrinsic: external parameters6 of them3 rotation angles, 3 translation param

    Imposing assumptions will reduce # params.Estimating params is called camera calibration.

    CS4243: Geometric Models

    Representing Rotations Euler angles

    pitch: rotation about x axis :

    yaw: rotation about y axis:

    roll: rotation about z axis:

    CS4243: Geometric Models

    Rotation MatrixTwo properties of rotation matrix:

    R is orthogonal: RTR = Idet(R) = 1

    CS4243: Geometric Models

    Orthographic ProjectionImage planeProjection rays are parallel

    CS4243: Geometric Models

    Orthographic Projection Equations: first two rows of R: first two elements of tThis projection has 5 degrees of freedom.

    CS4243: Geometric Models

    Weak-perspective Projection

    CS4243: Geometric Models

    Weak-perspective ProjectionThis projection has 7 degrees of freedom.

    CS4243: Geometric Models

    Para-perspective Projection

    CS4243: Geometric Models

    Para-perspective ProjectionCamera matrix given in textbook Table 2.1, page 36.Note: Written a bit differently.This has 9 d.o.f.

    CS4243: Geometric Models

    Real camerasReal cameras use lensesLens distortion causes distortion in imageLines may project into curvesSee Chap 1.2, 3.3 for details Change of focal length (zooming) scales the imagenot true if assuming orthographic projection)Color distortions too

    CS4243: Geometric Models

    Color Aberration Bad White Balance

    CS4243: Geometric Models

    Color AberrationPurple Fringing

    CS4243: Geometric Models

    Color Aberration Vignetting

    CS4243: Geometric Models

    Geometric Camera Calibration

    CS4243: Geometric Models

    RecapA general projective matrix:

    This has 11 d.o.f. , i.e. 11 parameters5 intrinsic, 6 extrinsic

    How to estimate all of these?Geometric camera calibration

    CS4243: Geometric Models

    Key IdeaCapture image of a known 3D object (calibration object).Establish correspondences between 3D points and their 2D image projections.Estimate MEstimate K, R, and t from MEstimation can be done using linear or non-linear methods.We will study linear methods first.

    CS4243: Geometric Models

    Setting things upCalibration object:3D object, or2D planar object captured at different locations

    CS4243: Geometric Models

    Setting it upSuppose we have N point correspondences:P1, P2, PN are 3D scene points u1, u2, uN are corresponding image points

    Let mT1, mT2, mT3 be the 3 rows of MLet (ui, vi) be the (non-homogeneous) coords of the i th image point.Let Pi be the homogeneous coords of i th scene point.

    CS4243: Geometric Models

    MathFor the i th point

    Each point gives 2 equations.

    Using all N points gives 2N equations:

    CS4243: Geometric Models

    MathA m = 0A : 2N x 12 matrix, rank is 11m : 12 x 1 column vectorNote that m has only 11 d.o.f.

    CS4243: Geometric Models

    MathSolution?m is in the Nullspace of A !

    Use SVD: A = U VT

    m = last column of V m is only up to an unknown scaleIn this case, SVD solution makes || m || = 1

    CS4243: Geometric Models

    Estimating K, R, tNow that we have M (3x4 matrix)

    c is the 3D coords of camera center wrt World axesLet be homogeneous coords of c It can be shown that M = 0So is in the Nullspace of MAnd t is computed from Rc, once R is known.

    CS4243: Geometric Models

    Estimating K, R, tB, the left 3x3 submatrix of M is KRPerform RQ decomposition on B.The Q is the required rotation matrix RAnd the upper triangular R is our K matrix !Impose the condition that diagonals of K are positive

    We are done!

    CS4243: Geometric Models

    RQ factorizationAny n x n matrix A can be factored into A = RQWhere both R, Q are n x nR is upper triangular, Q is orthogonalNot the same as QR factorizationTrick: post multiply A by Givens rotation matrices: Qx ,Qy, Qz

    CS4243: Geometric Models

    RQ factorizationc, s chosen to make a particular entry of A zeroFor example, to make a21 = 0, we solve

    Choose Qx ,Qy, Qz such that

    CS4243: Geometric Models

    DegeneracyCaution: The 3D scene points should not all lie on a plane.Otherwise no solutionIn practice, choose N >= 6 points in general position

    Note: the above assumes no lens distortion.See Chap 3.3 for how to deal with lens distortion.

    CS4243: Geometric Models

    SummaryPresented pinhole camera modelFrom this we get hierarchy of projection typesPerspective, Para-perspective, Weak-perspective, Orthographic

    Showed how to calibrate camera to estimate intrinsic + extrinsic parameters.

    Unlike human vision, cameras can not adapt their spectral responses to cope with different lighting conditions; as a result, the acquired image may have an undesirable shift in the entire color range. Digital cameras have an function called white balancing. If improperly white balanced, the color and tone of the whole picture will be wrong.

    Over-exposure at the time.When camera lens is wide open and there's high contrast in the scene, what normally should be a white line could appear purple. That is called "purple fringing" or a "halo" effect. The purple fringe around the edges is mainly caused by color interpolation combined with chromatic aberration. There are little patches of light purple discoloration all around the background and a heavy purple fringing artifact appeared along the leaning branch.we can notice that the corners of the print are getting darker. Darkening of an image around the edges and corners is called vignetting. Its mainly caused by two reasons: One is the design of the camera or lens itself. All lenses falloff toward the corners, and the physical lens barrier would prevent some light from entering the lens. Therefore, the central part of an image would be brighter than that of the parts at the corner or along the edges. The other cause may be light hitting the film or CCD sensor surface at an oblique angle. Its more often occurring when using wide-angle lenses. In flash photography a similar effect can be caused by the coverage of the flash being insufficient to illuminate the field of view.