Camera Models and Imaging

download Camera Models and Imaging

of 38

  • date post

    05-Feb-2017
  • Category

    Documents

  • view

    215
  • download

    0

Embed Size (px)

Transcript of Camera Models and Imaging

  • Camera Models and Imaging

    Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition

    CS4243 Camera Models and Imaging 1

  • Through our eyes Through our eyes we see the world.

    CS4243 Camera Models and Imaging 2

  • CS4243 Camera Models and Imaging 3

    lens

    pupil

    cornea

    iris

    retina

    optic nerves

    focus light

    control amount of light entering the eye

    captures image

    transmit image

  • Through their eyes Camera is computers eye.

    CS4243 Camera Models and Imaging 4

  • Camera is structurally the same the eye.

    CS4243 Camera Models and Imaging 5

    lens lens | cornea

    sensor | retina

  • CS4243 Camera Models and Imaging 6

    lens | cornea

    lens

    aperture | pupil

  • CS4243 Camera Models and Imaging 7

    aperture | pupil

    aperture plates | iris

  • Imaging Images are 2D projections of real world scene. Images capture two kinds of information:

    Geometric: positions, points, lines, curves, etc. Photometric: intensity, colour.

    Complex 3D-2D relationships. Camera models approximate relationships.

    CS4243 Camera Models and Imaging 8

  • Camera Models Pinhole camera model Orthographic projection Scaled orthographic projection Paraperspective projection Perspective projection

    CS4243 Camera Models and Imaging 9

  • Pinhole Camera Model

    CS4243 Camera Models and Imaging 10

    image plane pinhole

    virtual image plane

    3D object

    focal length

    2D image

  • Math representation

    CS4243 Camera Models and Imaging 11

    world frame / camera frame

    image frame

    camera centre

    image centre

    optical axis

    projection line

  • By default, use right-handed coordinate system.

    CS4243 Camera Models and Imaging 12

  • 3D point P = (X, Y, Z)T projects to 2D image point p = (x, y)T .

    By symmetry,

    i.e.,

    Simplest form of perspective projection.

    CS4243 Camera Models and Imaging 13

  • Orthographic Projection 3D scene is at infinite distance from camera. All projection lines are parallel to optical axis. So,

    CS4243 Camera Models and Imaging 14

  • Scaled Orthographic Projection

    CS4243 Camera Models and Imaging 15

    Scene depth

  • Perspective Projection

    CS4243 Camera Models and Imaging 16

    camera frame

    world frame

    image frame is parallel to cameras X-Y plane

  • Intrinsic Parameters Pinhole camera model

    Camera sensors pixels not exactly square x, y : coordinates (pixels) k, l : scale parameters (pixels/m) f : focal length (m or mm)

    CS4243 Camera Models and Imaging 17

  • f, k, l are not independent. Can rewrite as follows (in pixels):

    So,

    Image centre or principal point c may not be at origin. Denote location of c in image plane as cx, cy. Then,

    Along optical axis, X = Y = 0, and x = cx, y = cy.

    CS4243 Camera Models and Imaging 18

  • Image frame may not be exactly rectangular. Let denote skew angle between x- and y-axis. Then,

    Combine all parameters yield

    CS4243 Camera Models and Imaging 19

    homogeneous coordinates

    Intrinsic parameter matrix

  • Denoting = Z, we get

    Some books and papers absorb into :

    giving Be careful.

    A simpler form of K uses skew parameter s:

    CS4243 Camera Models and Imaging 20

  • Extrinsic Parameters Camera frame is not aligned with world frame. Rigid transformation between them:

    Coordinates of 3D scene point in camera frame. Coordinates of 3D scene point in world frame. Rotation matrix of world frame in camera frame. Position of world frames origin in camera frame.

    CS4243 Camera Models and Imaging 21

    frame

    object

  • Translation matrix

    Rotation matrix

    CS4243 Camera Models and Imaging 22

  • Rotation matrix is orthonormal:

    So, . In particular, let

    Then,

    CS4243 Camera Models and Imaging 23

  • Combine intrinsic and extrinsic parameters:

    Simpler notation:

    CS4243 Camera Models and Imaging 24

  • Lens Distortion Lens can distort images,

    especially at short focal length.

    CS4243 Camera Models and Imaging 25

    barrel pin-cushion fisheye

  • Radial distortion is modelled as

    with . Actual image coordinates

    CS4243 Camera Models and Imaging 26

    distorted coordinates

    undistorted coordinates

    distortion parameters

  • Camera Calibration Compute intrinsic / extrinsic parameters. Capture calibration pattern from various views.

    CS4243 Camera Models and Imaging 27

  • Detect inner corners in images.

    Run calibration program (available in OpenCV). CS4243 Camera Models and Imaging 28

  • Homography A transformation between projective planes. Maps straight lines to straight lines.

    CS4243 Camera Models and Imaging 29

  • Homography equation:

    Affine is a special case of homography:

    CS4243 Camera Models and Imaging 30

    image point

    image point homography

  • Scaling H by s does not change equation:

    Homography is defined up to unspecified scale. So, can set h33 = 1. Expanding homography equation gives

    CS4243 Camera Models and Imaging 31

  • Assembling equations over all points yields

    System of linear equations. Easy to solve.

    CS4243 Camera Models and Imaging 32

  • Example Circles: input corresponding points. Black dots: computed points, match input points.

    CS4243 Camera Models and Imaging 33

  • Before homographic transform

    CS4243 Camera Models and Imaging 34

  • After homographic transform

    CS4243 Camera Models and Imaging 35

  • Summary Simplest camera model: pinhole model. Most commonly used model: perspective model. Intrinsic parameters:

    Focal length, principal point.

    Extrinsic parameters: Camera rotation and translation.

    Lens distortion Homography: maps lines to lines.

    CS4243 Camera Models and Imaging 36

  • Further Reading Camera models: [Sze10] Section 2.1.5, [FP03]

    Section 1.1, 1.2, 2.2, 2.3. Lens distortion: [Sze10] Section 2.1.6, [BK08]

    Chapter 11. Camera calibration: [BK08] Chapter 11, [Zha00]. Image undistortion: [BK08] Chapter 11.

    CS4243 Camera Models and Imaging 37

  • References Bradski and Kaehler. Learning OpenCV. OReilly, 2008.

    D. A. Forsyth and J. Ponce. Computer Vision: A Modern Approach. Pearson Education, 2003.

    R. Szeliski. Computer Vision: Algorithms and Applications. Springer, 2010.

    Z. Zhang. A flexible new technique for camera calibration. IEEE Trans. PAMI, 22(11):13301334, 2000.

    CS4243 Camera Models and Imaging 38