Slide 1

EEM 561 Machine Vision Week 10 :Image Formation and Cameras Spring 2015 Instructor: Hatice nar Akakn, Ph.D. haticecinarakakin@anadolu.edu.tr Anadolu University

Slide 2

Figures Stephen E. Palmer, 2002 Image formation 3D world2D image Slide source: A.Torralba Images are projections of the 3-D world onto a 2-D plane

Slide 3

Image formation Lets design a camera Idea 1: put a piece of film in front of an object Do we get a reasonable image? Slide source: Seitz

Slide 4

Pinhole camera Add a barrier to block off most of the rays This reduces blurring The opening known as the aperture How does this transform the image? Slide source: Seitz The barrier blocks off most of the rays It gets inverted!!

Slide 5

Light rays from many different parts of the scene strike the same point on the paper. Forsyth & Ponce Each point on the image plane sees light from only one direction, the one that passes through the pinhole.

Slide 6

Pinhole camera Figure from Forsyth f f = focal length c = center of the camera c If we treat pinhole as a point, only one ray from any given point can enter the camera. Pinhole camera is a simple model to approximate imaging process, perspective projection

Slide 7

Photograph by Abelardo Morell, 1991 Pinhole camera Slide source: A.Torralba

Slide 8

Photograph by Abelardo Morell, 1991 Pinhole camera Slide source: A.Torralba

Slide 9

Photograph by Abelardo Morell, 1991 Pinhole camera Slide source: A.Torralba

Slide 10

Photograph by Abelardo Morell, 1991 Pinhole camera Slide source: A.Torralba

Slide 11

Effect of pinhole size Wandell, Foundations of Vision, Sinauer, 1995

Slide 12

Slide 13

Shrinking the aperture Why not make the aperture as small as possible? Less light gets through Diffraction effects... Slide source:N.Snavely

Slide 14

Shrinking the aperture Slide source:N.Snavely

Slide 15

Camera obscura: The pre- camera "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January 24, 1544, and later he used this illustration of the event in his book De Radio Astronomica et Geometrica, 1545. It is thought to be the first published illustration of a camera obscura..." Hammond, John H., The Camera Obscura, A Chronicle http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html In Latin, means dark room

Slide 16

Camera Obscura

Slide 17

Camera obscura Illustration of Camera ObscuraFreestanding camera obscura at UNC Chapel Hill Photo by Seth Ilys

Slide 18

Camera obscura at home Sketch from http://www.funsci.com/fun3_en/sky/sky.htm http://blog.makezine.com/archive/2006/02/how_to_room_ sized_camera_obscu.html

Slide 19

Accidental pinhole camera See Zomet, A.; Nayar, S.K. CVPR 2006 for a detailed analysis. Outside scene * Aperture Slide source: A.Torralba

Slide 20

Measuring distance Object size decreases with distance to the pinhole There, given a single projection, if we know the size of the object we can know how far it is. But for objects of unknown size, the 3D information seems to be lost.

Slide 21

Adding a lens A lens focuses light onto the film There is a specific distance at which objects are in focus other points project to a circle of confusion in the image Changing the shape of the lens changes this distance circle of confusion Slide source:N.Snavely

Slide 22

Cameras with lenses focal point F optical center (Center Of Projection) A lens focuses parallel rays onto a single focal point Gather more light, while keeping focus; make pinhole perspective projection practical Slide source:K.Grauman

Slide 23

Thin lens equation Any object point satisfying this equation is in focus Slide source:K.Grauman

Slide 24

Combining Lenses

Slide 25

The eye The human eye is a camera Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris Whats the film? photoreceptor cells (rods and cones) in the retina

Slide 26

Perspective projection y y f z camera world Cartesian coordinates: We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f) Ignore the third coordinate, and get Slide source: A.Torralba f: focal length O: camera center

Slide 27

Points go to points Lines go to lines Planes go to whole image or half-planes. Polygons go to polygons Degenerate cases line through focal point to point plane through focal point to line Geometric properties of projection Slide source: A.Torralba

Slide 28

Modeling projection Is this a linear transformation? Homogeneous coordinates to the rescue! homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates nodivision by z is nonlinear Slide by Steve Seitz

Slide 29

Perspective Projection Matrix divide by the third coordinate to convert back to non-homogeneous coordinates Projection is a matrix multiplication using homogeneous coordinates: Slide by Steve Seitz

Slide 30

Perspective Projection -- Ideal Case

Slide 31

Slide 32

Orthographic projection Given camera at constant distance from scene World points projected along rays parallel to optical access

Slide 33

Projection properties Parallel lines converge at a vanishing point Each direction in space has its own vanishing point But parallels parallel to the image plane remain parallel Slide source:N.Snavely

Slide 34

Vanishing points and lines Vanishing point Vanishing line Vanishing point Vertical vanishing point (at infinity) Slide from Efros, Photo from Criminisi source:J.Hays

Slide 35

Homogeneous coordinates 2D Points: 2D Lines: d (n x, n y ) Slide source: A.Torralba

Slide 36

Homogeneous coordinates Intersection between two lines: Slide source: A.Torralba

Slide 37

Homogeneous coordinates Line joining two points: Slide source: A.Torralba

Slide 38

2D Transformations tx ty = + 1 = 10tx 01ty. = 10tx 01ty 001. Example: translation Now we can chain transformations Slide source: A.Torralba

Slide 39

Recall:Summary of Affine Transformations

Slide 40

More Realistic Perspective Projection

Slide 41

Perspective projection (intrinsics) in general, : aspect ratio (1 unless pixels are not square) : skew (0 unless pixels are shaped like rhombi/parallelograms) : principal point ((0,0) unless optical axis doesnt intersect projection plane at origin) (upper triangular matrix) (converts from 3D rays in camera coordinate system to pixel coordinates) Slide source:N.Snavely

Slide 42

Slide Credit: Saverese Projection matrix x: Image Coordinates: (u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X: World Coordinates: (X,Y,Z,1) OwOw iwiw kwkw jwjw R,T

Slide 43

K Slide Credit: Saverese Projection matrix Intrinsic Assumptions Unit aspect ratio Optical center at (0,0) No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Slide 44

Remove assumption: known optical center Intrinsic Assumptions Unit aspect ratio No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Slide 45

Remove assumption: square pixels Intrinsic Assumptions No skew Extrinsic Assumptions No rotation Camera at (0,0,0)

Slide 46

Remove assumption: non-skewed pixels Intrinsic AssumptionsExtrinsic Assumptions No rotation Camera at (0,0,0) Note: different books use different notation for parameters Slide Credit: J. Hays

Slide 47

Oriented and Translated Camera OwOw iwiw kwkw jwjw t R Slide Credit: J. Hays

Slide 48

Allow camera translation Intrinsic AssumptionsExtrinsic Assumptions No rotation Slide Credit: J. Hays

Slide 49

3D Rotation of Points, : Rotation around the coordinate axes, counter-clockwise: p pppp y z Slide Credit: Saverese

Slide 50

Allow camera rotation Slide source:J.Hays

Slide 51

Degrees of freedom 56 How many known points are needed to estimate this? Slide source:J.Hays

Slide 52

Camera calibration Use the camera to tell you things about the world: Relationship between coordinates in the world and coordinates in the image: geometric camera calibration, see Szeliski, section 5.2, 5.3 for references (Relationship between intensities in the world and intensities in the image: photometric image formation, see Szeliski, sect. 2.2.) Slide source: A.Torralba

Slide 53

Things to remember Vanishing points and vanishing lines Pinhole camera model and camera projection matrix Homogeneous coordinates Vanishing point Vanishing line Vanishing point Vertical vanishing point (at infinity)