Camera models and single view geometry. Camera model.

of 86/86
amera models and single view geomet
  • date post

    20-Dec-2015
  • Category

    Documents

  • view

    224
  • download

    0

Embed Size (px)

Transcript of Camera models and single view geometry. Camera model.

  • Slide 1
  • Camera models and single view geometry
  • Slide 2
  • Camera model
  • Slide 3
  • Camera: optical system d 21 thin lens small angles: Y Z 2 1 curvature radius
  • Slide 4
  • Y Z incident light beam deviated beam deviation angle ? lens refraction index: n
  • Slide 5
  • Thin lens rules a) Y=0 = 0 f Y parallel rays converge onto a focal plane b) f = Y beams through lens center: undeviated independent of y
  • Slide 6
  • r f Y h Where do all rays starting from a scene point P converge ? Z Fresnel law P Obs. For Z , r f O p ?
  • Slide 7
  • d f a Z if d r focussed image: blurring circle)
  • Slide 8
  • the image of a point P belongs to the line (P,O) p P O p = image of P = image plane line(O,P) interpretation line of p: line(O,p) = locus of the scene points projecting onto image point p image plane r f Hp: Z >> a
  • Slide 9
  • Slide 10
  • the image of a point P belongs to the line (P,O) p P O p = image of P = image plane line(O,P) interpretation line of p: line(O,p) = locus of the scene points projecting onto image point p image plane r f Hp: Z >> a
  • Slide 11
  • p P O Z Y X c y x perspective projection f -nonlinear -not shape-preserving -not length-ratio preserving
  • Slide 12
  • Point [x,y] T expanded to [u,v,w] T Any two sets of points [u 1,v 1,w 1 ] T and [u 2,v 2,w 2 ] T represent the same point if one is multiple of the other [u,v,w] T [x,y] with x=u/w, and y=v/w [u,v,0] T is the point at the infinite along direction (u,v) In 2D: add a third coordinate, w Homogeneous coordinates
  • Slide 13
  • Transformations translation by vector [ d x,d y ] T scaling (by different factors in x and y) rotation by angle
  • Slide 14
  • Homogeneous coordinates In 3D: add a fourth coordinate, t Point [X,Y,Z] T expanded to [x,y,z,t] T Any two sets of points [x 1,y 1,z 1,t 1 ] T and [x 2,y 2,z 2,t 2 ] T represent the same point if one is multiple of the other [x,y,z,t] T [X,Y,Z] with X=x/t, Y=y/t, and Z=z/t [x,y,z,0] T is the point at the infinite along direction (x,y,z)
  • Slide 15
  • Transformations scaling translation rotation Obs: rotation matrix is an orthogonal matrix i.e.: R -1 = RTRT
  • Slide 16
  • Pinhole camera model
  • Slide 17
  • with Scene->Image mapping: perspective transformation With ad hoc reference frames, for both image and scene
  • Slide 18
  • Let us recall them O Z Y X c y x f scene reference - centered on lens center - Z-axis orthogonal to image plane - X- and Y-axes opposite to image x- and y-axes image reference - centered on principal point - x- and y-axes parallel to the sensor rows and columns - Euclidean reference
  • Slide 19
  • O Z Y X c y x f scene reference - not attached to the camera image reference - centered on upper left corner - nonsquare pixels (aspect ratio) noneuclidean reference Actual references are generic principal axis principal point
  • Slide 20
  • Principal point offset principal point
  • Slide 21
  • CCD camera
  • Slide 22
  • Scene-image relationship wrt actual reference frames image scene normally, s=0
  • Slide 23
  • K upper triangular : intrinsic camera parameters scene-camera tranformation extrinsic camera parameters orthogonal (3D rotation) matrix P: 10-11 degrees of freedom (10 if s=0)
  • Slide 24
  • i.e., defining x = [x, y, z] T with and
  • Slide 25
  • The locus of the points x whose image is u is a straight line through o having direction is independent of u o is the camera viewpoint (perspective projection center) line(o, d) = Interpretation line of image point u Interpretation of o: u is image of x if i.e., if
  • Slide 26
  • Intrinsic and extrinsic parameters from P M K and R RQ-decomposition of a matrix: as the product between an orthogonal matrix and an upper triangular matrix M and m t
  • Slide 27
  • Camera anatomy Camera center Column points Principal plane Axis plane Principal point Principal ray
  • Slide 28
  • Camera center null-space camera projection matrix For all A all points on AO project on image of A, therefore O is camera center Image of camera center is (0,0,0) T, i.e. undefined Finite cameras: Infinite cameras:
  • Slide 29
  • Column vectors Image points corresponding to X,Y,Z directions and origin
  • Slide 30
  • Row vectors Image of a point on the principal plane (the plane of the thin lens) is at the infinity w = 0 is the principal plane
  • Slide 31
  • note: p 1,p 2 dependent on image reparametrization is the plane through the u-axis is the plane through the v-axis similarly,
  • Slide 32
  • The principal point principal point
  • Slide 33
  • The principal axis vector vector defining front side of camera the direction of the normal to the principal plane
  • Slide 34
  • Action of projective camera on point Forward projection Back-projection (pseudo-inverse)
  • Slide 35
  • Depth of points (dot product)(PO=0) If, then m 3 unit vector in positive direction
  • Slide 36
  • Camera matrix decomposition Finding the camera center (use SVD to find null-space) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) Q R =( ) -1 = -1 -1 Q R (if only QR, invert)
  • Slide 37
  • When is skew non-zero? 1 arctan(1/s) for CCD/CMOS, always s=0 Image from image, s0 possible (non coinciding principal axes) resulting camera:
  • Slide 38
  • Euclidean vs. projective general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space
  • Slide 39
  • Camera calibration
  • Slide 40
  • from scene-point to image point correspondence to projection matrix
  • Slide 41
  • Basic equations
  • Slide 42
  • Basic equations ctd. with(12x1) singular matrix
  • Slide 43
  • minimal solution over-determined solution 5 correspondences needed (say 6) P has 11 dof, 2 independent eq./points n 6 points minimize subject to constraint p : eigenvector of A T A associated to its smallest eigenvalue
  • Slide 44
  • Degenerate configurations More complicate than 2D case (see Ch.21) (i)Camera and points on a twisted cubic (ii)Points lie on plane or single line passing through projection center
  • Slide 45
  • Data normalization (i)translate origin to gravity center (ii)(an)isotropic scaling
  • Slide 46
  • from line correspondences Extend DLT to lines (back-project image line) (2 independent eq.)
  • Slide 47
  • Geometric error
  • Slide 48
  • Gold Standard algorithm Objective Given n6 2D to 2D point correspondences {X i x i }, determine the Maximum Likelyhood Estimation of P Algorithm (i)Linear solution: (a)Normalization: (b)DLT: (ii)Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error: (iii)Denormalization: ~~ ~
  • Slide 49
  • Calibration example (i)Canny edge detection (ii)Straight line fitting to the detected edges (iii)Intersecting the lines to obtain the images corners typically precision
  • Slide 50
  • Exterior orientation Calibrated camera, position and orientation unkown Pose estimation 6 dof 3 points minimal (4 solutions in general)
  • Slide 51
  • Slide 52
  • short and long focal length Radial distortion
  • Slide 53
  • Slide 54
  • Slide 55
  • Slide 56
  • Correction of distortion Choice of the distortion function and center Computing the parameters of the distortion function (i)Minimize with additional unknowns (ii)Straighten lines (iii)
  • Slide 57
  • Properties of perspective transformations 1) vanishing points V image of the unproper point along direction d the interpretation line of V is parallel to d
  • Slide 58
  • O V d The images of parallel lines are concurrent lines
  • Slide 59
  • 2) cross ratio invariance Given four colinear points letbe their abscissae Properties of perspective transformations ctd.
  • Slide 60
  • Cross ratio invariance under perspective transformation a point on the line y=0=z its image belongs to a line its coordinate u
  • Slide 61
  • Object localization 1: three colinear points geometric model of an object a perspective image of the object position and orientation of the object ? A B C C A B O calibrated camera: A B C known known interpretation lines
  • Slide 62
  • A B C C A B O V a) orientation Cross ratio invariance: solve for V (image of ) V: vanishing point of the direction of (A,B,C) interpretation line of V parallel to (A,B,C)direction
  • Slide 63
  • b) position (e.g., distance(O,A)) A B C C A B O V interpretation lines angles and
  • Slide 64
  • Object localization 2: four coplanar points O (i)orientation of (A,E,C) (ii)orientation of (B,E,D) (iii)distance (O,A) E A B C D
  • Slide 65
  • a b b a Find vanishing point of the field-bottom direction Off-side images of symmetric segments
  • Slide 66
  • a and b : abscissae of the endpoints of a segment c=(a+b)/2 : abscissa of segment midpoint, d= : point at the infinite along the segment direction ac bd ( a,b ) and ( a, b ) are image of symmetric segments same image of the midpoint c, same vanishing point d Harmonic 4-tuple ( a,b,c,d )
  • Slide 67
  • solve { for c, d system of two linear equations in ( cd ) and ( c+d ) two degree equation, whose solutions are c and d among the two solutions, the one for d is the value external to the range [a,b]
  • Slide 68
  • Action of projective camera on planes The most general transformation that can occur between a scene plane and an image plane under perspective imaging is a plane projective transformation
  • Slide 69
  • Action of projective camera on lines forward projection back-projection with Interpretation plane of line l
  • Slide 70
  • Image of a conic therefore
  • Slide 71
  • Action of projective camera on conics back-projection of a conic C to cone C
  • Slide 72
  • example: with Interpretation cone of a conic C back-projection of a conic C to cone
  • Slide 73
  • Images of smooth surfaces The contour generator is the set of points X on S at which rays are tangent to the surface. The corresponding apparent contour is the set of points x which are the image of X, i.e. is the image of The contour generator depends only on position of projection center, depends also on rest of P
  • Slide 74
  • Action of projective camera on quadrics apparent contour of a quadric Q dual quadric is a plane quadric: the set of planes tangent to Q Let us consider only those planes that are backprojection of image lines with its dual is
  • Slide 75
  • The plane containing the apparent contour of a quadric Q from a camera center O follows from pole-polar relationship The cone with vertex V and tangent to the quadric Q is back-projection to cone =QO
  • Slide 76
  • What does calibration give? An image line l defines a plane through the camera center with normal n=K T l measured in the cameras Euclidean frame. In fact the backprojection of l is P T l=K T l
  • Slide 77
  • The image of the absolute conic mapping between to an image is given by the planar homogaphy x=Hd, with H=KR absolute conic (IAC), represented by I 3 within ( w=0 ) (i)IAC depends only on intrinsics (ii)angle between two rays (iii)DIAC= * =KK T (iv) K (Cholesky factorization) (v)image of circular points belong to (image of absolute conic) its image (IAC)
  • Slide 78
  • A simple calibration device (i)compute H i for each square (corners (0,0),(1,0),(0,1),(1,1)) (ii)compute the imaged circular points H i [1,i,0] T (iii)fit a conic to 6 imaged circular points (iv)compute K from K -T K -1 through Cholesky factorization (= Zhangs calibration method)
  • Slide 79
  • Orthogonality = pole-polar w.r.t. IAC
  • Slide 80
  • The calibrating conic
  • Slide 81
  • Vanishing points
  • Slide 82
  • Vanishing lines
  • Slide 83
  • Slide 84
  • Orthogonality relation
  • Slide 85
  • Calibration from vanishing points and lines
  • Slide 86