Post on 14-Dec-2015
Simultaneous surveillance camera calibration and foot-head homology estimation from
human detection1
Author : Micusic & Pajdla
Presenter : Shiu, Jia-Hau Advisor : Wang, Sheng-Jyh
1. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Introduction
• This paper uses people to calibrate the camera • Human contour detection (green)• Refined human detection with camera calibration
parameters (blue)• Foot-head homology(o:foot,x:head)
Concept
• Objects are human • Estimate camera parameters by observing a person
standing at several positions
3-D scene 2-D projection Image
Background
• Shape-based detector(Global search)– Detection rate drop significantly in presence of
occluded humans
• Part-based detector(Local search)
C. Beleznai and H. Bischof. ,“Fast Human Detection in Crowded Scenes by Contour Integration and Local Shape Estimation”, In CVPR,2009.
Background
Left - Shape based : Template matching with head and body
Right - Part based :
Obtain foreground image by background subtraction
Segmentation of detected human
Human Detection
• Line edges model a human• Offline: Create around 1000 human contours based
on 3D model and moving and rotating camera
Background : Camera Model
0
0 c
*[ ]X
*[ ] , u is camera point , X is 3-D point
11
Intrinsic parameters Extrinsic parameters
K= 0 P [ ]
0 0 1
= * = *
c
w
wc
w
x
y
x y
u z K R t
xu
yv z K R t
z
u
v R t
f mx f my
Homography matrix
11 12 13 14
3*4 21 22 23 24
31 32 33 34
11 12 13 14 21 22 23
31 32 33 34
*[ ] *
11 1 1
,
w w w
w w wc
w w w
w w w w w
w w w
x x xu H H H H
y y yv z K R t H H H H H
z z zH H H H
H x H y H z H H x H y H zu v
H x H y H z H
14
31 32 33 34
11 12 13 14 21 22 23 24 31 32 33 34
[ 1 0 0 0 0 - - - -u]* 0
[0 0 0 0 x 1 - - - -v]* 0
, [ ]
w
w w w
w w w w w w
w w w w w w
T
H
H x H y H z H
x y z u x u y u z h
y z v x v y v z h
where h H H H H H H H H H H H H
11 DOF
One pair(2D-3D) of points2 equation
Simple Calibration Example
• Measure 3-D position of special object points in 3-D scene
(0,0,0)
Correspond to camera 2-D point
(0,30,0)
(30,30,40)
(u1,v1)
x
y
z
(u2,v2)
Foot-head Homology Estimation
• 1. Camera model : Shifted Homographies• 2. Focal length, Rotation, Translation• 3. Quadratic Eigenvalue Problem(QEP)• 4. Foot-head Homology
Camera Parameters
• Assumptions intrinsic parameters – Square pixels – No principal point offset : Image coordinate at center
point (principal point) – No skew : angle of horizon axis and vertical axis = 90’
• Intrinsic parameters K = |f 0 0| |0 f 0| |0 0 1|
x
y
90’
x
y
z
(x1,y1,0)
(x1,y1,z0)
(x2,y2,0)
(x2,y2,z0)
(x3,y3,0)
(x3,y3,z0)
If x1,x2,x3,y1,y2,y3 are knownSix points => 12 equationCompute homography of H
x
y
z
(x1,y1,0)
(x1,y1,z0)
(x2,y2,0)
(x2,y2,z0)
(x3,y3,0)
(x3,y3,z0)
If x1,x2,x3,y1,y2,y3 are unknownHow to find homography of H?
x
y
z
(0,0,0)
(0,0,z0)
(x2,y2,0)
(x2,y2,z0)
(x3,y3,0)
(x3,y3,z0)
(0,0,0) & (0,0,z0) two point are known 4 equation
x2
y2
z
x1
y1
z
x
y
z
(0,0,0)
(0,0,z0)
(0,0,0)
(0,0,z0)
(0,0,0)
(0,0,z0)
x1 = x+dx1y1 = y+dy1
x2 = x+dx2y2 = y+dy2
Shifted Homographies
1 2 3 1 2
11 1
points , => 0, 0
3 1 21
1
x dx xu
y dy yv K R t K r r r r dx r dy t
z z
only pick two foot and head x y
uz
v K r r dx r dy t
3+3K unknown, Dof = 3+3K-1
unknown add equationK = 1 6 4K = 2 9 8K = 3 12 12
Shifted Homographies
• The 3D point X = (x, y, z,1) can be simplified assuming x = 0
• ri : is the i-th column of R
• 6+3K unknowns, K : number of detections
Finding Homographies
• This equation is extended with all the known point correspondences to form this equation:
M contains all the point correspondences h contains h1, h2 and the unknown h3 of the
homographies
[ 1 0 0 0 0 - - - -u]* 0
[0 0 0 0 x 1 - - - -v]* 0w w w w w w
w w w w w w
x y z u x u y u z h
y z v x v y v z h
h is fixed for standard camera calibration
Overdetermined Solution
• More than two homologies :solvable as a Quadratic Eigenvalue Problem (QEP)
• Find scalars λ and nonzero vectors x, satisfying (λ2D3 + λD2 + D1)x = 0 • The authors create D1, D2, D3 using the known
values in (7), λ = f.
Solving QEP
• One approach to solving the QEP : Convert it to a linear system (remove the f2):
• Solving ( A - f B ) v = 0
Foot-head Homology
• Result of QEP : K, R, t, f • From this construct the homology HFH with
uH H≃ FH*uF
– uH : image points of head
– uF :image points of feet
(x0k,y0
k,0)
(x0k,y0
k,l)
Hk
Hk
uF
uH
HFH
Camera Image 3-D points