A Unified Approach to Calibrate a Network of Camcorders & ToF Cameras

M2SFA2 Marseille France 2008

Li Guan Marc Pollefeys

{lguan, marc}@cs.unc.edu

UNC-Chapel Hill, USA ETH-Zurich, Switzerland

2

ToF Camera (RIM sensor)

• Theory– Time of Flight

Fig. from 3DV system website

• Products– Canesta cameras– Swiss Ranger– PMD cameras– ZCam

3

• Advantage – high frame-rate (50 fps.)

– Depth image + amplitude image

• Drawback– low resolution (e.g. 176x144, SR3100)

– depth measurement is still not stable

• Solution for reconstruction:– A network of ToF cameras & video camcorders

– Challenges• calibration

• robust shape estimation

http://www.3dcgi.com/images/face_2d_3d.jpg

3D Sensors (cont.)

4

• Recovering sensor location, orientation and imaging parameters

• Traditional calibration target– Checkerboard

Z. Zhang ICCV’99

J.-Y. Bouget’s toolbox

– Laser pointer, etcT.Svoboda MIT press ’05

Svobod’s toolbox

• Our proposal– A sphere

with unknown radius

Calibration of the Sensor Network

5

• Video Camcorder– Observation: due to projective distortion, the image of a sphere is

an ellipse, and sphere center is NOT the center of the ellipse,– An ellipse is defined with 5 parameters– If we know the intrinsics of the camera, it can be simplified to 3

Hough transform

Sphere Center Extraction

6

Hough Transform• Given the

undistorted optical center position, the ellipse detection is a 3-parameter Hough transform

– Radius of the sphere tangent to the cone at plane Z=-1

– Row and Col of the sphere center in the image

• Fit the final result to get sub-pixel accuracy

7

• ToF Camera– Observation: intensity highlight in the “amplitude image”

Detect & track the sphere highlight

Fit parabolic surface to get sub-pixel

accuracy

Sphere Center Extraction (cont.)

Camera optical center

8

• Setup– 4 fixed position vision

sensors• 2 Canon HG10,

1920x1080, 25Hz

• 2 SR3100, 176x144,20Hz

Calibration Result

9

• Radius recovery

• Scale recovery

Sphere Radius & Scale Recovery

R = 0.0248

S = 11.3386

R’ = RS =0.0248x11.3386 = 0.2824m

Measured circumference = 1.7925m, the actual radius = 0.2853m

10

• Overview

Robust Shape Estimation

11

Sensor Fusion Framework

• Notations– as the binary state space– as the sensor models– as the sensor observations

(L. Guan, J.-S. Franco, M. Pollefeys, 3DPVT 2008)

12

Main Formula

• Bayes rule

13

ResultsFor MATLAB code, check out

http://www.cs.unc.edu/~lguan

Volume size 2563

Threshold at 0.875

Computation Time ~ 3 min. (MATLAB)

14

Summary & Future Work• Calibration

– Depth calibration • Separate scale factor for each sensor

• reflection - depth accuracy analysis

• Reconstruction– More general sensor fusion– Ultimate challenge of outdoor environment

• Synchronization and video processing• GPU Algorithm speedup