Jan. 12, 1998 CS260 Winter 1999-Wittenbrink, lect. 3 1 CS 260 Computer Graphics Craig M. Wittenbrink...

Jan. 12, 1998CS260 Winter 1999-Wittenbrink, lect. 31

CS 260 Computer Graphics

Craig M. WittenbrinkLecture 3: Camera Calibration and Views

CS260 Winter 1999-Wittenbrink, lect. 3 2

Overview

• Tracking Applications• Tracking and Calibration (videos)• Camera Models• Tsai Camera Calibration• Project 1 Discussion• Conclusions


Tracking Applications

• Virtual Reality• Augmented Reality• Stereo/Crystal eyes displays• Multiuser Virtual Workbench• 3D Mouse• Animation through motion capture• Gait analysis• Image Based Rendering

(copyright Polhemus 1998)

(copyright Bouguet/CS Caltech 1998)(copyright Derby Gait Lab1998)


Applications for Tracking: Scanning

• Polhemus (www.polhemus.com)• Handheld Laser Scanner (HLS)• Scan by time of flight light sensing to create 3D

model• Magnetic tracking of position and orientation of

wand

Handheld Laser Scanner VRML Surface Model

Copyright Polhemus 1998

Copyright Polhemus 1998


Applications for Tracking: Camera Calibration

• Jean-Yves Bouguet “3D Photography with weak structured lighting” presentation given to Intel Santa Clara, Dec. 5, 1997.– www.vision.caltech.edu/bougetj/

• Use corners of squares to solve for camera intrinsic and extrinsic parameters

• Can then remove camera distortion in images

(copyright Bouguet/CS Caltech 1998)


Applications for tracking: Gait analysis

• Motion capture used for medical analysis, sports training, and natural looking animation– www.gait.com

• Derby Gait Analysis Laboratory• Use 3D Elite System from BTS Milan• Infrared tracking of highly reflective markers• Use multiple cameras• Diagnose and plan treatment

(copyright Derby Gait Lab 1998)


Applications for tracking: Virtual Studio

• InterSense Inertial tracker– www.isense.com

• Use optical gyros and accelerometers• Similar to technology in flight guidance control

systems• Couple with magnetic, ultrasonic, or infrared

tracking, IS 900CT• Use Kalman filtering to fuse sensor input

(copyrightInterSense 1998)


InterSense IS-900CT System

• Solid State inertial measurement– Angular rate of rotation and linear acceleration in 3

axes• Drift removed by Ultrasonic time-of-flight

distance measurements• Infrared triggers beacons

(copyright InterSense 1998)

(copyrightInterSense 1998)


Tracking Videos


Camera Models

• Watt and Watt• OpenGL• Tsai


Camera Models

• Watt and Watt page 7-8 (Figure 1.4)• Center of projection, C• View direction, N• View up vector, V• Positive X axis, U

x

z

y

World Coordinates

C

N

VU

Camera Coordinates


OpenGL Camera Model

• Where camera points by default in world coordinates

• Points down negative z axis, from origin• Specified through model, perspective, and

viewport matrices

x

z

y

World Coordinates

Camera

View direction


Tsai Camera Model

• Distortion due to non pinhole camera– Fig. 1, page 326 from paper: Roger Y. Tsai, “A

Versatile Camera Calibration …”, IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, Aug, 1987, pages 323-344.

image plane

wx

wz

wy

World Coordinates

Camera Coordinates

x

z

y

O

iO

Y

X),( uuu YXP

),( ddd YXP),,( zyxP

),,( www zyxP

or

Effective focal length


Tsai Camera Model

• Looks up positive Z– OpenGL looks down negative Z (or is left handed)

x

z

y

x

z

y

OpenGL camera modelTsai camera model


Camera Calibration

• Two good sources:• Tsai, R.Y. “A Versatile Camera Calibration Technique for High-

Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses”, IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, pages 323-344.

• Code available: www.cs.cmu.edu/People/rgw/TsaiCode.html• Heikkila, Janne and Olli Silven, “A Four-step Camera Calibration

Procedure with Implicit Image Correction”, In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’97), San Juan, Puerto Rico, pages 1106-1112, June 1997.

• Matlab tool box available: www.ee.oulu.fi/~jth/calibr/Calibration.html


Camera Calibration (cont.)

• Basic idea, use known 3D world coordinate points to solve for the camera’s transformation and distortion

• Solve for two types of parameters:– intrinsic parameters-internal camera optical

characteristic– extrinsic parameters-3D position and orientation of

camera


Camera Calibration-Motivation

• Infer 3D location from 2D images• Detect features such as laser spots, corners of

equipment, structured light• Determine Ray in 3D space that object point

must lie on• Applicable to many applications (as seen

previously)• Tsai’s approach: 1) autonomous 2) accurate 3)

reasonably efficient


Tsai’s review of techniques

• 1. Full-scale nonlinear optimization– advantage: accuracy comparable to Tsai– disadvantage: not automatic, compute intensive

– 2. Computing perspective transformation matrix first using linear equation solving– adv: linear optimization used– dis: lens distortion not included, error prone

– 3. Two plane method– adv: linear optimization dis: more unknowns– 4. Geometric technique– adv: linear optimization dis: must provide focal

length


Tsai Experimental Setup

• Use a CCD camera• Aquire a grayscale image, Threshold• Link edge points to extract boundary edges.• Compute across edges to get true edge• fit straight lines, and solve for corners• Letraset sheet positioned on steel block

Find corners


Tsai Transformations

• Four transformations• All encapsulated in the following parameters

– Rotation Rx, Ry, Rz– Translation Tx, Ty, Tz– focal length f– image center Cx, Cy– scale factor sx– radial distortion kappa1


Tsai’s approach, Camera Model

• Four steps in transformation (Fig 2 from paper)

Rigid bodytransformation

ProjectionMatrix

Radial lensdistortion

Viewporttransform

3D WorldCoordinates

3D Cameracoordinates

Ideal undistorted imagecoordinate

Distorted imagecoordinate

Computerimagecoordinate


Tsai’s approach, Camera Model

• Four steps in transformation

wx

wz

wy

World Coordinates

Camera Coordinates

x

z

y

O

iO

Y

X),( uuu YXP


),,( www zyxP

or


Step 1: Rigid Body Transformation

• 3D World Coordinate to 3D camera coordinate system

• Rotation followed by translation• Similar to OpenGL Model Transformation

1110001333231

232221

131211

w

w

w

xyzw

w

w

z

y

x

z

y

x

RRTRz

y

x

Trrr

Trrr

Trrr

z

y

x


Step 2: Perspective Projection

• 3D camera coordinates to Ideal undistorted image coordinate, involves focal length, and perspective foreshortening

z

y

x

zfY

X

u

u1

1


Step 3: Radial Distortion

• is the distorted or true image coordinate on the image plane, and

• The distortion is calculated by)( 2

1rXD dx

1100

10

01

1u

u

y

x

d

d

Y

X

D

D

Y

X

)( 21rYD dy 22

dd YXr

),( dd YX


Step 4: Viewport calculation

• Calculate actual discrete pixel address location within image, with given centering

1100

10

0

1d

d

y

xx

f

f

Y

X

C

Cs

Y

X


Tsai transformation operations

• World space coordinates to computer image space coordinates

1

/1w

w

w

xyzf

f

z

y

x

RRzPTRVD

NA

NA

Y

X

Step 1: Rigid body Step2: Projection

Step 3: distortion

Distorted image coordinates

Undistorted image coordinatescamera coordinates


Tsai’s versus OpenGL

• Four steps in transformation (Fig 2 from paper)

Rigid bodytransformation

ProjectionMatrix

Radial lensdistortion

Viewporttransform

3D WorldCoordinates

3D Cameracoordinates

Ideal undistorted imagecoordinate

Distorted imagecoordinate

Computerimagecoordinate

Model-ViewMatrix

ProjectionMatrix

PerspectiveDivision

ViewportTransformation

Objectcoordinates

eyecoordinates

clipcoordinates

Normalized devicecoordinates

windowcoordinates


OpenGL Camera Model

• glTranslatef(0.0, 0.0, -5.0)• Can see either camera moving, or object

moving, it only modifies the Model-view matrix

x

z

y

-5.0


OpenGL transformation operations

• Normalized device coordinates from object coordinates

1

/1

1o

o

o

cd

d

d

z

y

x

PMwz

y

x

Model-View MatrixProjection Matrix

Perspective division

Normalized device coordinates

Clip coordinatesEye coordinates


OpenGL transformation Operations cont.

• Calculation of window coordinates

• Window center• width height• factor and offset computed with zNear zFar

2/)(]2/)[(

)2/(

)2/(

fnznf

oyp

oxp

z

y

x

d

ydy

xdx

w

w

w

yx oo ,

xp yp

fn


OpenGL transformation Operations cont.

• Viewport set by glviewport(int x, int y, sizei w, sizei h);

• Z depth factor and offset are set by glDepthRange(campd n, clampd f)(zNear, zFar)

2/,2/ hyowxo yx

0.1,0.0 fn


How the calibration code works

• Approach, use the radial alignment as a constraint

• Function of only the relative rotation and translation between the camera and calibration points

wx

wz

wy

World Coordinates

Camera Coordinates

x

z

y

O

iO

Y

X),( uuu YXP


),,( www zyxP

or

Oi and Pd and Pu on same line


Project 1, Set Tsai view in OpenGL

• Description of files• Data set description

No glyphs becauseof occlusion

G01bsmall.gif/ppm Output from my solution


Project 1: File definitions

• Pink blocks show what is to be implemented

Error

G01.corr G01.cpccImageproc.

Tsai

TsaiView

35 mmcamera

Project1G01.ppm

Output image

cmp


Project 1: File definitions cont.

• G01.cor - correlation file• xw,yw,zw, Xu,Yu for all seeable glyph

locations. • Images have been warped to remove radial

distortion• G01.cpcc - view parameters from Tsai: Rx, Ry,

Rz, Tx, Ty, Tz, f, Cx, Cy, sx, kappa1• Code is provided to read *.cor and *.cpcc• Computed Errors are on the order of a pixel ~ 1


Project 1: cont

• 4 file sets are given, different viewpoints• Original images are very large (1536x1024 is

subsampled from Kodak Photo CD resolution 4X in each dimension of that)

• A default view is provided with gluLookAt• You could compute errors with that, and they

will be very large• Read README, run demo scripts RUN, RUN2


Project 1: cont.

• Actual world coordinates description• planes are drawn in provided code at locations

of poster, and glyphs are drawn too.

wx

wz

wy

World Coordinates(x=587,y=588,z=-824)

(x=587,y=588,z=0)

(x=-825,y=-706,z=0)

Delta z = 824Delta y = 1294Delta x = 1412


Conclusions

• Tracking has wide application in 3D graphics• Camera calibration is important for your first

project and specifying view is essential in 3D graphics as well

• Tsai’s parameters were described• Project 1 was reviewed

– Tsai, R.Y. “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses”, IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, pages 323-344.

• Next time: Assigned Reading Szeliski’s slides on image based rendering.

Jan. 12, 1998 CS260 Winter 1999-Wittenbrink, lect. 3 1 CS 260 Computer Graphics Craig M. Wittenbrink...

Documents

Transcript of Jan. 12, 1998 CS260 Winter 1999-Wittenbrink, lect. 3 1 CS 260 Computer Graphics Craig M. Wittenbrink...