Image formation

Image formation

ECE 847:Digital Image Processing

Stan BirchfieldClemson University

Cameras

• First photograph due to Niepce

• Basic abstraction is the pinhole camera– lenses required to ensure image is not too

dark– various other abstractions can be applied

F. Dellaert, http://www.cc.gatech.edu/~dellaert/vision/html/materials.html

http://auden.hrc.utexas.edu/PhotoFiles/dbimages/d2k0030.html

Image formation overview

Image formation involves• geometry – path traveled by light• radiometry – optical energy flow• photometry – effectiveness of light to produce

“brightness” sensation in human visual system• colorimetry – physical specifications of light

stimuli that produce given color sensation• sensors – converting photons to digital form

Pinhole camera

D. Forsyth, http://luthuli.cs.uiuc.edu/~daf/book/bookpages/slides.html

Parallel lines meet: vanishing point

• each set of parallel lines (=direction) meets at a different point– The vanishing point for

this direction

• Sets of parallel lines on the same plane lead to collinear vanishing points. – The line is called the

horizon for that plane

Perspective projection

k

O

P

Q

j

ip

q

C

f

Properties of projection:• Points go to points• Lines go to lines• Planes go to whole image• Polygons go to polygons• Degenerate cases

– line through focal point to point

– plane through focal point to line


Perspective projection (cont.)

Z

Y

w

vv

Z

X

w

uu

T

Z

Y

X

PI

w

v

u

p

ˆ

ˆ

:gnormalizinby scoordinate )(Euclidean imageRecover

0100

0010

0001

]0[

scoordinate e)(projectiv shomogeneou ofn nsformatioLinear tra


Weak perspective projection

• perspective effects, but not over the scale of individual objects

• collect points into a group at about the same depth, then divide each point by the depth of its group


Weak perspective (cont.)

Pretend depth is constant (often OK !)

ˆ u X

Zr

ˆ v Y

Zr

Can also be written as a linear transformation :

u

v

w

1

Zr

1 0 0 0

0 1 0 0

0 0 0 Zr

X

Y

Z

T


Orthographic projection

Let Z0=1:


Pushbroom cameras

Pinhole too big - many directions are averaged, blurring the image

Pinhole too small- diffraction effects blur the image

Generally, pinhole cameras are dark, becausea very small set of raysfrom a particular pointhits the screen.

Pinhole size


The reason for lenses


The thin lens

z 1

z'

1

f

†

1

z'-

1

z=

1

f

focal points


Focusing

http://www.theimagingsource.com

Thick lens

• thick lens has 6 cardinal points:– two focal points (F1 and F2)

– two principal points (H1 and H2)

– two nodal points (N1 and N2)

• complex lens is formed by combining individual concave and convex lenses

http://physics.tamuk.edu/~suson/html/4323/thick.html


Complex lens

http://www.cambridgeincolour.com/tutorials/camera-lenses.htm

All but the simplest cameras contain lenses which are actually composed of several lens elements

Choosing a lens

• How to select focal length:– x=fX/Z– f=xZ/X

• Lens format should be >= CCD format to avoid optical flaws at the rim of the lens

http://www.theimagingsource.com/en/resources/whitepapers/download/choosinglenswp.en.pdf

Lenses – Practical issues

• standardized lens mount has two varieties:– C mount– CS mount

• CS mount lenses cannot be used with C mount cameras

http://www.theimagingsource.com/en/resources/whitepapers/download/choosinglenswp.en.pdf

Spherical aberration

perfect lens actual lens

On a real lens, even parallel rays are not focused perfectly

http://en.wikipedia.org/wiki/Spherical_aberration

Chromatic aberration

On a real lens, different wavelengths are not focused the same

http://en.wikipedia.org/wiki/Chromatic_aberration

Radial distortion

straight lines are curved:

uncorrected corrected

Radial distortion (cont.)

barrel distortion(more common)

pincushion distortion

http://en.wikipedia.org/wiki/Image_distortion

pincushion

barrel

http://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpg

Two types:

Vignetting

vignetting – reduction of brightness at periphery of imageD. Forsyth, http://luthuli.cs.uiuc.edu/~daf/book/bookpages/slides.html

Normalized Image coordinates

P

Ou=X/Z = dimensionless !

1


Pixel units

P

Ou=k f X/Z = in pixels !

[f] = m (in meters)[k] = pixels/m

f

Pixels are on a grid of a certain dimension


Pixel coordinates

P

Ou=u0 + k f X/Z

f

We put the pixel coordinate origin on topleft


Pixel coordinates in 2D

j

i(u0,v0)

(0.5,0.5) 640

480

(640.5,480.5)

u0 kfX

Z,v0 lf

Y

Z


Summary: Intrinsic Calibration

33 Calibration Matrix K

p u

v

w

K[I 0]P

s u0

v0

1

1 0 0 0

0 1 0 0

0 0 1 0

X

Y

Z

T

Recover image (Euclidean) coordinates by normalizing:

ˆ u uw

X sY u0

Z

ˆ v v

w

Y v0

Z

skew

5 Degrees of Freedom !


Camera Pose

In order to apply the camera model, objects in the scenemust be expressed in camera coordinates.

WorldCoordinates

CameraCoordinates

Calibration target looks tilted from cameraviewpoint. This can be explained as adifference in coordinate systems.

wc T

y

x

z

z

x

y


Rigid Body Transformations

• Need a way to specify the six degrees-of-freedom of a rigid body.

• Why are there 6 DOF?

A rigid body is acollection of pointswhose positionsrelative to eachother can’t change

Fix one point,three DOF

Fix second point,two more DOF(must maintaindistance constraint)

Third point addsone more DOF,for rotationaround line


Notations

• Superscript references coordinate frame• AP is coordinates of P in frame A• BP is coordinates of P in frame B

• Example:

A P

A xA yA z

OP A x iA A y jA A z kA

kA

jA

iA

OA

PF. Dellaert, http://www.cc.gatech.edu/~dellaert/vision/html/materials.html

Translation

kA

jA

iA

kB

jB

iB

OB

OA

B PAPBOA

P


Translation

• Using homogeneous coordinates, translation can be expressed as a matrix multiplication.

• Translation is commutative

B A BAP P O

1 0 1 1

B B AAP I O P


Rotation

A B

A BA A A B B B

A B

x x

OP i j k y i j k y

z z

77777777777777

B B AAP R P

BA R

means describing frame A inThe coordinate system of frame B


Rotation

. . .

. . .

. . .

A B A B A BBA A B A B A B

A B A B A B

R

i i j i k i

i j j j k j

i k j k k k

B B BA A A i j k

Orthogonal matrix!

A TB

A TB

A TB

i

j

k


Example: Rotation about z axis

What is the rotation matrix?


Rotation in homogeneous coordinates

• Using homogeneous coordinates, rotation can be expressed as a matrix multiplication.

• Rotation is not communicative

B B AAP R P

0

1 0 1 1

B B AAP R P


Rigid transformations

B B A BA AP R P O


Rigid transformations (con’t)

• Unified treatment using homogeneous coordinates.

1 0

1 0 1 0 1 1

1 1

B B B AA A

B B AA A

T

P O R P

R O P

0

1 1

B ABA

P PT


Projective Camera Matrix

CameraCalibrationProjectionExtrinsics

p u

v

w

K[I 0]TP

s u0

v0

1

1 0 0 0

0 1 0 0

0 0 1 0

R t

0 1

X

Y

Z

T

K R t P MP

5+6 DOF = 11 !


Projective Camera Matrix

T

Z

Y

X

mmmm

mmmm

mmmm

w

v

u

MPPtRKp

34333231

24232221

14131211

5+6 DOF = 11 !


Columns & Rows of M

4321

3

2

1

mmmm

m

m

m

M

m2P=0

i

ii

i

ii

Pm

Pmv

Pm

Pmu

3

2

3

1

O


Effect of Illumination

Light source strength and direction has a dramatic impact on distribution of brightness in the image (e.g. shadows, highlights, etc.)

(Subject 8 from the Yale face database due to P. Belhumeur et. al.)


Image formation

• Light source emits photons

• Absorbed, transmitted, scattered

• fluorescenceCamerasource


Surfaces receives and emits

• Incident light from lightfield

• Act as a light source

• How much light ?


Irradiance

• Irradiance – amount of light falling on a surface patch

• symbol=E, units = W/m2

dA


Radiosity

• power leaving a point per area

• symbol=B, units = W/m2

dA


Light = Directional

• Light emitted varies w. direction


Steradians (Solid Angle)

• 3D analogue of 2D angle

A

R

angle L

R, circle 2 radians

solid angle A

R2, sphere 4 steradians


Steradians (cont’d)

1 m2 subtends a solid angle of 1 steradian (sr).

Sphere 4, hemisphere 2 steradians.

Solid angle of a small planar patch of area d A at a distance R :

R

A

d dAcos

R2


Polar Coordinates

Any direction from patch can be expressed as two angles ,.

d d(d sin)


Intensity

• Intensity – amount of light emitted from a point per steradian

• symbol=I, units = W/sr

intensity I


QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Irradiance and Intensity

dA

Example : point light source :

intensity I /4

watts

irradiance E d /dA Id /dA I /r2

intensity I


Radiance

• Radiance – amount of light passing through an area dA and

• symbol=L, units = W x m-2 x sr-1

radiance L(P,)

P

dA

d


Radiance is important

• Response of camera/eye is proportional to radiance

• Pixel values

• Constant along a ray


Lightfield = Gibson optic array !

• 5DOF: Position = 3DOF, 2 DOF for direction


Lightfield Sampler


Lightfield Sample


Lambertian Emitters

• Lambertian = constant radiance• More photons emitted straight up• Oblique: see fewer photons, but area looks smaller• Same brightness !• Total power is proportional to wedge area• “Cosine law”• Sun approximates Lambertian: Different angle, same

brightness• Moon should be less bright at edges, as gets less light from

sun.• Reflects more light at grazing angles than a Lambertian

reflector


Radiance Emitted/Reflected

• Radiance – amount of light emitted from a surface patch per steradian per area

• foreshortened !

dA

radiance L(P,,)


Calculating Radiosity

constant radiance L

radiosity B Lcos d Lcos sindd L

radiance L(P,,)

If reflected light is not dependent on angle, then can integrate over angle: radiosity is an approximate radiometric unit


Example: Sun

• Power= 3.91 1026 W

• Surface Area:6.07 1018 m2

• Power = Radiance . Area . • L = 2.05 107 W/m2.sr

Example from P. Dutre SIGGRAPH tutorialF. Dellaert, http://www.cc.gatech.edu/~dellaert/vision/html/materials.html

http://images.google.com/imgres?imgurl=http://www.geo.cornell.edu/ses/earth.jpg&imgrefurl=http://www.geo.cornell.edu/ses/SES_home_new.html&h=434&w=480&sz=80&tbnid=IlyjT2eBNGAJ:&tbnh=112&tbnw=124&start=13&prev=/images%3Fq%3Dearth%26hl%3Den%26lr%3D%26ie%3DUTF-8%26sa%3DN

http://images.google.com/imgres?imgurl=http://www.sufistudies.net/art/sun-outline-small.jpg&imgrefurl=http://www.sufistudies.net/&h=200&w=201&sz=30&tbnid=Iccnr236ED4J:&tbnh=98&tbnw=98&start=47&prev=/images%3Fq%3Dsun%26start%3D40%26hl%3Den%26lr%3D%26ie%3DUTF-8%26sa%3DN

Irradiance (again)

• Integrate incoming radiance over hemisphere

i


Example: Sun

irradiance E L dA cosd

Area

(2.05107W/m2.sr)(1m2)(6.710 5 sr)

1373.5W


http://images.google.com/imgres?imgurl=http://www.geo.cornell.edu/ses/earth.jpg&imgrefurl=http://www.geo.cornell.edu/ses/SES_home_new.html&h=434&w=480&sz=80&tbnid=IlyjT2eBNGAJ:&tbnh=112&tbnw=124&start=13&prev=/images%3Fq%3Dearth%26hl%3Den%26lr%3D%26ie%3DUTF-8%26sa%3DN

http://images.google.com/imgres?imgurl=http://www.sufistudies.net/art/sun-outline-small.jpg&imgrefurl=http://www.sufistudies.net/&h=200&w=201&sz=30&tbnid=Iccnr236ED4J:&tbnh=98&tbnw=98&start=47&prev=/images%3Fq%3Dsun%26start%3D40%26hl%3Den%26lr%3D%26ie%3DUTF-8%26sa%3DN

BRDF

L

E

oi

Bidirectional Reflectance Distribution Function (BRDF) :

(o,o;i,i) outgoing radiance L

incident irradiance E

Symmetric in incoming and outgoing directions – this is the Helmholtz reciprocity principle


BRDF Example


Lambertian surfaces and albedo

• For some surfaces, the DHR is independent of illumination direction too– cotton cloth, carpets,

matte paper, matte paints, etc.

• For such surfaces, radiance leaving the surface is independent of angle

• Called Lambertian surfaces (same Lambert) or ideal diffuse surfaces

• Use radiosity as a unit to describe light leaving the surface

• DHR is often called diffuse reflectance, or albedo

• for a Lambertian surface, BRDF is independent of angle, too.

• Useful fact:

brdf d


Specular surfaces

• Another important class of surfaces is specular, or mirror-like.– radiation arriving along a

direction leaves along the specular direction

– reflect about normal– some fraction is absorbed,

some reflected– on real surfaces, energy

usually goes into a lobe of directions

– can write a BRDF, but requires the use of funny functions


Lambertian + specular

• Widespread model– all surfaces are Lambertian plus specular component

• Advantages– easy to manipulate

– very often quite close true

• Disadvantages– some surfaces are not

• e.g. underside of CD’s, feathers of many birds, blue spots on many marine crustaceans and fish, most rough surfaces, oil films (skin!), wet surfaces

– Generally, very little advantage in modelling behaviour of light at a surface in more detail -- it is quite difficult to understand behaviour of L+S surfaces


Radiometry vs. Photometry

http://www.optics.arizona.edu/Palmer/rpfaq/rpfaq.htm

Name Unit Symbol Name Unit Symbolradiant flux watt (W) luminous flux lumen (lm)

irradiance/radiosity W/m2 E/M illuminance lm/m2 = lux (lx) Evradiant intensity W/sr I luminous intensity lm/sr = candela (cd) Iv

radiance W/m2-sr L luminance lm/m2-sr = cd/m2 = nit Lv

RADIOMETRIC QUANTITIES PHOTOMETRIC QUANTITIES


Sensors

• CCD vs. CMOS

• Types of CCDs: linear, interline, full-frame, frame-transfer

• Bayer filters

• progressive scan vs. interlacing

• NTSC vs. PAL vs. SECAM

• framegrabbers

• blooming


Bayer color filter

http://en.wikipedia.org/wiki/Bayer_filter

Image formation

Documents

Transcript of Image formation