3D Computer Vision and Video Computing Image Formation...
Transcript of 3D Computer Vision and Video Computing Image Formation...
![Page 1: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/1.jpg)
3D Computer Visionand Video Computing Image FormationImage Formation
Topic 3 of Part 1Image Formation
CSC I6716Spring 2004
http://www-cs.engr.ccny.cuny.edu/~zhu/VisionCourse-I6716.html
![Page 2: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/2.jpg)
3D Computer Visionand Video Computing AcknowledgementsAcknowledgements
The slides in this lecture were adopted from
Professor Allen HansonUniversity of Massachusetts at Amherst
![Page 3: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/3.jpg)
3D Computer Visionand Video Computing Lecture OutlineLecture Outline
Light and OpticsPinhole camera model Perspective projectionThin lens modelFundamental equationDistortion: spherical & chromatic aberration, radial distortion (*optional)Reflection and Illumination: color, lambertian and specular surfaces, Phong, BDRF (*optional)
Sensing LightConversion to Digital ImagesSampling TheoremOther Sensors: frequency, type, ….
![Page 4: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/4.jpg)
3D Computer Visionand Video Computing Abstract ImageAbstract Image
An image can be represented by an image function whose general form is f(x,y).f(x,y) is a vector-valued function whose arguments represent a pixel location.The value of f(x,y) can have different interpretations in different kinds of images.
Examples
Intensity Image - f(x,y) = intensity of the sceneRange Image - f(x,y) = depth of the scene from
imaging systemColor Image - f(x,y) = fr(x,y), fg(x,y), fb(x,y)Video - f(x,y,t) = temporal image sequence
![Page 5: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/5.jpg)
3D Computer Visionand Video Computing Basic RadiometryBasic Radiometry
Radiometry is the part of image formation concerned with the relation among the amounts of light energy emitted from light sources, reflected from surfaces, and registered by sensors.
Surface
Optics
CCD Array
P
Light Source
L(P,d)
in
p
e
![Page 6: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/6.jpg)
3D Computer Visionand Video Computing Light and MatterLight and Matter
The interaction between light and matter can take many forms:
ReflectionRefractionDiffractionAbsorptionScattering
![Page 7: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/7.jpg)
3D Computer Visionand Video Computing Lecture AssumptionsLecture Assumptions
Typical imaging scenario:visible lightideal lensesstandard sensor (e.g. TV camera)opaque objects
Goal
To create 'digital' images which can be processed to recover some of the characteristics of the 3D world which was imaged.
![Page 8: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/8.jpg)
3D Computer Visionand Video Computing StepsSteps
World Optics SensorSignal Digitizer
Digital Representation
World realityOptics focus light from world on sensorSensor converts light to electrical energySignal representation of incident light as continuous electrical energyDigitizer converts continuous signal to discrete signalDigital Rep. final representation of reality in computer memory
![Page 9: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/9.jpg)
3D Computer Visionand Video Computing Factors in Image FormationFactors in Image Formation
Geometryconcerned with the relationship between points in the three-dimensional world and their images
Radiometryconcerned with the relationship between the amount of light radiating from a surface and the amount incident at its image
Photometryconcerned with ways of measuring the intensity of light
Digitizationconcerned with ways of converting continuous signals (in both space and time) to digital approximations
![Page 10: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/10.jpg)
3D Computer Visionand Video Computing Image FormationImage Formation
Light (Energy) Source
Surface
Pinhole Lens
Imaging Plane
World Optics Sensor Signal
B&W Film
Color Film
TV Camera
Silver Density
Silver densityin three colorlayers
Electrical
![Page 11: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/11.jpg)
3D Computer Visionand Video Computing GeometryGeometry
Geometry describes the projection of:two-dimensional (2D) image plane.
three-dimensional (3D) world
Typical AssumptionsLight travels in a straight line
Optical Axis: the axis perpendicular to the image plane and passing through the pinhole (also called the central projection ray)Each point in the image corresponds to a particular direction defined by a ray from that point through the pinhole.Various kinds of projections:
- perspective - oblique- orthographic - isometric- spherical
![Page 12: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/12.jpg)
3D Computer Visionand Video Computing Basic OpticsBasic Optics
Two models are commonly used:Pin-hole cameraOptical system composed of lenses
Pin-hole is the basis for most graphics and visionDerived from physical construction of early camerasMathematics is very straightforward
Thin lens model is first of the lens modelsMathematical model for a physical lensLens gathers light over area and focuses on image plane.
![Page 13: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/13.jpg)
3D Computer Visionand Video Computing Pinhole Camera ModelPinhole Camera Model
World projected to 2D ImageImage invertedSize reducedImage is dimNo direct depth information
f called the focal length of the lensKnown as perspective projection
Pinhole lens
Optical Axis
f
Image Plane
![Page 14: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/14.jpg)
3D Computer Visionand Video Computing Pinhole camera imagePinhole camera image
Photo by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html
Amsterdam
![Page 15: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/15.jpg)
3D Computer Visionand Video Computing Equivalent GeometryEquivalent Geometry
Consider case with object on the optical axis:
fz
More convenient with upright image:
- fz
Projection plane z = 0
Equivalent mathematically
![Page 16: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/16.jpg)
3D Computer Visionand Video Computing
f
IMAGEPLANE
OPTICAXIS
LENS
i o
1 1 1f i o
= + ‘THIN LENS LAW’
Thin Lens ModelThin Lens Model
Rays entering parallel on one side converge at focal point.Rays diverging from the focal point become parallel.
![Page 17: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/17.jpg)
3D Computer Visionand Video Computing Coordinate SystemCoordinate System
Simplified Case:Origin of world and image coordinate systems coincideY-axis aligned with y-axisX-axis aligned with x-axisZ-axis along the central projection ray
WorldCoordinateSystem
Image Coordinate System
Z
X
Y
Y
ZX
(0,0,0)
y
x
P(X,Y,Z)p(x,y)
(0,0)
![Page 18: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/18.jpg)
3D Computer Visionand Video Computing Perspective ProjectionPerspective Projection
Compute the image coordinates of p in terms of the world coordinates of P.
Look at projections in x-z and y-z planes
x
y
Z
P(X,Y,Z)p(x, y)
Z = 0
Z=-f
![Page 19: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/19.jpg)
3D Computer Visionand Video Computing X-Z ProjectionX-Z Projection
By similar triangles:
Z- f
X
x
=xf
XZ+f
=xfX
Z+f
![Page 20: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/20.jpg)
3D Computer Visionand Video Computing Y-Z ProjectionY-Z Projection
By similar triangles: =yf
YZ+f
=yfY
Z+f
- f
Z
Yy
![Page 21: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/21.jpg)
3D Computer Visionand Video Computing Perspective EquationsPerspective Equations
Given point P(X,Y,Z) in the 3D worldThe two equations:
transform world coordinates (X,Y,Z) into image coordinates (x,y)
Question:What is the equation if we select the origin of both coordinate systems at the nodal point?
=yfY
Z+f=x
fXZ+f
![Page 22: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/22.jpg)
3D Computer Visionand Video Computing Reverse ProjectionReverse Projection
Given a center of projection and image coordinates of a point, it is not possible to recover the 3D depth of the point from a single image.
In general, at least two images of the same point taken from two different locations are required to recover depth.
All points on this linehave image coordi-nates (x,y).
p(x,y)
P(X,Y,Z) can be any-where along this line
![Page 23: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/23.jpg)
3D Computer Visionand Video Computing Stereo GeometryStereo Geometry
Depth obtained by triangulationCorrespondence problem: pl and pr must correspond to the left and right projections of P, respectively.
Object point
CentralProjection
Rays
Vergence Angle
pl pr
P(X,Y,Z)
![Page 24: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/24.jpg)
3D Computer Visionand Video Computing RadiometryRadiometry
Image: two-dimensional array of 'brightness' values.Geometry: where in an image a point will project.Radiometry: what the brightness of the point will be.Brightness: informal notion used to describe both scene and image brightness.Image brightness: related to energy flux incident on the image plane: =>
IRRADIANCEScene brightness: brightness related to energy flux emitted (radiated) from a surface: =>
RADIANCE
![Page 25: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/25.jpg)
3D Computer Visionand Video Computing LightLight
Electromagnetic energyWave modelLight sources typically radiate over a frequency spectrumΦ watts radiated into 4π radians
r
Φ watts
dω
R = Radiant Intensity = dωdΦ Watts/unit solid
angle (steradian)(of source)
Φ = ∫ dΦsphere
![Page 26: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/26.jpg)
3D Computer Visionand Video Computing IrradianceIrradiance
Light falling on a surface from all directions.How much?
dA
Irradiance: power per unit area falling on a surface.
dΦ
Irradiance E = dΑdΦ
watts/m2
![Page 27: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/27.jpg)
3D Computer Visionand Video Computing Inverse Square LawInverse Square Law
Relationship between radiance (radiant intensity) and irradiance
r
Φ watts
dωdA
dAdω =
r2
E =dΑ
dΦ
R = dωdΦ
= =r2 dΦ
dΑr2 E
E = R r2
![Page 28: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/28.jpg)
3D Computer Visionand Video Computing Surface RadianceSurface Radiance
Surface acts as light sourceRadiates over a hempisphere
Radiance: power per unit forshortened area emitted into a solid angle
L = d ΦdA dωf
2
SurfaceNormal
dω
dAf
(watts/m2 - steradian)
![Page 29: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/29.jpg)
3D Computer Visionand Video Computing GeometryGeometry
Goal: Relate the radiance of a surface to the irradiance in the image plane of a simple optical system.
dAi
dAs Lens Diameter d
α
e α
i
F
![Page 30: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/30.jpg)
3D Computer Visionand Video Computing Light at the SurfaceLight at the Surface
E = flux incident on the surface (irradiance) = dΑdΦ
Φ wattsi = incident anglee = emittance angleg = phase angler = surface reflectance
eEmitted ray
N (surface normal)i
r
gd sA
Incident Ray
dw
r
We need to determine dΦ and dA
![Page 31: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/31.jpg)
3D Computer Visionand Video Computing Reflections from a Surface IReflections from a Surface I
dA = dAscos i foreshortening effect in direction of light source
dAdA
dΦ = flux intercepted by surface over area dAdA subtends solid angle dω = dAs cos i / r2
dΦ = R dω = R dAs cos i / r2
E = dΦ / dAs
dΦdΦ
Surface Irradiance: E = R cos i / r2
![Page 32: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/32.jpg)
3D Computer Visionand Video Computing Reflections from a Surface IIReflections from a Surface II
Now treat small surface area as an emitter….because it is bouncing light into the world
How much light gets reflected?
N (surface normal)igedAs
Emitted Ray
Incident Ray
E is the surface irradianceL is the surface radiance = luminanceThey are related through the surface reflectance function:
= ρ(i,e,g,λ)LsE
May also be a function of thewavelength of the light
![Page 33: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/33.jpg)
3D Computer Visionand Video Computing Power Concentrated in LensPower Concentrated in Lens
What is the power of the surface patch as a source in the direction of the lens?
dAi
Image Plane
Z -f
dAs Lens Diameter d
αθ
α
d Φ2
dA dωsL =s Luminance of patch (known from previous step)
d Φ2 dA dωsL s=
![Page 34: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/34.jpg)
3D Computer Visionand Video Computing Through a Lens DarklyThrough a Lens Darkly
In general:L is a function of the angles i and e.Lens can be quite largeHence, must integrate over the lens solid angle to get dΦ
dAs Lens Diameter d
αe Ω
s
dΦ = dA dΩL s•
•Ω
s
![Page 35: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/35.jpg)
3D Computer Visionand Video Computing Simplifying AssumptionSimplifying Assumption
Lens diameter is small relative to distance from patch
L is a constant and can be removed from the integral
dΦ = dA dΩL s•
•Ω
Surface area of patch in direction of lens
= dA cos e
Solid angle subtended by lens in direction of patch
=Area of lens as seen from patch
(Distance from lens to patch) 2
=π (d/2) cos α
2
(z / cos α)2
s
dΦ = dA dΩL s•
•Ω
ss
s
![Page 36: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/36.jpg)
3D Computer Visionand Video Computing Putting it TogetherPutting it Together
Power concentrated in lens:
= dA cos e L s sπ (d/2) cos α2
(z / cos α)2
Assuming a lossless lens, this is also the power radiated by the lens as a source.
dΦ = dA dΩL s•
•Ω
s
dΦ = L dA cos e cos αsdZ
2π 4 s 3
![Page 37: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/37.jpg)
3D Computer Visionand Video Computing Through a Lens DarklyThrough a Lens Darkly
Image irradiance at dA =
dAi
Image Plane
Z -f
dAs Lens Diameter d
αθ
α
idΦdA i
=Ei
E =i cos e cos αdZ
2π 4sL dAs
dAi3
ratio of areas
![Page 38: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/38.jpg)
3D Computer Visionand Video Computing Patch ratioPatch ratio
dAi
Image Plane
Z -f
dAs Lens Diameter d
αθ
α
The two solid angles are equal
dA cos es(Z / cos α) 2
dA cos αi(-f / cos α) 2= cos α
cos eZ-f
2=
dAsdAi
![Page 39: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/39.jpg)
3D Computer Visionand Video Computing The Fundamental ResultThe Fundamental Result
Source Radiance to Image Sensor Irradiance:
cos αcos e
Z-f
2=
dAs
dAi
E =i cos e cos αdZ
2π 4sL
dAsdA i
3
E =i sL cos e cos αdZ
2π 4
3cos αcos e
Z-f
2
E =i sL cos αd-f
2π 4
4
![Page 40: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/40.jpg)
3D Computer Visionand Video Computing Radiometry Final ResultRadiometry Final Result
Image irradiance is proportional to:Scene radiance LFocal length of lens fDiameter of lens d
f/d is often called the f-number of the lens
Off-axis angle α
E =i sL cos αd-f
2π 4
4
s
![Page 41: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/41.jpg)
3D Computer Visionand Video Computing Cos α Light FalloffCos α Light Falloff44
x
y
π/2−π/2
−π/2
Lens Center Top view shaded by height
![Page 42: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/42.jpg)
3D Computer Visionand Video Computing Limitation of Radiometry ModelLimitation of Radiometry Model
Surface reflection ρ can be a function of viewing and/or illumination angle
Surface normal N
Arbitrary reference axis
eig
ϕϕi
e
ρ(i,e,g,ϕ , ϕ ) = e idL(e, ϕ )e
dE(i, ϕ )i
ρ may also be a function of the wavelength of the light sourceAssumed a point source (sky, for example, is not)
![Page 43: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/43.jpg)
3D Computer Visionand Video Computing Lambertian SurfacesLambertian Surfaces
The BRDF for a Lambertian surface is a constant
ρ(i,e,g,ϕ , ϕ ) = kfunction of cos e due to the forshortening effectk is the 'albedo' of the surfaceGood model for diffuse surfaces
Other models combine diffuse and specular components (Phong, Torrance-Sparrow, Oren-Nayar)References available upon request
ie
![Page 44: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/44.jpg)
3D Computer Visionand Video Computing PhotometryPhotometry
Photometry:Concerned with mechanisms for converting light energy
into electrical energy.
World Optics SensorSignal Digitizer
Digital Representation
![Page 45: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/45.jpg)
3D Computer Visionand Video Computing B&W Video SystemB&W Video System
.
OpticsImage Plane
A/D Converterand Sampler
E(x,y) : Electricalvideo signal
Image L(x,y)
VideoCamera
I(i,j) Digital Image
22 34 22 0 18 ¥¥¥
¥¥¥¥¥¥
Grayscale Image Data
Computer Memory
![Page 46: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/46.jpg)
3D Computer Visionand Video Computing Color Video SystemColor Video System
.
Blue ChannelA/D ConverterGreen Channel
A/D Converter
OpticsImage Plane
Digital Image
E(x,y) : Electricalvideo signal
Image L(x,y)
Computer Memory
22 3422 0 18 ¥¥¥
¥¥¥¥¥¥
Red ChannelA/D Converter
VideoCamera
B(i,j)G(i,j)
R(i,j)
![Page 47: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/47.jpg)
3D Computer Visionand Video Computing Color RepresentationColor Representation
Color Cube and Color Wheel
For color spaces, please readColor Cube http://www.morecrayons.com/palettes/webSmart/Color Wheel http://home.att.net/~rocq/SIHwheel.htmlhttp://www.netnam.vn/unescocourse/computervision/12.htmhttp://www-viz.tamu.edu/faculty/parke/ends489f00/notes/sec1_4.html
R
B
GS
I
H
![Page 48: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/48.jpg)
3D Computer Visionand Video Computing Digital Color CamerasDigital Color Cameras
Three CCD-chips camerasR, G, B separately, AND digital signals instead analog video
One CCD CamerasBayer color filter array
http://www.siliconimaging.com/RGB%20Bayer.htm
http://www.fillfactory.com/htm/technology/htm/rgbfaq.htm
Image Format with Matlab (show demo)
![Page 49: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/49.jpg)
3D Computer Visionand Video Computing
Human Eye CCD Camera
Spectral SensitivitySpectral Sensitivity
Figure 1 shows relative efficiency of conversion for the eye (scotopic and photopiccurves) and several types of CCD cameras. Note the CCD cameras are much more sensitive than the eye.Note the enhanced sensitivity of the CCD in the Infrared and Ultraviolet (bottom two figures)Both figures also show a hand-drawn sketch of the spectrum of a tungsten light bulb
Tungsten bulb
![Page 50: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/50.jpg)
3D Computer Visionand Video Computing
Human Eyes and Color PerceptionHuman Eyes and Color Perception
Visit a cool site with Interactive Java tutorial:http://micro.magnet.fsu.edu/primer/lightandcolor/vision.html
Another site about human color perception:http://www.photo.net/photo/edscott/vis00010.htm
![Page 51: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/51.jpg)
3D Computer Visionand Video Computing CharacteristicsCharacteristics
In general, V(x,y) = k E(x,y) wherek is a constantγ is a parameter of the type of sensor
γ=1 (approximately) for a CCD cameraγ=.65 for an old type vidicon camera
Factors influencing performance:Optical distortion: pincushion, barrel, non-linearitiesSensor dynamic range (30:1 CCD, 200:1 vidicon)Sensor Shading (nonuniform responses from different locations)
TV Camera pros: cheap, portable, small sizeTV Camera cons: poor signal to noise, limited dynamic range, fixed array size with small image (getting better)
γ
![Page 52: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/52.jpg)
3D Computer Visionand Video Computing Sensor PerformanceSensor Performance
Optical Distortion: pincushion, barrel, non-linearitiesSensor Dynamic Range: (30:1 for a CCD, 200:1 Vidicon)Sensor Blooming: spot size proportional to input intensitySensor Shading: (non-uniform response at outer edges of image)Dead CCD cells
There is no “universal sensor”. Sensors must be selected/tuned for a particular domain and application.
![Page 53: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/53.jpg)
3D Computer Visionand Video Computing Lens AberrationsLens Aberrations
In an ideal optical system, all rays of light from a point in the object plane would converge to the same point in the image plane, forming a clear image.The lens defects which cause different rays to converge to different points are called aberrations.
Distortion: barrel, pincushionCurvature of fieldChromatic Aberration Spherical aberrationComaAstigmatism
Aberration slides after http://hyperphysics.phy-astr.gsu.edu/hbase/geoopt/aberrcon.html#c1
![Page 54: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/54.jpg)
3D Computer Visionand Video Computing Lens AberrationsLens Aberrations
Distortion
Curved Field
![Page 55: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/55.jpg)
3D Computer Visionand Video Computing Lens AberrationsLens Aberrations
Chromatic Aberration
Focal Length of lens depends on refraction and The index of refraction for blue light (short wavelengths) is larger than that of red light (long wavelengths).Therefore, a lens will not focus different colors in exactly the same place The amount of chromatic aberration depends on the dispersion (change of index of refraction with wavelength) of the glass.
![Page 56: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/56.jpg)
3D Computer Visionand Video Computing Lens AberrationLens Aberration
Spherical Aberration
Rays which are parallel to the optic axis but at different distances from the optic axis fail to converge to the same point.
![Page 57: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/57.jpg)
3D Computer Visionand Video Computing Lens AberrationsLens Aberrations
Coma
Rays from an off-axis point of light in the object plane create a trailing "comet-like" blur directed away from the optic axisBecomes worse the further away from the central axis the point is
![Page 58: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/58.jpg)
3D Computer Visionand Video Computing Lens AberrationsLens Aberrations
Astigmatism
Results from different lens curvatures in different planes.
![Page 59: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/59.jpg)
3D Computer Visionand Video Computing Sensor SummarySensor SummaryVisible Light/Heat
Camera/Film combinationDigital CameraVideo CamerasFLIR (Forward Looking Infrared)
Range SensorsRadar (active sensing)
sonarlaser
Triangulationstereostructured light
– striped, patternedMoireHolographic InterferometryLens FocusFresnel Diffraction
OthersAlmost anything which produces a 2d signal that is related to the scene can be used as a sensor
![Page 60: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/60.jpg)
3D Computer Visionand Video Computing DigitizationDigitization
Digitization: conversion of the continuous (in space and value) electrical signal into a digital signal (digital image)Three decisions must be made:
Spatial resolution (how many samples to take)Signal resolution (dynamic range of values- quantization)Tessellation pattern (how to 'cover' the image with sample points)
World Optics SensorSignal Digitizer
Digital Representation
![Page 61: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/61.jpg)
3D Computer Visionand Video Computing Digitization: Spatial ResolutionDigitization: Spatial Resolution
Let's digitize this imageAssume a square sampling patternVary density of sampling grid
![Page 62: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/62.jpg)
3D Computer Visionand Video Computing Spatial ResolutionSpatial Resolution
Coarse Sampling: 20 points per row by 14 rows
Finer Sampling: 100 points per row by 68 rows
••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
Sam
plin
g in
terv
alSample picture at each red point
![Page 63: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/63.jpg)
3D Computer Visionand Video Computing Effect of Sampling Interval - 1Effect of Sampling Interval - 1
Look in vicinity of the picket fence:
Sampling Interval:
100 100 100100 100 100
100 100 100100 100 100
100 100 100100 100 100
100 100 100100 100 100
40 40 40 40 40 4040 40 40 40 40 4040 40 40 40 40 4040 40 40 40 40 40
White Image! Dark Gray Image!
NO EVIDENCEOF THE FENCE!
![Page 64: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/64.jpg)
3D Computer Visionand Video Computing Effect of Sampling Interval - 2Effect of Sampling Interval - 2
Look in vicinity of picket fence:
Sampling Interval:
40 100 10040 40
40 100 10040 40
40 100 10040 40
40 100 10040 40
Now we've got a fence!
What's the difference between this attempt and the last one?
![Page 65: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/65.jpg)
3D Computer Visionand Video Computing The Missing Fence FoundThe Missing Fence Found
Consider the repetitive structure of the fence:
d
s'
s
Sampling Intervals
Case 1: s' = d The sampling interval is equal to the size of the repetitive structure
NO FENCE
Case 2: s = d/2 The sampling interval is one-half the size of the repetitive structure
FENCE
![Page 66: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/66.jpg)
3D Computer Visionand Video Computing The Sampling TheoremThe Sampling Theorem
IF: the size of the smallest structure to be preserved is dTHEN: the sampling interval must be smaller than d/2
Can be shown to be true mathematicallyRepetitive structure has a certain frequency ('pickets/foot')
To preserve structure must sample at twice the frequencyHolds for images, audio CDs, digital television….
Leads naturally to Fourier Analysis (later in course)
![Page 67: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/67.jpg)
3D Computer Visionand Video Computing SamplingSampling
Rough Idea: Ideal Case
Dirac Delta Function 2D "Comb"
"Continuous Image"
"Digitized Image"
δ(x,y) = 0 for x = 0, y= 0
δ(x,y) dx dy = 1
f(x,y)δ(x-a,y-b) dx dy = f(a,b)
δ(x-ns,y-ns) for n = 1….32 (e.g.)
23
s
![Page 68: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/68.jpg)
3D Computer Visionand Video Computing SamplingSampling
Rough Idea: Actual CaseCan't realize an ideal point function in real equipment"Delta function" equivalent has an areaValue returned is the average over this area
23
s
![Page 69: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/69.jpg)
3D Computer Visionand Video Computing Mixed Pixel ProblemMixed Pixel Problem
![Page 70: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/70.jpg)
3D Computer Visionand Video Computing Signal QuantizationSignal Quantization
Goal: determine a mapping from a continuous signal (e.g. analog video signal) to one of K discrete (digital) levels.
I(x,y) = .1583 volts
= ???? Digitalvalue
![Page 71: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/71.jpg)
3D Computer Visionand Video Computing QuantizationQuantization
I(x,y) = continuous signal: 0 ≤ I ≤ MWant to quantize to K values 0,1,....K-1K usually chosen to be a power of 2:
Mapping from input signal to output signal is to be determined.Several types of mappings: uniform, logarithmic, etc.
K #Levels #Bits2 2 14 4 28 8 316 16 432 32 564 64 6128 128 7256 256 8
![Page 72: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/72.jpg)
3D Computer Visionand Video Computing Choice of KChoice of K
Original
Linear Ramp
K=2 K=4
K=16 K=32
![Page 73: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/73.jpg)
3D Computer Visionand Video Computing Choice of KChoice of K
K=2 (each color)
K=4 (each color)
![Page 74: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/74.jpg)
3D Computer Visionand Video Computing Choice of Function: UniformChoice of Function: Uniform
Uniform sampling divides the signal range [0-M] into K equal-sized intervals.The integers 0,...K-1 are assigned to these intervals.All signal values within an interval are represented by the associated integer value.Defines a mapping:
3
M
210
K-1
0Signal Value
¥¥¥
![Page 75: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/75.jpg)
3D Computer Visionand Video Computing Logarithmic QuantizationLogarithmic Quantization
Signal is log I(x,y).Effect is:
3
M
210
K-1
0Signal Value
¥¥¥
Detail enhanced in the low signal values at expense of detail in high signal values.
![Page 76: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/76.jpg)
3D Computer Visionand Video Computing Logarithmic QuantizationLogarithmic Quantization
Original
Logarithmic Quantization
Quantization Curve
![Page 77: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/77.jpg)
3D Computer Visionand Video Computing Tesselation PatternsTesselation Patterns
32
Hexagonal Triangular
Rectangular Typical
![Page 78: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/78.jpg)
3D Computer Visionand Video Computing Digital GeometryDigital Geometry
NeighborhoodConnectednessDistance Metrics
Picture Element or Pixel
Pixel value I(I,j) =0,1 Binary Image0 - K-1 Gray Scale ImageVector: Multispectral Image
32
I(i,j) (0,0)
i
j
![Page 79: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/79.jpg)
3D Computer Visionand Video Computing Connected ComponentsConnected Components
Binary image with multiple 'objects'Separate 'objects' must be labeled individually
6 Connected Components
![Page 80: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/80.jpg)
3D Computer Visionand Video Computing Finding Connected ComponentsFinding Connected Components
Two points in an image are 'connected' if a path can be found for which the value of the image function is the same all along the path.
P1
P2
P3
P4
P1 connected to P2
P3 connected to P4
P1 not connected to P3 or P4
P2 not connected to P3 or P4
P3 not connected to P1 or P2
P4 not connected to P1 or P2
![Page 81: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/81.jpg)
3D Computer Visionand Video Computing AlgorithmAlgorithm
Pick any pixel in the image and assign it a labelAssign same label to any neighbor pixel with the same value of the image functionContinue labeling neighbors until no neighbors can be assigned this labelChoose another label and another pixel not already labeled and continueIf no more unlabeled image points, stop.
Who's my neighbor?
![Page 82: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/82.jpg)
3D Computer Visionand Video Computing ExampleExample
Lab. Im. - 4th Component Final Labeling
Image 'Label' Image
Lab. Im. - 1st Component Lab. Im. - 2nd Component Lab. Im. - 3rd Component
![Page 83: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/83.jpg)
3D Computer Visionand Video Computing NeighborNeighbor
Consider the definition of the term 'neighbor'Two common definitions:
Consider what happens with a closed curve.One would expect a closed curve to partition the plane into two connected regions.
Four Neighbor Eight Neighbor
![Page 84: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/84.jpg)
3D Computer Visionand Video ComputingAlternate Neighborhood DefinitionsAlternate Neighborhood Definitions
Neither neighborhood definition satisfactory!
4-neighborconnectedness
8-neighborconnectedness
![Page 85: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/85.jpg)
3D Computer Visionand Video Computing Possible SolutionsPossible Solutions
Use 4-neighborhood for object and 8-neighborhood for background
requires a-priori knowledge about which pixels are object and which are background
Use a six-connected neighborhood:
![Page 86: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/86.jpg)
3D Computer Visionand Video Computing Digital DistancesDigital Distances
Alternate distance metrics for digital images
i
m
n
j
i
m
n
j
i
m
n
j
Euclidean Distance City Block Distance Chessboard Distance
= (i-n) 2 + (j-m) 2 = |i-n| + |j-m| = max[ |i-n|, |j-m| ]
![Page 87: 3D Computer Vision and Video Computing Image Formation ...ermak.cs.nstu.ru/neurotech/html/teaching/2007/MV/mv_2_app.pdf · 3D Computer Vision and Video Computing Abstract ImageAbstract](https://reader034.fdocuments.net/reader034/viewer/2022042108/5e8856b295990d52e71f912c/html5/thumbnails/87.jpg)
3D Computer Visionand Video Computing NextNext
Next:Features