CS4501: Introduction to Computer Vision Projective ...
Transcript of CS4501: Introduction to Computer Vision Projective ...
![Page 1: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/1.jpg)
CS4501: Introduction to Computer VisionProjective Geometry and Light
Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA).
![Page 2: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/2.jpg)
Announcing Office Hours
Masum(mb2vj at virginia.edu)
Office Hours: Tuesdays 3 to 4pm (ET)
Anshuman(as9rw at virginia.edu)
Office Hours: Wednesdays 2 to 3pm (ET)
Fridays 1 to 2pm (ET)
Vicente(vicente at virginia.edu)
Office Hours: Mondays 3 to 5pm (ET)
![Page 3: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/3.jpg)
Last Class
What is a camera? Who invented cameras?Photography – Life of a Photograph - from Light to PixelsShutter Speed / Aperture Size / ISO sensitivity
![Page 4: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/4.jpg)
• Recap from Last Class on Shutter Speed / Aperture / ISO sensitivity• Camera Parameters• Brief Introduction to Projective Geometry (Computer Graphics)• Light Models (Diffuse Light vs Specular Light)
Today’s Class
![Page 5: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/5.jpg)
Life of a Photograph
Slide by Steve Seitz
![Page 6: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/6.jpg)
How to Shoot Photos in Manual?
•Shutter speed•Aperture size•Focus / Auto-focus (Yes, you can shoot in
manual and also probably should focus in manual)
•ISO sensitivity
![Page 7: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/7.jpg)
Fast Shutter Speed
http://www.photographymad.com/pages/view/shutter-speed-a-beginners-guide
Doesn’t allow much light Allows a lot of light
Slow Shutter Speed
Large Aperture SizeSmall Aperture Size
![Page 8: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/8.jpg)
ISO sensitivity – Should be small ideally
https://www.exposureguide.com/iso-sensitivity/
![Page 9: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/9.jpg)
Projection: world coordinatesàimage coordinates
Camera Center (0, 0, 0)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z
Y
úû
ùêë
é=VU
p
.V
U
If X = 2, Y = 3, Z = 5, and f = 2What are U and V?
![Page 10: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/10.jpg)
Projection: world coordinatesàimage coordinates
Camera Center (0, 0, 0)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z
Y
úû
ùêë
é=VU
p
.V
U
ZfXU *-=
ZfYV *-=
52*2-=U
52*3-=V
![Page 11: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/11.jpg)
Projection: world coordinatesàimage coordinates
Camera Center
(tx, ty, tz)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z Y
úû
ùêë
é=vu
p
.Optical Center (u0, v0)
v
u
![Page 12: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/12.jpg)
Homogeneous coordinates vs Cartesian coordinatesConversion
Converting to homogeneous coordinates
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
![Page 13: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/13.jpg)
Invariant to scaling
Point in Cartesian is ray in Homogeneous
úû
ùêë
é=ú
û
ùêë
éÞ
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
wywx
kwkykwkx
kwkykx
wyx
k
Homogeneous Coordinates
Cartesian Coordinates
Homogeneous coordinates vs Cartesian coordinates
![Page 14: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/14.jpg)
Projection: world coordinatesàimage coordinates
Camera Center (0, 0, 0)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z
Y
úû
ùêë
é=VU
p
.V
U
ZfXU *-=
ZfYV *-=
52*2-=U
52*3-=V
![Page 15: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/15.jpg)
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
10100000000
1zyx
ff
vu
w
K
Slide Credit: Savarese
Projection matrix: from World to Image Coordinates
Intrinsic Assumptions• Unit aspect ratio• Optical center at (0,0)• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
X
x
![Page 16: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/16.jpg)
Remove assumption: known optical center
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
101000000
10
0
zyx
vfuf
vu
w
Intrinsic Assumptions• Unit aspect ratio• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
![Page 17: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/17.jpg)
Remove assumption: square pixels
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
101000000
10
0
zyx
vu
vu
w ba
Intrinsic Assumptions• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
![Page 18: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/18.jpg)
Remove assumption: non-skewed pixels
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
10100000
10
0
zyx
vus
vu
w ba
Intrinsic Assumptions Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Note: different books use different notation for parameters
![Page 19: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/19.jpg)
Oriented and Translated Camera
Ow
iw
kw
jw
t
R
X
x
![Page 20: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/20.jpg)
Allow camera translation
[ ]XtIKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
1100010001
1000
0
10
0
zyx
ttt
vu
vu
w
z
y
x
ba
Intrinsic Assumptions Extrinsic Assumptions• No rotation
![Page 21: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/21.jpg)
3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:
úúú
û
ù
êêê
ë
é -=
úúú
û
ù
êêê
ë
é
-=
úúú
û
ù
êêê
ë
é-=
1000cossin0sincos
)(
cos0sin010
sin0cos)(
cossin0sincos0001
)(
gggg
g
bb
bbb
aaaaa
z
y
x
R
R
R
p
p’ g
y
z
Slide Credit: Savarese
![Page 22: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/22.jpg)
Allow camera rotation
[ ]XtRKx =
úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
1100
01 333231
232221
131211
0
0
zyx
trrrtrrrtrrr
vus
vu
w
z
y
x
ba
![Page 23: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/23.jpg)
Slide Credit: Savarese
Projection matrix (Word Coordinates to Image Coordinates)
[ ]XtRKx = x: Image Coordinates: (u,v,1)K: Intrinsic Matrix (3x3)R: Rotation (3x3) t: Translation (3x1)X: World Coordinates: (X,Y,Z,1)
Ow
iw
kw
jwR,t
X
x
Intrinsic Camera Properties: K
Extrinsic Camera Properties: [R t]
![Page 24: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/24.jpg)
Degrees of freedom
[ ]XtRKx =
úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
1100
01 333231
232221
131211
0
0
zyx
trrrtrrrtrrr
vus
vu
w
z
y
x
ba
5 6
![Page 25: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/25.jpg)
Assignment – World Coordinates to Image Coordinates
What the camera can “see”assuming this is a wireframe
object
(0,0,0)
(0,0,3)
(1,1,4)
![Page 26: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/26.jpg)
Things to Remember for Quiz
• Pinhole camera model• Focal length in the pinhole camera model• Shutter Time / Aperture / ISO• Homogeneous Coordinates• Extrinsic Camera Properties and Intrinsic Camera Properties• Describe mathematically (and intuitively) the conversion process
from World Coordinates to Image Coordinates
![Page 27: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/27.jpg)
Light
• What determines the color of a pixel?
Figure from Szeliski
![Page 28: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/28.jpg)
BRDF (Bidirectional reflectance distribution function)
Slide by Aaron Bobick
![Page 29: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/29.jpg)
BRDF (Bidirectional reflectance distribution function)
Slide by Aaron Bobick
![Page 30: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/30.jpg)
Reflection
Slide by Aaron Bobick
![Page 31: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/31.jpg)
Phong Reflection Model
Slide by Aaron Bobick
![Page 32: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/32.jpg)
Phong Reflection Model
https://en.wikipedia.org/wiki/Phong_reflection_model
![Page 33: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/33.jpg)
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
![Page 34: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/34.jpg)
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
![Page 35: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/35.jpg)
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
![Page 36: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/36.jpg)
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
![Page 37: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/37.jpg)
Phong’s Shading / Illumination Model
• Originally from Vietnam / PhD from Utah, Professor at Utah, and later Stanford.
• Died at age 32 from leukemia
• Phong’s professor Ivan Sutherland went on to win the Turing Award (Nobel Prize in CS) for lifelong contributions to Computer Graphics
![Page 38: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/38.jpg)
Same ideas used in Computer Graphics
• Ray Tracing• Radiosity• Photon Mapping
![Page 39: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/39.jpg)
Reflection
Slide by Aaron Bobick
![Page 40: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/40.jpg)
Diffuse Reflection – Lambertian Surface / BRDF
Slide by Aaron Bobick
• Light intensity does not depend on the outgoing direction. Only incoming.
• It is independent of where the viewer stands.
• Smooth surface, not glossy. Can think of any examples?
![Page 41: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/41.jpg)
Slide by Aaron Bobick
![Page 42: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/42.jpg)
The other extreme – Only Specular Reflection
Slide by Aaron Bobick
![Page 43: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/43.jpg)
Problem in Computer Vision: Intrinsic Image Decomposition
Given this Extract this
Images by Marc Serra
![Page 44: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/44.jpg)
Given this Extract this
Images by Aaron Bobick
Problem in Computer Vision: Shape from Shading
![Page 45: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/45.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
?
Slide by James Hays
![Page 46: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/46.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
![Page 47: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/47.jpg)
A photon’s life choices
• Absorption• Diffuse Reflection• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
![Page 48: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/48.jpg)
A photon’s life choices
• Absorption• Diffusion• Specular Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
![Page 49: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/49.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
![Page 50: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/50.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
![Page 51: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/51.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ1
light source
λ2
Slide by James Hays
![Page 52: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/52.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
![Page 53: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/53.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
t=1
light source
t=n
Slide by James Hays
![Page 54: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/54.jpg)
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
(Specular Interreflection)
Slide by James Hays
![Page 55: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/55.jpg)
The Eye
• The human eye is a camera! (not quite)• Iris - colored annulus with radial muscles• Pupil - the hole (aperture) whose size is controlled by the iris• What’s the “film”?
– photoreceptor cells (rods and cones) in the retina
Slide by Steve Seitz
![Page 56: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/56.jpg)
Next Class: Image Processing and Image Filters
![Page 57: CS4501: Introduction to Computer Vision Projective ...](https://reader030.fdocuments.net/reader030/viewer/2022012518/61926a99b4457d516445039b/html5/thumbnails/57.jpg)
Questions?
57