Lecture 12 Modules Employing Gradient Descent Computing Optical Flow Shape from Shading.
-
Upload
ella-barrett -
Category
Documents
-
view
216 -
download
0
Transcript of Lecture 12 Modules Employing Gradient Descent Computing Optical Flow Shape from Shading.
Lecture 1Lecture 122
Modules Employing Gradient Descent
Computing Optical Flow
Shape from Shading
2
Gibbs SamplerGibbs Sampler
ZP
TE
i
i
TEi i
ZP
1E/T is big
big is overflow
TE
ZP
1
11
TE
ZP
2
12
TE
ZP
3
13
1001 T
E
1202 T
E
1403 T
E
01 T
E
202 T
E
403 T
E
3
Gibbs SamplerGibbs Sampler
T
E
iPmax
Emax/T that will overflow
KTE
K
TE
11
1
KTE
K
TE
22
1
= BIGGEST DOUBLE
4
2 Modules that Employ Gradient Descent2 Modules that Employ Gradient Descent
1. Computing Optical Flow for Motion Using Gradient Based Approach
2. Shape from Shading
5
Optical FlowOptical Flow
Motion Field in Image Plane
udt
dx u
dt
dx
6
Optical FlowOptical Flow
2 Methods:
1. Featured Based - similar to stereo where you solve
- correspondence (matching) problem between 2 consecutive frames
2. Gradient of Intensity Based - No matching needed
- Works well when images have much texture
- Dense map of (u,v) at each pixel
7
Gradient of Intensity BasedGradient of Intensity Based
1 2 3
I(x,y,t1) I(x,y,t2) I(x,y,t3)
I(x,y,t)
16/sec
t
ALIASING
t 0t- Spatial Resolution (x,y) pixels per cm
- Temporal Resolution frames per second
8
AliasingAliasing
Problems noticeable when your sampling cannot truly estimate the underlying frequency
Have to sample double the frequency
9
Chain RuleChain Rule : I(x,y,t): I(x,y,t)
Assumption:“As an object moves, its intensity does not change”
t
I
dt
dy
y
I
dt
dx
x
I
dt
dI
),,(),,( 12 tyxItyxI
),,(),,( tyxIdttdyydxxI
0dt
dI
10
Specular RegionsSpecular Regions
Specular regions are noise for Computer Vision
2 2
11
Gradient of Intensity Gradient of Intensity BasedBased
t
I
dt
dy
y
I
dt
dx
x
I
dt
dI
0),(),(),(),(),( yxvyxIyxuyxIyxI yxt
Ix u Iy v It
12
Gradient of Intensity BasedGradient of Intensity Based
),(),1( 11 yxIyxIx
II ttx
t
IyxIyxII t
),(),( 12 0t
),()1,( 11 yxIyxIy
II tty
13
Gradient of Intensity BasedGradient of Intensity Based
2),()1,(12
),(),1(12
,
)( yxyxyxyxtyx
yx uuuuIvIuIE
2),()1,(12
),(),1(1 yxyxyxyx vvvv
Unknowns : u at each (x,y)v at each (x,y)
14
Gradient of Intensity BasedGradient of Intensity Based
Use Gradient Descent :
E(u,v)
u
E
dt
du
u
Euu tt
1
v
Ecv tt
1
Update Rule
Highly Textured
Knowns : Ix, Iy, It at (x,y)
15
Research TopicsResearch Topics
Find (u,v) through gradientMethod: Coarse-to-Fine
How to choose 1, 2 automatically
How to get the annealing schedule automaticallyT high Random Walk
T low Greedy
16
Shape from ShadingShape from Shading
Point Light at ∞
ping-pongviewer
viewer
Image Observed: f (viewer position, camera model, shape of object, material of object, light color, light model, light position)
17
Material of ObjectMaterial of Object
Color Shiny Transparency Texture Bumpy
18
Light ModelLight Model
Ambient – light (constant) at each point Spot
Omni – Neon – All Direction
Point Light - “Sun”
19
LightLight
R,G,B
I(x,y) = Ambient + Diffuse + Specular
= Iaka + kdIdcos + kss(cos)
Ia : Ambient LightId : Diffuse Light – Main Lightka : Ambient Constant “glow in dark”kd : Main Color Diffuse Constant
White is high , Black is lowks : Mirror Like, Specularity Constant
ks = 0 for ping pong = 0.5 for apple = 1 for billioud
20
Shininess FactorShininess Factor
= 20
= 1
Sharp Shiny Blurry Shiny
21
Shininess FactorShininess Factor
)(cosds Ik
: angle between V and R
: angle between L and N
cosq = L.N = |L||N|cos= cos
22
Shininess FactorShininess Factor
Diffuse = kd Id cos
cos decrease I
brighterdarker
0o 45o 85o
23
ShapeShape
Shape = Normal at a surface
(Nx, Ny, Nz) unit
24
NormalNormal
0 DCzByAx
0C
Dzy
C
Bx
C
A
C
Dy
C
Bx
C
Az
pC
A
x
z
)(
qC
B
y
z
)(
Equation of Plane
25
NormalNormal
Normal is different at every point
1),,(),,(
)1),,(),,((),(
yxqyxp
yxqyxpyxN
1),,(
222
CBA
CBAN
1
)1,,()1,,(
22
qp
qp
C
B
C
AN
26
Light DirectionLight Direction
L is the same at every point222 1
)1,,(
ba
baL
11
)1,,).(1,,().(
22222
qpba
qpbaIkNLIkI dd
dd
222 1),(),(
)1),(),((
yxqyxp
yxbqyxapkI
Contour of Constant Intensity
27
SFS: Data ConstraintSFS: Data Constraint
222 1
)1(
qp
bqapkI
kkbqkapqpI 222 1
01222 kkbqkapqpI Data Constraint
28
SFS: Energy FunctionSFS: Energy Function
Known : Ia, kd, (a,b,1), I(x,y) Unknown : p,q
2,
222 1 yx
kkbqkapqpIE
22 ),()1,(),(),1( yxpyxpyxpyxp ss
22 ),()1,(),(),1( yxqyxqyxqyxq ss
q
Eqq tt
1
p
Epp tt
1