Course 11 Optical Flow. 1. Concept ----- Observe the scene by moving viewer. ----- Optical flow...
12
Course 11 Optical Flow
-
Upload
eric-hicks -
Category
Documents
-
view
219 -
download
0
Transcript of Course 11 Optical Flow. 1. Concept ----- Observe the scene by moving viewer. ----- Optical flow...
Course 11 Optical Flow
1. Concept ----- Observe the scene by moving viewer.
----- Optical flow provides a clue to recover the motion.
2. Constraint equation of optical flow image observed from a scene at time t: at t+dt for the same scene:
If image intensity change is small in time interval , we have:
i.e.
),,( tyxf),,( dttdyydxxf
),,(),,( tyxfdttdyydxxf
0),,(
dt
tyxdf
dt
By chain rule of differentiation,
Denote optical flow
and image gradient
we have:
0
t
f
dt
dy
y
f
dt
dx
x
f
dt
dyv
dt
dxu ,
t
ff
y
ff
x
ff tyx
,,
tyx fvfuf
This equation is called constraint equation of optical flow. From a pair of images of a small time interval, fx, fy and ft can be calculated. However, u, v cannot be solved from only constraint equation of optical flow. This is called aperture problem.
3. Solve for optical flow
Considering continuity constraint of optical flow:0)()()( 222
y
u
x
uu
0)()()( 222
y
v
x
vv
Using variation method, we get
Let
i.e.
It yields that:
dxdyvufvfufyxe tyx ]})()[(){(),( 2222
min),( yxe
;0),(
u
yxe0
),(
v
yxe
txyxx ffuvffuf 222
tyyxy ffvuffvf 222
Solve the equation, get:
Since and need u and v and their neighbor
pixels to calculate, we can use iterative method to
solve them.
)/(][ 222yxtyxx fffvfuffuu
)/(][ 222yxtyxy fffvfuffvv
u v
222
)()()()1( ][
yx
tn
yn
xxnn
ff
fvfuffuu
222
)()()()1( ][
yx
tn
yn
xynn
ff
fvfuffvv
4. Understand optical flow
1) Focus of Expansion (FOE).
When one does translational motion along a fixed direction, the world seems to be flowing out of one particular retinal point. This point is called focus of expansion (FOE).
Remark: (1) FOE is a point in image plane. (2) FOE appears only for tranlational motion.
Considering a 3D point at (x0, y0, z0) initially and observer is moving at velocity . After a time interval t, the 3D position of the point will be at (x0+wxt, y0+wyt, z0+wzt) and the image point is:
),,( zyx www
),(),(0
0
0
0
twz
twy
twz
twxYX
z
y
z
x
When time , we get FOE:
2) Time–to–adjacency relation [Ballard, Computer Vision]
Let p be a 3D point at a translational moving object;
be the 3D speed of p in z-direction.
Let D be the distance from FOE to the image point
along straight flow;
be the optic flow speed of the image
point.
t
),(),(FOEdz
dy
dz
dx
w
w
w
w
z
y
z
x
dt
tdzwz
)(
dt
tdDtV
)()(
Then:
thus:
Recalling the formula of perspective projection,
we have
)(
)(
)(
)(
tw
tz
tV
tD
z
)(
)()(
tV
tDwtz z
)(
)()()(
tV
tDwtXtx z
)(
)()()(
tV
tDwtYty z
)(
)()(
tV
tDwtz z
This indicates that if we know the 3D speed of an
object in z-direction, the distance from FOE to the
corresponding image point and the flow speed in the
image plane, the 3D position of the object can be
computed.
3) Using Optical flow to segment image
of moving objects
----- Continuing property of optical flow.
----- FOE.
