Optical Flow
CS 510 Lecture #21
April 11h, 2014
Where are we?
• We know how to implement a BoW • BoW was invented to match still images • We can update the feature extraction step
to find STIPs (spatio-temporal IPs) – 3D corners (Laptev) – Gabor peaks (Dollár)
• But if we can’t describe motion…
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 2
Describing Motion : Flow
• Motion is best described as the 2D motion of surface points over time – Find an easy-to-recognize point on object – Record its (x1,y1) position at time T1
– Record its (x2,y2) position at time T2
– Its flow vector (dx/dt, dy/dt) is (x2-x1, y2-y1) • Of course, the devil is in the details
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 3
Example of Sparse Flow
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 4
hDp://docs.opencv.org/trunk/doc/py_tutorials/py_video/py_lucas_kanade/py_lucas_kanade.html
Computing Sparse Flow (simple) • Pick good points to track
– Point matching must be spatially precise • Frame to frame movements are small • Corners (structure tensors) were invented for this
purpose – Patterns must be unique
• Repeated patterns must be avoided • Checkerboards, brick walls, etc. should be avoided
– OpenCV has a goodFeaturesToTrack function • Use image around point as template
– Find position of best match in next image – Compute flow from position of match
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 5
Sparse Flow Issues
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 6
Example of Dense Flow • Scene from PETS
video • Farneback 2003 • From OpenCV
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 7
Focus of Expansion (FoE)
• If the camera is moving into the scene, all flow field vectors point away from the point the camera is moving toward
• This point is called the “focus of expansion”
• Simple form of navigation (insects use it)
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 8
hDp://w
ww.hizook.com
/blog/2010/02/16/learning-‐es4mate-‐robot-‐m
o4on-‐and-‐find-‐unexpected-‐objects-‐op4cal-‐flow
Modeling Camera Motion
4/16/14 CS 510, Image Computa4on, ©Ross Beveridge & Bruce Draper 9
Which way is the camera moving in these scenes?
hDp://w
ww.hizook.com
/blog/2010/02/16/learning-‐es4mate-‐robot-‐m
o4on-‐and-‐find-‐unexpected-‐objects-‐op4cal-‐flow
Top Related