Application of feature point matching to video stabilization
-
Upload
nikhil-prathapani -
Category
Engineering
-
view
259 -
download
4
description
Transcript of Application of feature point matching to video stabilization
![Page 1: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/1.jpg)
APPLICATION OF FEATURE POINT MATCHING TO VIDEO STABILIZATION
By
Nikhil Prathapani
Student Member, IEEE
![Page 2: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/2.jpg)
INTRODUCTION
Along with advancement, digital video has introduced new
problems like video noising, video de-stabilization and video
jitter.
In order to overcome these problems, new techniques like video
enhancement and video stabilization have been proposed.
Of the proposed video stabilization techniques, all most all of
them require prior knowledge of prominent frame.
But the proposed technique is based on RANSAC*, SSD and SIFT,
it does not require any erstwhile knowledge of prominent video
frame for stabilization purpose.
*Tordoff, B; Murray, DW. "Guided sampling and consensus for motion estimation."European Conference n Computer Vision, 2002.
![Page 3: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/3.jpg)
Why estimate visual motion? Visual Motion can be annoying
Camera instabilities, jitter Measure it; remove it (stabilize)
Visual Motion indicates dynamics in the scene Moving objects, behavior Track objects and analyze trajectories
![Page 4: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/4.jpg)
Getting six parameters
SIFT algorithm – Find corresponding pairs At time k
It needs three pairs to determine a unique solution
Y X A
![Page 5: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/5.jpg)
SIFT correspondence from frame 200,201 in outdoor sequence STREET
![Page 6: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/6.jpg)
The fundamental matrix F
C C’T=C’-C
Rp p’
TRp'p Two reference frames are related via the extrinsic parameters
![Page 7: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/7.jpg)
The fundamental matrix F The fundamental matrix is the algebraic
representation of epipolar geometry
The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’ in the two images
0Fx'xT 0lxT
![Page 8: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/8.jpg)
RANSAC (Random Sampling and Consensus )
repeatselect minimal sample (8 matches)compute solution(s) for Fdetermine inliers
until (#inliers,#samples)>95% or too many times
compute F based on all inliers
![Page 9: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/9.jpg)
SSD (sum of squared differences) surface – textured area
![Page 10: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/10.jpg)
SSD surface – edge
![Page 11: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/11.jpg)
SSD – homogeneous area
![Page 12: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/12.jpg)
ALGORITHM
![Page 13: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/13.jpg)
ANALYSIS AND RESULTS
Step 1- Reading frames from a movie file
Step 2- Collecting Salient Points from Each Frame
SIFT
Step 3- Selecting Correspondences Between Points
SSD
Step 4-. Estimating Transform from Noisy Correspondence
RANSACaffine transform will be a 3-by-3 matrix:[a_1 a_3t_r;a_2 a_4t_c;
0 0 1]The parameters ‘a’ define scale, rotation, and sheering effects of the transform, while the parameters ‘t’ are
translation parameters.
![Page 14: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/14.jpg)
Step 5- Transform Approximation and Smoothing
Step 6- Run on the Full Video
Raw input mean and Corrected sequence mean images
PSNR MSE
RAW INPUT MEAN 22.5406 3.62
CORRECTED SEQUENCE MEAN
25.5725 3.59
![Page 15: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/15.jpg)
CONCLUSION
The paper presents a comprehensive and thorough
approach to video stabilizing videos using MATLAB.
This kind of novel approach to video stabilizing [6, 7, 8]
without prior knowledge of prominent features in the frames
has targeted many applications in the fields of motion
estimation, remote sensing, and airborne applications
![Page 16: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/16.jpg)
REFERENCES
[1]P. A. Keller, The cathode-ray tube: technology, history and applications,Palisades Press, 1991, ISBN 0963155903.[2]W. C. O’Mara, Liquid crystal flat panel display: manufacturing science andtechnology, Van Nostrand Reinhold, 1993, ISBN 0442014287.[3]J. Hutchison, “Plasma display panels: the colorful history of an Illinois tech-nology”, ECE alumni news, university of Illinois, vol. 36(1), 2002.[4]C. Poynton, Digital video and HDTV algorithms and interfaces, MorganKaufmann, 2003, ISBN 1558607927.[5] Tordoff, B; Murray, DW. "Guided sampling and consensus for motion estimation."European Conference n Computer Vision, 2002.[6] Lee, KY; Chuang, YY; Chen, BY; Ouhyoung, M. "Video Stabilization using Robust Feature Trajectories." National Taiwan University, 2009.[7] Litvin, A; Konrad, J; Karl, WC. "Probabilistic video stabilization using Kalman filtering and mosaicking." IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications and Proc., 2003.[8] Matsushita, Y; Ofek, E; Tang, X; Shum, HY. "Full-frame Video Stabilization." Microsoft® Research Asia.CVPR 2005.
![Page 17: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/17.jpg)
Acknowledgements
I am deeply indebted to my parents who have always backed me equally during all times.
![Page 18: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/18.jpg)
THANK YOU
![Page 19: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/19.jpg)
![Page 20: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/20.jpg)
Any Queries?
For research articles, papers and projects in the fields of Image Processing and Nanoelectronics,
you can connect to my research profile:
http://jntuhcej.academia.edu/NikhilPrathapani
![Page 21: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/21.jpg)
SIFT detector proposed considers local image characteristic and retrieves feature points that are invariant to image rotation, scaling, translation, partly illumination changes and projective transform.
The scale-invariant feature extractor detects feature points through a staged filtering approach that identifies stable points in the scale-space.
Scale Invariant Feature Transform
![Page 22: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/22.jpg)
Why Features? A brief yet comprehensive representation of
the image Can be used for:
Image alignment Object recognition 3D reconstruction Motion tracking Indexing and database search More…
![Page 23: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/23.jpg)
Desired Feature Properties• Robustness => Invariance to changes in
illumination, scale, rotation, affine, perspective • Locality => robustness to occlusion and clutter.
• Distinctiveness => easy to match to a large database of objects.
• Quantity => many features can be generated for even small objects
• Efficiency => computationally “cheap”, real-time performance
![Page 24: Application of feature point matching to video stabilization](https://reader036.fdocuments.net/reader036/viewer/2022062307/5574816ed8b42a0f178b5315/html5/thumbnails/24.jpg)
Algorithm1. Scale-space extrema detection2. Keypoint localization3. Orientation assignment4. Keypoint descriptor