Movement Tracking in Real-time Hand Gesture Recognition
-
Upload
pranav-kulkarni -
Category
Science
-
view
55 -
download
0
Transcript of Movement Tracking in Real-time Hand Gesture Recognition
![Page 1: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/1.jpg)
Movement Tracking in
Real-time Hand Gesture Recognition
PresentersPurohit Pankaj (W579074)Salagar Muaaz (W579080)Kulkarni Pranav (2010BCS203)Desale Ritesh (2010BCS211)
Seminar GuideMrs. S. S. Solapure
![Page 2: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/2.jpg)
Agenda IntroductionReference Paper and Its Contents
ApplicationConclusionQuestions
![Page 3: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/3.jpg)
Agenda IntroductionReferenced Paper and Its Contents
ApplicationConclusionQuestions
![Page 4: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/4.jpg)
Introduction
![Page 5: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/5.jpg)
Introduction
Gesture Recognition
Hand Gesture Recognition
![Page 6: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/6.jpg)
![Page 7: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/7.jpg)
Introduction (Contd.)
What is it?
How it works?
![Page 8: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/8.jpg)
![Page 9: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/9.jpg)
Introduction (Contd.)
What is its NEED?
Advantages Natural Interaction Builds a Richer Bridge Remote Interaction Wonderful Gaming Experience
![Page 10: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/10.jpg)
Agenda IntroductionReference Paper and Its Contents
ApplicationConclusionQuestions
![Page 11: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/11.jpg)
Reference Paper
Movement Tracking in Real-time Hand Gesture
RecognitionAuthored by
Hong-Min Zhu & Chi-Man PunDepartment of Computer and Information
ScienceUniversity of Macau, Macau SAR, China
{ma86560, cmpun} [at] umac.mo
9th IEEE/ACIS International Conference on Computer and Information Science
![Page 12: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/12.jpg)
Reference PaperWhat does it say?This paper deals with overcoming of SCHD technique for Hand Gesture Recognition using newly improved Algorithm, IFDHD
![Page 13: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/13.jpg)
Procedures in General Framework of Gesture Recognition
![Page 14: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/14.jpg)
Previous Work DoneTemporal Hand GestureAssumptions
Camera User Synchronization Uniform Lightening Condition Simple Background Features Frame Rate – Gesture Speed
Coordination
![Page 15: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/15.jpg)
S C H D Skin Color based Hand Detection
J. Kovac and P. Peer – Designed Skin Classifier
Rules Pixel is classified as a skin pixel if:
Value of Red > 95, Green > 40 and Blue > 20 & max{R, G, B} - min{R, G, B} > 15 & |R - G| > 15 and R > G and R > B
![Page 16: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/16.jpg)
Proposed Solution Problems with SCHD
Computationally Expensive Skin-like Object Ambiguity Illumination Parameters Skin Color Variation
Solution – Motivated from BSHD
IFDHDInter-Frame Difference based Hand Detection
![Page 17: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/17.jpg)
Proposed Solution
Hand DetectionModule
Motion TrackingModule
![Page 18: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/18.jpg)
Hand Detection Module
Figure 3.1 Zoomed Mode for Hand Detection Module
![Page 19: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/19.jpg)
Algorithm for Hand Detection
Input: Frames Fi = 1..N from video segmentSteps:
1. Convert frame F1 to grayscale2. Repeat (until end of video segment)
1. Convert frame Fi to grayscale2. Intensity difference image D0 = |Fi – F1|3. Binary image I = (D0 > T0)4. Do image opening on I followed by closing5. Splitting of Large regions into max size boundary box
as 60x806. Calculate center co-ordinate
Output: Center coordinate of each region in each frame
![Page 20: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/20.jpg)
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
![Page 21: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/21.jpg)
Fig. 4. Result of SCHD(a) original frame
![Page 22: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/22.jpg)
Fig. 4. Result of SCHD(b) Skin Pixel Classification
![Page 23: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/23.jpg)
Fig. 4. Result of SCHD(c) De-noise & Region
Connection
![Page 24: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/24.jpg)
Fig. 4. Result of SCHD(d) Region Splitting
![Page 25: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/25.jpg)
Fig. 4. Result of SCHD(e) Centers of Each Region
![Page 26: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/26.jpg)
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
![Page 27: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/27.jpg)
Fig. 5. Effect of Lightening Condition
(a) Original Frame
![Page 28: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/28.jpg)
Fig. 5. Effect of Lightening Condition
(b) Skin Pixel Classification
![Page 29: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/29.jpg)
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
![Page 30: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/30.jpg)
Fig. 6. Result of IFDHD
(a) 1st Frame
![Page 31: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/31.jpg)
Fig. 6. Result of IFDHD
(b) 11th Frame
![Page 32: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/32.jpg)
Fig. 6. Result of IFDHD
(c) Subtraction: (b) - (a)
![Page 33: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/33.jpg)
Fig. 6. Result of IFDHD
(d) Threshold of (c)
![Page 34: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/34.jpg)
Fig. 6. Result of IFDHD
(e) De-noise
![Page 35: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/35.jpg)
Fig. 6. Result of IFDHD
(f) Region Splitting
![Page 36: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/36.jpg)
Fig. 6. Result of IFDHD
(g) Centers of Regions
![Page 37: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/37.jpg)
System Domain (Contd.)
![Page 38: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/38.jpg)
Movement Tracking Module
Figure 3.2 Zoomed Mode for Movement Tracking Module
![Page 39: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/39.jpg)
Algorithm for Movement Tracking
Input: Region centers Detected in each frameSteps:
1. Initialize the start of frame2. Repeat (for each frame > 1)
1. Identify tail locations and store2. Calculate matrix of distances between centers and
tail locations3. Repeatedly select – min(Distance ( I ), Distance
( J )) 4. If Distance( I ) < Threshold then append Center to
Gesture and delete Distance( I ) Else initialize new Gesture start location
5. Select Gesture Frame that has the maximal standard deviation
6. Smooth movement track Gesture and interpolate it to Number of Center Coordinate falls coordinates
Output: Encoding of movement track
![Page 40: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/40.jpg)
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
![Page 41: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/41.jpg)
Fig. 7. SCHD Based Movement TrackingFirst Row: Last Frame of Video Segment
![Page 42: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/42.jpg)
Fig. 7. SCHD Based Movement TrackingSecond Row: Detected Digit Track
![Page 43: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/43.jpg)
Fig. 7. SCHD Based Movement TrackingThird Row: Smoothed Track
![Page 44: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/44.jpg)
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
![Page 45: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/45.jpg)
Fig. 8. IFDHD Based Movement TrackingFirst Row: Last Row of Video Segment
![Page 46: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/46.jpg)
Fig. 8. IFDHD Based Movement TrackingSecond Row: Detected Digit Track
![Page 47: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/47.jpg)
Fig. 8. IFDHD Based Movement TrackingThird Row: Smoothed Track
![Page 48: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/48.jpg)
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
![Page 49: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/49.jpg)
Efficiency Measurement
Table 1. Comparing the Efficiencies of SCHD and IFDHD
![Page 50: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/50.jpg)
Outline IntroductionReferenced PaperApplication ScenarioConclusionQuestions
![Page 51: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/51.jpg)
Reference PaperAmerican Sign Language Recognition System for Hearing Impaired People Using Cartesian
Genetic ProgrammingAuthored ByFahad Ullah
Department of Computer Systems Engineering,University Of Engineering & Technology,
Peshawar, Pakistan
Proceeding of 5th International Conference on Automation, Robotics and Applications, New
Zealand
![Page 52: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/52.jpg)
Application Scenario
Why the interfaces are changing ?
How many Apps Out there? Have you tried X-box,PSP-2,Mac-OSX January 9, 2012, 66 million Xbox 360
consoles have been sold worldwide.
New era of Interfaces
![Page 53: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/53.jpg)
Application Scenario What if you can’t speak? ASL CGP (Cartesian Genetic Programming) How it works? Genetic programming an Overview:
Probabilistic search Darwinian principle of natural
selection Naturally occurring genetic operations
such as crossover and mutation.
![Page 54: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/54.jpg)
• Better individuals are preferred• Best is not always picked• Worst is not necessarily excluded• Nothing is guaranteed• Mixture of greedy exploitation and
adventurous exploration• Similarities to simulated annealing
(SA)
Probabilistic Selection Based On Fitness
![Page 55: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/55.jpg)
Workflow
![Page 56: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/56.jpg)
![Page 57: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/57.jpg)
ASL using CGP
26 English language alphabets are trained and Identified
The system uses 26 binary images representing the different alphabets
Mentioned system with a Dictionary correction ability in order to increase the overall accuracy of the system.
![Page 58: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/58.jpg)
Outline IntroductionReferenced PaperApplicationConclusionQuestions
![Page 59: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/59.jpg)
ConclusionProposed IFDHDServing Feature Extraction Stage
Overcoming the pitfalls of SCHD
![Page 60: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/60.jpg)
Outline IntroductionReferenced PaperApplicationConclusionQuestions
![Page 61: Movement Tracking in Real-time Hand Gesture Recognition](https://reader031.fdocuments.net/reader031/viewer/2022012923/58ef8d4d1a28ab9e448b4591/html5/thumbnails/61.jpg)
Questions, IF ANY?
Q?Thank You