Probabilistic Tracking and Recognition of Non-rigid Hand Motion

Huang Fei, Ian ReidDepartment of Engineering Science

Oxford University

The Problem• Simultaneous Tracking and Recognition• Articulation and Self-Occlusion• Cluttered Background Scene and Occlusion

Two Successive Frames From A Video Sequence Two Successive Frames From A Video Sequence

Previous Research

• Kinematic Model v.s. Appearance Model• Toyama & Blake “Metric Mixture Tracker” Merits: -Exemplar v.s. Model -Spatial-Temporal Filtering Disadvantages: -Contour (Edges) v.s. Region (Silhouettes) -Joint Observation Density of Two Independent Processes

Method

• System Diagram of Joint Bayes Filter

• The Interaction Between Two Components in Joint Bayes Filter

Discrete Appearance Tracker

• Non-Rigid Appearances v.s. Speech Signal• Assumption: -Representative Hand Appearances -Non-Rigid Motion Observe Markov Dependence• The Aim of Learning: -Exemplar as Shape Tracker Representation -Articulated Human Motion Dynamics

Visualizing Non-Rigid Hand Motion• Local Linear Embedding Algorithm (S.Roweis & L.Saul 2000)

Robust Region Tracker

• Use Probabilistic Colour Histogram Tracker (P.Prez

et.al. ECCV 2002) As Global Region Estimator

• Tracking Global Region and Articulated Motion

Experiments

Frame 1Frame 1 Frame 2Frame 2 Frame 3Frame 3 Frame 4Frame 4 Frame 5Frame 5

Frame 6Frame 6 Frame 7Frame 7 Frame 8Frame 8 Frame 9Frame 9 Frame 10Frame 10

Frame 11Frame 11 Frame 12Frame 12 Frame 13Frame 13 Frame 14Frame 14 Frame 15Frame 15

• Coping with Occlusion Clutter

Frame 1Frame 1 Frame 2Frame 2 Frame 3Frame 3 Frame 4Frame 4 Frame 5Frame 5

Frame 6Frame 6 Frame 7Frame 7 Frame 8Frame 8 Frame 9Frame 9 Frame 10Frame 10

Frame 11Frame 11 Frame 12Frame 12 Frame 13Frame 13 Frame 14Frame 14 Frame 15Frame 15

Conclusion

• Two Independent Dynamic Processes • Two Bayesian Tracker =>Joint Bayes Filter• Robust Global Region Estimator• Robust State-Based Inference