Face to Face Face to Face with the Greeks Primary Schools ...
AAM based Face Tracking with Temporal Matching and Face Segmentation Mingcai Zhou 1 、 Lin Liang 2...
-
Upload
clyde-holland -
Category
Documents
-
view
217 -
download
0
Transcript of AAM based Face Tracking with Temporal Matching and Face Segmentation Mingcai Zhou 1 、 Lin Liang 2...
AAM based Face Tracking with Temporal Matching and Face SegmentationMingcai Zhou1 、 Lin Liang2 、 Jian Sun2 、 Yangsheng Wang11Institute of Automation Chinese Academy of Sciences, Beijing, China2Microsoft Research Asia Beijing, China
CVPR 2010
2
Outline• AAM Introduction• Related Work• Method and Theory• Experiment
3
AAM Introduction• A statistical model of shape and grey-level appearance
Shape model Appearance model
4
Shape Model Building:mean shape
:shape bases
,shape parameters
learn by PCAgenerate mean shape 、shape bases
5
Texture Model Building:mean appearance
:appearance bases
:appearance parameters
W(x)
灰階值
Shape-free patchMean shape
6
AAM Model Building
7
AAM Model Search• Find the optimal shape parameters and appearance parameters to minimize the difference between the warped-back appearance and synthesized appearance
( , )W x p
p
( ( , ))I W x p
map every pixel x in the model coordinate to its corresponding image point( , )W x p
0s
8
Problems- AAM tracker• Difficultly generalize to unseen images• Clutterd backgrounds
9
How to do?• A temporal matching constraint in AAM fitting -Enforce an inter-frame local appearance constraint between frames• Introduce color-based face segmentation as a soft constraint
10
Related Work -feature-based (mismatched local feature) Integrating multiple visual cues for robust real-time 3d face tracking, W.-K. Liao, D. Fidaleo, and G. G. Medioni. 2007 -intensity-based (fast illumination changes) Improved face model fitting on video sequences, X. Liu, F. Wheeler, and P. Tu. 2007
temporal matching constraint
11
Method and Theory• Extend basic AAM to Multi-band AAM– The texture(appearance) is a concatenation of three texture band values• The intensity (b)• X-direction gradient strength (c)• Y-direction gradient strength (d)
12
1. Select feature points with salient local appearances at previous frame2. I(t−1) to the Model coordinate and get the appearance A(t-1)
3. Use warping function W(x;pt) maps R(t-1) to a patch R(t) at frame t
Temporal Matching Constraint
13
Shape parameter Initialization
,
Face Motion Direction
14
Shape parameter Initialization
When r reaches the noise level expected in the correspondences, the algorithm stops
15
Shape parameter Initialization-Comparison
Motion direction
Feature matching Previous frame’s shape
16
Face Segmentation ConstraintWhere are the locations of the selected outline points in the model coordinate
{ }kx
17
Face Segmentation Constraint-Face Segmentation
18
Face Segmentation Constraint
19
Experiments
Lost frame num
20
Experiments
21
Conclusion─ Our tracking algorithm accurately localizes the facial components, such as eyes, brows, noses and mouths, under illumination changes as well as large expression and pose variations.
─ Our tracking algorithm runs in real-time . On a Pentium-4 3.0G computer, the algorithm’s speed is about 50 fps for the video with 320 × 240 resolution
22
Future Work─ Our tracker cannot robustly track profile views with large angles
─ The tracker’s ability to handle large occlusion also needs to be improved