Facial Motion Cloning Using Global Shape Deformation

Marco Fratarcangeli and Marco SchaerfUniversity of Rome “La Sapienza”

http://www.dis.uniroma1.it/~frat

University of Rome "La Sapienza" - Department of Computer and Systems Science

Motivation

• Reduce artistic skill and time required for animation by re-using existing animation

• Solution: Facial Motion Cloning

Source Target 3Target 2Target 1

University of Rome "La Sapienza" - Department of Computer and Systems Science

Overview

• Motivation

• Related Work

• Our Approach– Input & Output– Shape fitting– Cloning process

• Results

• Conclusions

University of Rome "La Sapienza" - Department of Computer and Systems Science

Related Work

• Expression Cloning [Noh et Neumann 2001]

University of Rome "La Sapienza" - Department of Computer and Systems Science

Related Work

• Facial Motion Cloning [Pandzic 2003]

University of Rome "La Sapienza" - Department of Computer and Systems Science

Input & Output

• Manually picked feature points on the input meshes;

• Feature points are compliant with the MPEG-4 specification;(Facial Definition Points)

University of Rome "La Sapienza" - Department of Computer and Systems Science

Input & Output

• 84 key movements defined by MPEG-4(Facial Animation Parameters)

…

…

• Output: corresponding key positions for the target face

University of Rome "La Sapienza" - Department of Computer and Systems Science

Source Shape Fitting

Find the morphing function f(P) fitting the source into the target(Radial Basis Functions [Fang 96])

Volume Morphing

University of Rome "La Sapienza" - Department of Computer and Systems Science

Radial Basis Functions (RBF)

if we find a suitable set of correspondence points between source and target,

then, we can calculate the interpolation function f(p) fitting the source into the target. [Fang 96]

That is, given the known data ui = f (pi), we can compute uk = f (pk).

University of Rome "La Sapienza" - Department of Computer and Systems Science

Iterative Fitting – Visual Example

Enriching the correspondence set precise fitting

University of Rome "La Sapienza" - Department of Computer and Systems Science

Iterative Fitting

• After the initial rough fitting…

1. Project target vertices towards source surface

2. Insert the target vertices having

biggest distance into the correspondences

set

3. Recompute the interpolation function

f(P) and perform morphing

4. Is the source face

fitted into the target?

NO YES

Map & clone…

University of Rome "La Sapienza" - Department of Computer and Systems Science

Source Shape Fitting

Find the morphing function f(P) fitting the source into the target(Radial Basis Functions [Fang 96])

Volume Morphing Map the target into the deformed source

University of Rome "La Sapienza" - Department of Computer and Systems Science

Cloning Process

Use f(P) to deform all the source key positions...

... and map the displacements to the target faces.

University of Rome "La Sapienza" - Department of Computer and Systems Science

University of Rome "La Sapienza" - Department of Computer and Systems Science

Results - Performance

Iterations Total Time (s)

beta data 8 8.8

data beta 3 1.7

joakim data 8 10.0

data joakim 4 2.1

beta kevin 9 10.0

kevin beta 5 14.0

University of Rome "La Sapienza" - Department of Computer and Systems Science

Results – Error Assessment

source source

Beta 0.028 %

Data 0.280 %

Joakim 0.084 %

kevin 0.141 %

Average of the difference between the initial coordinates and the final coordinates

University of Rome "La Sapienza" - Department of Computer and Systems Science

Conclusions

• PROs:– Pre-existing facial animation becomes reusable;– Fast cloning process: in seconds we have a brand

new talking head (1-20 secs on a AMD 2,14 GHz);– Animation through linear interpolation of the key

positions leads real-time performance; – MPEG-4 parametrization permits very low bit-rate

transmission [Model-based coding - Forchheimer 83];• CONs:

– Target models can not have higher resolution than source models;

– Projection method (Target Source) is not perfect ...

University of Rome "La Sapienza" - Department of Computer and Systems Science

The End

Further material soon here:

http://www.dis.uniroma1.it/~frat

University of Rome "La Sapienza" - Department of Computer and Systems Science