Facial Motion Cloning Using Global Shape Deformation
description
Transcript of Facial Motion Cloning Using Global Shape Deformation
![Page 1: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/1.jpg)
Facial Motion Cloning Using Global Shape Deformation
Marco Fratarcangeli and Marco SchaerfUniversity of Rome “La Sapienza”
http://www.dis.uniroma1.it/~frat
![Page 2: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/2.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Motivation
• Reduce artistic skill and time required for animation by re-using existing animation
• Solution: Facial Motion Cloning
Source Target 3Target 2Target 1
![Page 3: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/3.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Overview
• Motivation
• Related Work
• Our Approach– Input & Output– Shape fitting– Cloning process
• Results
• Conclusions
![Page 4: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/4.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Related Work
• Expression Cloning [Noh et Neumann 2001]
![Page 5: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/5.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Related Work
• Facial Motion Cloning [Pandzic 2003]
![Page 6: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/6.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Input & Output
• Manually picked feature points on the input meshes;
• Feature points are compliant with the MPEG-4 specification;(Facial Definition Points)
![Page 7: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/7.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Input & Output
• 84 key movements defined by MPEG-4(Facial Animation Parameters)
…
…
• Output: corresponding key positions for the target face
![Page 8: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/8.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Source Shape Fitting
Find the morphing function f(P) fitting the source into the target(Radial Basis Functions [Fang 96])
Volume Morphing
![Page 9: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/9.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Radial Basis Functions (RBF)
if we find a suitable set of correspondence points between source and target,
then, we can calculate the interpolation function f(p) fitting the source into the target. [Fang 96]
That is, given the known data ui = f (pi), we can compute uk = f (pk).
![Page 10: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/10.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Iterative Fitting – Visual Example
Enriching the correspondence set precise fitting
![Page 11: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/11.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Iterative Fitting
• After the initial rough fitting…
1. Project target vertices towards source surface
2. Insert the target vertices having
biggest distance into the correspondences
set
3. Recompute the interpolation function
f(P) and perform morphing
4. Is the source face
fitted into the target?
NO YES
Map & clone…
![Page 12: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/12.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Source Shape Fitting
Find the morphing function f(P) fitting the source into the target(Radial Basis Functions [Fang 96])
Volume Morphing Map the target into the deformed source
![Page 13: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/13.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Cloning Process
Use f(P) to deform all the source key positions...
... and map the displacements to the target faces.
![Page 14: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/14.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
![Page 15: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/15.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Results - Performance
Iterations Total Time (s)
beta data 8 8.8
data beta 3 1.7
joakim data 8 10.0
data joakim 4 2.1
beta kevin 9 10.0
kevin beta 5 14.0
![Page 16: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/16.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Results – Error Assessment
source source
Beta 0.028 %
Data 0.280 %
Joakim 0.084 %
kevin 0.141 %
Average of the difference between the initial coordinates and the final coordinates
![Page 17: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/17.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
Conclusions
• PROs:– Pre-existing facial animation becomes reusable;– Fast cloning process: in seconds we have a brand
new talking head (1-20 secs on a AMD 2,14 GHz);– Animation through linear interpolation of the key
positions leads real-time performance; – MPEG-4 parametrization permits very low bit-rate
transmission [Model-based coding - Forchheimer 83];• CONs:
– Target models can not have higher resolution than source models;
– Projection method (Target Source) is not perfect ...
![Page 18: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/18.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science
The End
Further material soon here:
http://www.dis.uniroma1.it/~frat
![Page 19: Facial Motion Cloning Using Global Shape Deformation](https://reader034.fdocuments.net/reader034/viewer/2022051517/568153dc550346895dc1d607/html5/thumbnails/19.jpg)
University of Rome "La Sapienza" - Department of Computer and Systems Science