Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards...
Transcript of Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards...
![Page 1: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/1.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human Dynamics
Towards Human-Realistic Animation by Observing Real Human Dynamics
Darren CoskerDepartment of Computer Science,University of Bath
![Page 2: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/2.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
MotivationMotivation
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
![Page 3: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/3.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
OverviewOverview
• Analysis and Re-Synthesis of Facial Dynamics– Dynamics > CG Puppet– Dynamics > us– Dynamics > Computer Vision
• Intuitive/Automatic methods for animation– Voice Driven Animation/Analysis– Reactive Avatars
![Page 4: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/4.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/Resynthesising DynamicsAnalysing/Resynthesising Dynamics
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
Time
Par
amet
er M
agni
tude
![Page 5: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/5.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/Resynthesising DynamicsAnalysing/Resynthesising Dynamics
• Study effect of manipulating temporal characteristics of emotions, e.g. smiles
QuickTime™ and a decompressor
are needed to see this picture.
[Emotion 2007]
![Page 6: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/6.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
![Page 7: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/7.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics
![Page 8: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/8.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics
• What did we learn?– ‘Genuine’ Dynamics = More
likeable/Trustworthy/Employable– ‘Fake’ Dynamics = Less
Likeable/Trustworthy/Employable
• Relationship between Attractiveness and Facial Dynamics
• To Do! - Mock Video Game Scenario
![Page 9: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/9.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
1. Define facial regions local
to sub-facial actions
2. Construct new training
set of artificial images
Analysing Multiple ActionsAnalysing Multiple Actions
![Page 10: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/10.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing Multiple ActionsAnalysing Multiple Actions
3. Create a corresponding artificial training set of shape vectors
![Page 11: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/11.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Performance AnalysisPerformance Analysis
![Page 12: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/12.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Image Based TransferImage Based Transfer
QuickTime™ and aH.264 decompressor
are needed to see this picture.
![Page 13: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/13.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Performance AnalysisPerformance Analysis
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
![Page 14: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/14.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Model Transfer3D Model Transfer
• Animation of a 3D facial model
![Page 15: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/15.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Model Transfer3D Model Transfer
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
Dynamic knowledge easily applied to existing pipelines
![Page 16: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/16.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Facial Modelling3D Facial Modelling
• 4D Camera– Continuous 3D Scanning (Cardiff/Surrey)
![Page 17: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/17.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Facial Modelling3D Facial Modelling
(Link to 3D Example Expression)
![Page 18: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/18.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Facial Modelling3D Facial Modelling
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
![Page 19: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/19.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Voice Driven AnimationVoice Driven AnimationVisual Speech Synthesis
‘Non-Verbal’ Action Synthesis
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aCinepak decompressor
are needed to see this picture.
QuickTime™ and aNone decompressor
are needed to see this picture.
![Page 20: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/20.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Perceptual EvaluationPerceptual Evaluation
• Motivation: How best to evaluate facial animation?– Perceptual test based on McGurk effect
QuickTime™ and aSorenson Video decompressorare needed to see this picture.
[ACM TAP 2005]
![Page 21: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/21.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Perceptual EvaluationPerceptual Evaluation
• Record participant responses
• Compare real and synthetic
• Subtle - participants blind to real motivation
• Isolate strengths and weaknesses
![Page 22: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/22.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Reactive Full Body Avatar AnimationReactive Full Body Avatar Animation
• Motivation:– Animate based on
an input performance
– Key avatar drives other avatar behaviour
QuickTime™ and aSorenson Video 3 decompressorare needed to see this picture.
![Page 23: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/23.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Conclusions and Future DirectionsConclusions and Future Directions• Humans are very sensitive to different dynamics
• Different interpretation depending on timing
• Human realism will depend on understanding dynamic behaviours
• Realism will also depend on the accurate modelling and resynthesis of dynamics - e.g. McGurk effect
• Technology now allows study of 3D dynamics - our work has only just started!
![Page 24: Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren.](https://reader035.fdocuments.net/reader035/viewer/2022081518/5515e0c5550346d46f8b4cdf/html5/thumbnails/24.jpg)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Thanks for listening!
Acknowledgements:• Eva Krumhuber, Antony Manstead, Lanthao Benedikt,
David Marshall, Paul Rosin, Alistair Calvert, Cathy Holt, Daniel Mason, Gemma Whatling, Simon Rushton, Susan Paddock
• RAEng, EPSRC.