1The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
BUNRAKUPerception, decision and action of real and virtual humans in
virtual environments and impact on real environments
Georges [email protected]
Rennes, France
2The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Why Bunraku ?
Form of traditional Japanese anthropomorphicpuppet theatre
3The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Team Composition in May 2009
9 Teachers-Researchers: B. Arnaldi, K. Bouatouch, Y. Beckers R. Cozot, T. Duval, G. Dumont, V. Gouranton, F. Lamarche, M. Marchal
3 Researchers : S. Donikian, A. Lécuyer, J. Pettré
2 Visiting Researchers : S. Gibet, M. Christie
2 Collaborators: F. Multon, R. Kulpa
2 Research Engineers : A. Chauffaut, C. Bouville
3 Post-doc: N. Ouarti, Z. Gao, T. Regia-Corte
18 PhD students : 3 Chinese students 7 Technical staff
Total : 46
4The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Our challenges
Increase cross fertilization betweentwo complementary research themes Virtual Reality and Virtual Humans
Allow natural interaction between Real and Virtual Humans
Combine two kinds of interaction: Cognitive and physical
5The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Complementary Research Themes
Multimodal Interaction with objects within the world A generic multi-level model of an object Multimodal rendering
Visual, haptic, audio, cognitive Acting on the objects of the world
Language, gesture, mind
Expressive Autonomous Characters Complex and believable movements for
human-like characters Unified architecture to model individual
and collective human behaviours
Interactive scenario languages
6The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Real-time blur effects
User interactivity in a poly-artistic work
Visual display of contact states
Path planning in a 3D model
A multi-field scientific background
Object modelling Physics and mechanical science: B. Arnaldi, G. Dumont, M. Marchal
Rendering Visual (Computer Graphics and Optic): K. Bouatouch, R. Cozot Haptic (Robotic, cognitive sciences): G. Dumont, A. Lécuyer, M.
Marchal Interaction
3D Interaction (Man-Machine Interface): T. Duval, A. Lécuyer Collaboration (Distributed architecture): T. Duval, V. Gouranton
Virtual human Motion control (Biomechanics): F. Multon, R. Kulpa Behaviour (Artificial Intelligence, behavioural and cognitive sciences):
S. Donikian, F. Lamarche, J. Pettré Scenario
Languages (Computer science): Y. Bekkers, S. Donikian Interactive drama (Literature, cinema, theatre): S. Donikian
7The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Hardware and software platform for VirtualReality
http://www.openmask.org
OpenMASK:an open sourceVR middleware
A. Chauffaut
8The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Different application areas
Industry Simulation Training Collaborative work
Sport Sport Performance Analysis Sport Training
Narrative & interactive Arts Works Video Games Interactive Drama Interactive Choreography
9The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
French and European Collaborative Projects
Multimodal Interaction with objects within the world Collaborative interaction in VR. [2007-2009], ANR Remote collaborative multi-domain pre/post-processing [2009-2011], ANR Brain-Computer Interfaces [2006-2012], ANR Interactive deformation and haptic feedback [2007-2010], competitive Natural Interactive Walking [2008-2010], FET Open
Expressive Autonomous Characters Simulation of user activity in train stations [2006-2008], Contract Human motion and behaviour control [2005-2009], competitive Crowd Simulation [2009-2011], ANR Human locomotion [2008-2010], ANR Sign-based Communication [2008-2010], ANR
Interactive scenario languages Virtual Training for maintenance activities [2002-2010], contract Human activity inside a digital factory [2006-2009], ANR Interactive Storytelling [2009-2011], NOE FP7 New generation virtual studio [2009-2010], small enterprise+Local council
Creation of Start-up : Stéphane Donikian [2009]
10The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
International Collaborations
Associate team on Virtual Humansand Augmented Reality State Key Lab CAD & CG (Q. Peng)
Core group of the FP6 European Network ofExcellence INTUITION [2004-2008] 60 industrial and academic partners Leader of the Haptic Working Group and Sustainability
Work Package (EuroVR)
Associate team on Real-Time Renderingand Augmented Reality University Central Florida (S. Pattanaik)
JST/CNRS on Virtual Reality The University of Tokyo (M. Hirose), Keio University (M. Inami, P. Codognet)
PHC on the use of dynamics for athletic movements University of Edinburgh (T. Komura)
11The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Thank you for your attention
Next…
« From Motion Capture to Muscles Forces in Human Elbow Aiming at Improving Ergonomics of Working Stations »
Julien Pettré:« Interactions between Virtual Walkers »
Frank Multon:« Animating real-time humans for analysing the perception/action coupling in interactive environments »
12The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
From Motion Capture to Muscles Forces in Human ElbowAiming at Improving Ergonomics of Working Stations
Charles Pontonnier’s workPresented by Georges Dumont
ENS Cachan / IRISA – BUNRAKUhttp://www.irisa.fr/bunraku
{Charles.Pontonnier,Georges.Dumont}@irisa.fr
13The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Aim
To improve ergonomics of working stations for the forearm and thehand
To use of Virtual Reality to check the ergonomics of the workingstation during the design phase
To develop a faster method and a faster model than the actualones in order to implement a real-time application
14The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Partial survey on musculoskeletal models
[Maurel 1999]: Realistic 3D model of the shoulder : kinematics and topological information on muscles No functional application
[Chalfoun 2005]: Inverse method by optimization under non-linear constraints EMG comparison provides good results No computation time data and no motion capture data as input
[Yamane 2005] Complete musculoskeletal system (155 dofs driven by 366 muscles) Physiological models of muscle activations and EMG validation Heavy computation time (0,9s/frame)
[Holzbaur 2005] Relies on OpenSim project [Delp 2007] Validated method Heavy computation time: 1Hz
15The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Synthesis of related works
Two main methods [Buchanan 2004] : Direct : from muscle activity measurement to motion Inverse : from motion to muscle forces computation
Four examples of musculoskeletal models: Medical diagnosis oriented models They does not reach real-time constraint
Used for validating our model
16The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
External forces
Inverse Kinematics
Muscle forces optimization undernon-linear constraints
Motion capture
Markers position
Joint positions
Muscle forces
Model
Inverse Dynamics
Joint torques
1
2
3
4
Obtaining muscle forces (Inverse method)
17The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Inverse kinematics : skeleton mapping
épaule
coude
poignet
main
0R
trunk
1 2 3, ,q q q
4q
5q
6q
7q
Kinema'cal modelof the right arm
Mo'on capture
2
( , ) ( )d q df q X X q X= !"uuuuuurr uuur r uuur
Minimize for each .me stepand for each link of :
Joint angles
With respect of ar.cular limits
Scaling for skeleton adapta.on
18The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Joint torques associated to the mo'on
Joint angles associated to the mo'on
• Dynamical model designed with MATLAB‐Simmechanics
• Automa.c scaling
Inverse dynamics
19The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Joint torques Muscles topology
Physiological limits
Muscle forces associated to the mo'on
Costfunc.on :muscleac.vity
Addi.onalconstraints(co‐contrac.on)
2
max
1 2 3 4
max
For each frame :
min ( )( )
( , , , )
Under Constraints :
( ) 0
( ) ( ) 0
i
m
i i
m
i i i i
Ff F
F
F F F F F
h F c FR
g F F F
! "= # $
% &
=
= ' =
= ' (
)
)
Computation of muscle forces
!2 1 3 4 2( ) ( )h F F F F q F!= + + "
20The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Results : extension of the elbow
Triceps activity
21The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Discussion
Computation time is quite important ~25 Hz to compare to ~1Hz for OpenSim)
Static Co-contraction factor Should be modified by the motion dynamics
Need for further validation New motion capture experiments are mandatory
22The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Future work
External forces
Inverse Kinematics
Muscle forces optimization
Motion capture
Markers position
Joint positions
Muscle forces
Model
Inverse Dynamics
Joint torques
Muscle forces interpolation Database
23The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Many Thanks For Your Attention
From Motion Capture to Muscles Forces in Human ElbowAiming at Improving Ergonomics of Working Stations
Charles Pontonnier’s workPresented by Georges Dumont
24The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
We will be pleased to Welcome you toCASA 2010 in Saint-Malo
25The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
Overall Objective of our Research Team
26The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont
k
r4q
0R
Humerus
1 2 3, ,q q q
iF
Radius
B
A
O
iF
Biceps contribu.on for .me step i to theextension/flexion torque of the elbow :
A is the inser.on of the bicepsB his originO center of the elbow ar.cula.on
[ ^ ].i i i
ABFR OA F k
AB
=
uuuruuur r
uuur
Torques for muscle contributions
Top Related