BUNRAKU Perception, decision and action of real and ... 2nd Sino-French Symposium, Yellow Mountains...

1The 2nd Sino-French Symposium, Yellow Mountains City, august 2009, Georges Dumont

BUNRAKUPerception, decision and action of real and virtual humans in

virtual environments and impact on real environments

Georges [email protected]

Rennes, France


Why Bunraku ?

Form of traditional Japanese anthropomorphicpuppet theatre


Team Composition in May 2009

9 Teachers-Researchers: B. Arnaldi, K. Bouatouch, Y. Beckers R. Cozot, T. Duval, G. Dumont, V. Gouranton, F. Lamarche, M. Marchal

3 Researchers : S. Donikian, A. Lécuyer, J. Pettré

2 Visiting Researchers : S. Gibet, M. Christie

2 Collaborators: F. Multon, R. Kulpa

2 Research Engineers : A. Chauffaut, C. Bouville

3 Post-doc: N. Ouarti, Z. Gao, T. Regia-Corte

18 PhD students : 3 Chinese students 7 Technical staff

Total : 46


Our challenges

Increase cross fertilization betweentwo complementary research themes Virtual Reality and Virtual Humans

Allow natural interaction between Real and Virtual Humans

Combine two kinds of interaction: Cognitive and physical


Complementary Research Themes

Multimodal Interaction with objects within the world A generic multi-level model of an object Multimodal rendering

Visual, haptic, audio, cognitive Acting on the objects of the world

Language, gesture, mind

Expressive Autonomous Characters Complex and believable movements for

human-like characters Unified architecture to model individual

and collective human behaviours

Interactive scenario languages


Real-time blur effects

User interactivity in a poly-artistic work

Visual display of contact states

Path planning in a 3D model

A multi-field scientific background

Object modelling Physics and mechanical science: B. Arnaldi, G. Dumont, M. Marchal

Rendering Visual (Computer Graphics and Optic): K. Bouatouch, R. Cozot Haptic (Robotic, cognitive sciences): G. Dumont, A. Lécuyer, M.

Marchal Interaction

3D Interaction (Man-Machine Interface): T. Duval, A. Lécuyer Collaboration (Distributed architecture): T. Duval, V. Gouranton

Virtual human Motion control (Biomechanics): F. Multon, R. Kulpa Behaviour (Artificial Intelligence, behavioural and cognitive sciences):

S. Donikian, F. Lamarche, J. Pettré Scenario

Languages (Computer science): Y. Bekkers, S. Donikian Interactive drama (Literature, cinema, theatre): S. Donikian


Hardware and software platform for VirtualReality

http://www.openmask.org

OpenMASK:an open sourceVR middleware

A. Chauffaut


Different application areas

Industry Simulation Training Collaborative work

Sport Sport Performance Analysis Sport Training

Narrative & interactive Arts Works Video Games Interactive Drama Interactive Choreography


French and European Collaborative Projects

Multimodal Interaction with objects within the world Collaborative interaction in VR. [2007-2009], ANR Remote collaborative multi-domain pre/post-processing [2009-2011], ANR Brain-Computer Interfaces [2006-2012], ANR Interactive deformation and haptic feedback [2007-2010], competitive Natural Interactive Walking [2008-2010], FET Open

Expressive Autonomous Characters Simulation of user activity in train stations [2006-2008], Contract Human motion and behaviour control [2005-2009], competitive Crowd Simulation [2009-2011], ANR Human locomotion [2008-2010], ANR Sign-based Communication [2008-2010], ANR

Interactive scenario languages Virtual Training for maintenance activities [2002-2010], contract Human activity inside a digital factory [2006-2009], ANR Interactive Storytelling [2009-2011], NOE FP7 New generation virtual studio [2009-2010], small enterprise+Local council

Creation of Start-up : Stéphane Donikian [2009]


International Collaborations

Associate team on Virtual Humansand Augmented Reality State Key Lab CAD & CG (Q. Peng)

Core group of the FP6 European Network ofExcellence INTUITION [2004-2008] 60 industrial and academic partners Leader of the Haptic Working Group and Sustainability

Work Package (EuroVR)

Associate team on Real-Time Renderingand Augmented Reality University Central Florida (S. Pattanaik)

JST/CNRS on Virtual Reality The University of Tokyo (M. Hirose), Keio University (M. Inami, P. Codognet)

PHC on the use of dynamics for athletic movements University of Edinburgh (T. Komura)


Thank you for your attention

Next…

« From Motion Capture to Muscles Forces in Human Elbow Aiming at Improving Ergonomics of Working Stations »

Julien Pettré:« Interactions between Virtual Walkers »

Frank Multon:« Animating real-time humans for analysing the perception/action coupling in interactive environments »


From Motion Capture to Muscles Forces in Human ElbowAiming at Improving Ergonomics of Working Stations

Charles Pontonnier’s workPresented by Georges Dumont

ENS Cachan / IRISA – BUNRAKUhttp://www.irisa.fr/bunraku

{Charles.Pontonnier,Georges.Dumont}@irisa.fr


Aim

To improve ergonomics of working stations for the forearm and thehand

To use of Virtual Reality to check the ergonomics of the workingstation during the design phase

To develop a faster method and a faster model than the actualones in order to implement a real-time application


Partial survey on musculoskeletal models

[Maurel 1999]: Realistic 3D model of the shoulder : kinematics and topological information on muscles No functional application

[Chalfoun 2005]: Inverse method by optimization under non-linear constraints EMG comparison provides good results No computation time data and no motion capture data as input

[Yamane 2005] Complete musculoskeletal system (155 dofs driven by 366 muscles) Physiological models of muscle activations and EMG validation Heavy computation time (0,9s/frame)

[Holzbaur 2005] Relies on OpenSim project [Delp 2007] Validated method Heavy computation time: 1Hz


Synthesis of related works

Two main methods [Buchanan 2004] : Direct : from muscle activity measurement to motion Inverse : from motion to muscle forces computation

Four examples of musculoskeletal models: Medical diagnosis oriented models They does not reach real-time constraint

Used for validating our model


External forces

Inverse Kinematics

Muscle forces optimization undernon-linear constraints

Motion capture

Markers position

Joint positions

Muscle forces

Model

Inverse Dynamics

Joint torques

1

2

3

4

Obtaining muscle forces (Inverse method)


Inverse kinematics : skeleton mapping

épaule

coude

poignet

main

0R

trunk

1 2 3, ,q q q

4q

5q

6q

7q

Kinema'cal modelof the right arm

Mo'on capture

2

( , ) ( )d q df q X X q X= !"uuuuuurr uuur r uuur

Minimize for each .me stepand for each link of :

Joint angles

With respect of ar.cular limits

Scaling for skeleton adapta.on


Joint torques associated to the mo'on

Joint angles associated to the mo'on

• Dynamical model designed with MATLAB‐Simmechanics

• Automa.c scaling

Inverse dynamics


Joint torques Muscles topology

Physiological limits

Muscle forces associated to the mo'on

Costfunc.on :muscleac.vity

Addi.onalconstraints(co‐contrac.on)

2

max

1 2 3 4

max

For each frame :

min ( )( )

( , , , )

Under Constraints :

( ) 0

( ) ( ) 0

i

m

i i

m

i i i i

Ff F

F

F F F F F

h F c FR

g F F F

! "= # $

% &

=

= ' =

= ' (

)

)

Computation of muscle forces

!2 1 3 4 2( ) ( )h F F F F q F!= + + "


Results : extension of the elbow

Triceps activity


Discussion

Computation time is quite important ~25 Hz to compare to ~1Hz for OpenSim)

Static Co-contraction factor Should be modified by the motion dynamics

Need for further validation New motion capture experiments are mandatory


Future work

External forces

Inverse Kinematics

Muscle forces optimization

Motion capture

Markers position

Joint positions

Muscle forces

Model

Inverse Dynamics

Joint torques

Muscle forces interpolation Database


Many Thanks For Your Attention

From Motion Capture to Muscles Forces in Human ElbowAiming at Improving Ergonomics of Working Stations

Charles Pontonnier’s workPresented by Georges Dumont


We will be pleased to Welcome you toCASA 2010 in Saint-Malo


Overall Objective of our Research Team


k

r4q

0R

Humerus

1 2 3, ,q q q

iF

Radius

B

A

O

iF

Biceps contribu.on for .me step i to theextension/flexion torque of the elbow :

A is the inser.on of the bicepsB his originO center of the elbow ar.cula.on

[ ^ ].i i i

ABFR OA F k

AB

=

uuuruuur r

uuur

Torques for muscle contributions

BUNRAKU Perception, decision and action of real and ... 2nd Sino-French Symposium, Yellow Mountains...

Documents

Transcript of BUNRAKU Perception, decision and action of real and ... 2nd Sino-French Symposium, Yellow Mountains...