H-Anim Motion Capture
Transcript of H-Anim Motion Capture
H-Anim Motion Capture
SC24 WG9 & Web3D Meetings
2016. 1. 11
Myeong Won Lee
The University of Suwon
Status of H-Anim Revision
SC24 WG9 & Web3D Meetings January 11-14, 2016 2
• NWIPs and Working Drafts
– ISO/IEC 19774 Humanoid Animation – Part 1: Architecture
– ISO/IEC 19774 Humanoid Animation – Part 2: Motion Capture
• Implementation Software and examples for H-Anim modeling and motion capture
animation
– H-Anim modeling
• H-Anim modeling guideline for general graphics tools
• Wrl-to-x3d H-Anim model converter for LOA1, LOA2, and LOA3 H-Anim
models
– H-Anim motion capture animation
• Mocap parameters to X3D interpolators converter
• H-Anim motion viewer for LOA1, LOA2, and LOA3 H-Anim models
• H-Anim motion editor with LOA1, LOA2, and LOA3 H-Anim models and
motion capture animation
ISO/IEC 19774 Humanoid Animation V1.0: 2006
• Foreword
• Introduction
• 1. Scope
• 2. Normative references
• 3. Terms and definitions
• 4. Concepts
• 5. Abstract data types
• 6. Object interfaces
• 7. Conformance
• Annexes
• A. Nominal body dimensions and levels of articulation
• B. Feature points for the human body
• C. VRML binding
• D. X3D binding
• E. Guidelines for H-Anim in VRML and X3D worlds
• Bibliography
SC24 WG9 & Web3D Meetings January 11-14, 2016 3
ISO/IEC 19775-1:2013 X3D Architecture and base components V3
Clause 26 Humanoid Animation (H-Anim) component
• 26.1 Introduction
– 26.1.1 Name
– 26.1.2 Overview
• 26.2 Concepts
• 26.3 Node reference
– 26.3.1 HAnimDisplacer
– 26.3.2 HAnimHumanoid
– 26.3.3 HAnimJoint
– 26.3.4 HAnimSegment
– 26.3.5 HAnimSite
• 26.4 Support levels
• Table 26.1 – Topics
• Table 26.3 – H-Anim component support levels
SC24 WG9 & Web3D Meetings January 11-14, 2016 4
Humanoid Animation V1.0 - Part 1: Architecture
SC24 WG9 & Web3D Meetings January 11-14, 2016 5
• Foreword
• Introduction
• 1. Scope
• 2. Normative references
• 3. Terms and definitions
• 4. Concepts
• 5. Abstract data types
• 6. Object interfaces
• 7. Conformance
• Annexes
• A. Nominal body dimensions and levels of articulation
• B. Feature points for the human body
• C. VRML binding
• D. X3D binding
• E. Guidelines for H-Anim in VRML and X3D worlds
• Bibliography
Humanoid Animation-Part 2: Motion Capture
SC24 WG9 & Web3D Meetings January 11-14, 2016 6
Contents
Foreword
Introduction
1 Scope
2 Normative references
3 Terms and definitions
4 Concepts
4.1 Introduction
4.2 Humanoid animation data
4.3 H-Anim modelling data for motion
capture animation
4.3.1 LOA 1 modelling
4.3.2 LOA 2 modelling
4.3.3 LOA 3 modelling
4.3.4 LOA joint mapping
4.4 H-Anim animation data for motion
capture animation
4.5 Motion definition parameters for
motion capture animation
5 H-Anim motion capture animation using X3D
interpolators
5.1 Definition of motion capture data for X3D
interpolators
5.2 Comparison of different LOA motion capture
animation using X3D interpolators
6 H-Anim motion definition using new nodes
based on X3D
6.1 Joint node expansion
6.2 Motion node
6.3 Numbers of channels and transformation
parameters
7 Schema expansion for H-Anim motion definition
8 Conformance
Annex
A procedure for generating an H-Anim motion
capture animation
Bibliography
Topics in H-Anim Architecture and Motion Capture
SC24 WG9 & Web3D Meetings January 11-14, 2016 7
1. H-Anim Architecture
‒ LOA 4 modelling (H-Anim hands and feet)
‒ Clauses to be updated in the 19774 for LOA 4
2. H-Anim Motion Capture
‒ Additional subjects to be included for the 19774-2 WD ?
‒ Motion capture animation (slide 23)
• Using X3D interpolators (BVH conversion is necessary, redundant )
• Using new Motion nodes (Schema update is necessary
‒ BVH to X3D interpolators conversion (inclusion or not in the WD)
‒ Joint node (object ?) update
• No of channels channels, sensor
‒ Motion node (object ?)
• No of frames, frame time, transformations
‒ Dick’s comments (slides 8, 9, and 10)
Review of comments about 19775-2 WD (1)
SC24 WG9 & Web3D Meetings January 11-14, 2016 8
1. References to X3D should mostly be removed or generalized to refer to any
presentation system. ISO/IEC 19774 is intended to be separate from X3D (but is itself
normatively referenced by ISO/IEC 19775-1). If desired, it may be possible to use
X3D to show some of the examples. However, example usage of ISO/IEC 19774-1
objects should be done using the expressions described in ISO/IEC 19774-1.
2. The schema should be removed. The one that is shown should be used as input to the
X3D encoding for Version 4.0 (or later). Since the schema assumes XML, it is not
possible to include it in ISO/IEC 19774-1 which does not have encodings.
3. This first WD of ISO/IEC 19774-2 seems to assume that there will only be three
levels of detail. Instead, we should assume an arbitrary number of levels of detail with
rules for mapping motion capture data (and channels) to joints in the desired level of
detail. Then, the example LOD mappings can be included provided that all those in
Part 1 are shown. We know that there will be at least four levels of detail from Part 1
(to include hands, feet, and maybe faces? we need to discuss this).
Review of comments about 19775-2 WD (2)
SC24 WG9 & Web3D Meetings January 11-14, 2016 9
4. In general, we should provide general guidelines for using the features described in
this specification and then show specific instances. In addition, if the humanoid is
non-human, the author will choose which joints exist and which channel is mapped to
which joint.
5. ISO/IEC 19774 supports both human humanoids and non-human humanoids which
may contain an arbitrary skeleton with an arbitrary number of limbs. Therefore, the
rules for mapping channels to joints should first be described in general fashion
followed by explicit mappings to human humanoids at particular levels of detail.
6. There is a tendency in your draft to refer to H-Anim objects as “nodes”. They should
always be called objects per Part 1. They are only nodes in X3D.
7. It is not clear where the Motion object is defined. Is it art of the HumanoidRoot object,
a Joint object, or elsewhere?
Review of comments about 19775-2 WD (3)
SC24 WG9 & Web3D Meetings January 11-14, 2016 10
8. If skinning is specified, it is not clear what impact that has on motion capture
animation. This should also be discussed somewhere.
9. The diagrams need better presentation. The diagrams should be big enough to span
part or most of the width of an A4 page with text that is not scaled (i.e., the diagram
itself should not be scaled in the specification but should be 1-to-1. I suggest creating
the diagrams using either Visio or Powerpoint (for org chart type diagrams). The
figures of the H-Anim figures are good and do not need to be improved. Where
possible, figures that contain only text should be inserted into the spec should use the
<pre></pre> HTML construct with a two-point smaller font size.
10. Please use constructs from the style sheet when showing lists, examples, notes, etc.
Then, global changes in the future will be easy. For example, the <p> construct has
already been specified to provide proper basic text presentation. When specific
changes are need (such as for figure and table captions), a <p class=”classname”>
construct is used. A quick study of the available classes can be determined by reading
the .css file.
Concepts (1)
SC24 WG9 & Web3D Meetings January 11-14, 2016 11
Figure 4.1 – Procedure of humanoid animation
The figure above needs to have better text.
In addition, it is not clear why a schema is required--schemas are part of the XML
encoding of a particular representation system and are not inherent herein.
Moreover, references to X3D should be removed as this is a standard not directed
specifically at X3D.
Table 4.2 – Representation details of each level of articulation
Basic positions of human LOA modelling are specified in ISO/IEC 19774-1.
Basic position of non-human humanoids is specified by the modeller.
This clause specifies how motion data can be applied to segments and joints for
motion capture animation.
4.4.2 General articulation of joints and segments
We need to add general concepts here so that non-human humanoids can use motion
capture data.
Annexes
SC24 WG9 & Web3D Meetings January 11-14, 2016 12
A Schema expansion for H-Anim motion definition
This annex should be deleted and used for instructions for the X3D vs. 4 XML
encoding.
Alternatively, the schema should only use H-Anim data types and objects.
H-Anim Hierarchy
(H-Anim 1.1)
SC24 WG9 & Web3D Meetings January 11-14, 2016 13
LOA3 (Level of Articulation 3) (1) http://h-anim.org/Specifications/H-Anim200x/ISO_IEC_FCD_19774/concepts.html#Hierarchy
SC24 WG9 & Web3D Meetings January 11-14, 2016 14
LOA3 (Level of Articulation 3) (2) http://h-anim.org/Specifications/H-Anim200x/ISO_IEC_FCD_19774/concepts.html#Hierarchy
SC24 WG9 & Web3D Meetings January 11-14, 2016 15
LOA 1
SC24 WG9 & Web3D Meetings January 11-14, 2016 16
LOA 2
SC24 WG9 & Web3D Meetings January 11-14, 2016 17
LOA 3
SC24 WG9 & Web3D Meetings January 11-14, 2016 18
LOA Comparison
LOA1
LOA2
LOA3
No. of joints
18
71
94
No. of segments
18
71
94
Representation
details
Basic joint
hierarchy
Increase of backbone
joints compared to LOA1,
addition of finger joints
Increase of backbone joints
compared to LOA2, 7 eye
joints in face
Motion capture
devices
MS Kinect
Part of LOA2 joints
available depending on
mocap devices (between
LOA1 and LOA2)
None
Animation levels
Simple humanoid
animation
Detailed animation such
as finger animation for
piano and guitar
compared to LOA1
Detailed animation upgraded
with facial animation such as
eyeball, and other eye joint
animation, Natural animation
than LOA2
January 11-14, 2016 SC24 WG9 & Web3D Meetings 19
l_shoulder
l_acromioclavicular
l_sternoclavicular
LOA2 Modeling
SC24 WG9 & Web3D Meetings January 11-14, 2016 20
3D Model
Data
Model
Data
Converter
H-Anim
X3D Model
Joint
Mapping
Motion
Capture
Data
Motion
Viewer
H-Anim
Motion File
Motion
Definition
Component
Motion
Parser
New
H-Anim
Schema
Mocap Animation
Keyframe Animation
Keyframe
Animator
Motion by
Interpolator
nodes
3D Scanner
Data
Motion
Algorithm
H-Anim Character Animation Generation
SC24 WG9 & Web3D Meetings January 11-14, 2016 21
Design using general tools (e.g. 3ds Max)
Model a segment with H-Anim joint name, define center of each joint, integrate
segments, and complete an H-Anim hunan figure
Design using 3D scanner data
Obtain scanner data , divide each segment,data, provide each segment joint name of
H-Anim, provide each center of joint, and integrate segments, and complete an H-
Anim human figure
Modeling an H-Anim Character
SC24 WG9 & Web3D Meetings January 11-14, 2016 22
Mocap and H-Anim LOA1 Joint
SC24 WG9 & Web3D Meetings January 11-14, 2016 23
Mocap (BVH) Joint Name
Hips
LeftHip
LeftKnee
LeftAnkle
RightHip
RightKnee
RightAnkle
Chest
LeftCollar
LeftShoulder
LeftElbow
LeftWrist
RightCollar
RightShoulder
RightElbow
RightWrist
Neck
Head
H-Anim Joint Name (19774:2006 v1.0)
HumanoidRoot
l_hip
l_knee
l_ankle
r_midtarsal
r_hip
r_knee
r_ankle
vl5
l_shoulder
l_elbow
l_wrist
l_midtarsal
r_shoulder
r_elbow
r_wrist
sacroiliac
Skullbase
typedef enum _NUI_SKELETON_POSITION_INDEX { NUI_SKELETON_POSITION_HIP_CENTER, NUI_SKELETON_POSITION_SPINE, NUI_SKELETON_POSITION_SHOULDER_CENTER, NUI_SKELETON_POSITION_HEAD, NUI_SKELETON_POSITION_SHOULDER_LEFT, NUI_SKELETON_POSITION_ELBOW_LEFT, NUI_SKELETON_POSITION_WRIST_LEFT, NUI_SKELETON_POSITION_HAND_LEFT, NUI_SKELETON_POSITION_SHOULDER_RIGHT, NUI_SKELETON_POSITION_ELBOW_RIGHT, NUI_SKELETON_POSITION_WRIST_RIGHT, NUI_SKELETON_POSITION_HAND_RIGHT, NUI_SKELETON_POSITION_HIP_LEFT, NUI_SKELETON_POSITION_KNEE_LEFT, NUI_SKELETON_POSITION_ANKLE_LEFT, NUI_SKELETON_POSITION_FOOT_LEFT, NUI_SKELETON_POSITION_HIP_RIGHT, NUI_SKELETON_POSITION_KNEE_RIGHT, NUI_SKELETON_POSITION_ANKLE_RIGHT, NUI_SKELETON_POSITION_FOOT_RIGHT, NUI_SKELETON_POSITION_COUNT } NUI_SKELETON_POSITION_INDEX;
Constant Description
NUI_SKELETON_POSITION_HIP_CENTER Center, between hips
NUI_SKELETON_POSITION_SPINE Spine
NUI_SKELETON_POSITION_SHOULDER_CENTER
Center, between shoulders
NUI_SKELETON_POSITION_HEAD Head
NUI_SKELETON_POSITION_SHOULDER_LEFT
Left shoulder
NUI_SKELETON_POSITION_ELBOW_LEFT Left elbow
NUI_SKELETON_POSITION_WRIST_LEFT Left wrist
NUI_SKELETON_POSITION_HAND_LEFT Left hand
NUI_SKELETON_POSITION_SHOULDER_RIGHT
Right shoulder
NUI_SKELETON_POSITION_ELBOW_RIGHT Right elbow
NUI_SKELETON_POSITION_WRIST_RIGHT Right wrist
NUI_SKELETON_POSITION_HAND_RIGHT Right hand
NUI_SKELETON_POSITION_HIP_LEFT Left hip
NUI_SKELETON_POSITION_KNEE_LEFT Left Knee
NUI_SKELETON_POSITION_ANKLE_LEFT Left ankle
NUI_SKELETON_POSITION_FOOT_LEFT Left foot
NUI_SKELETON_POSITION_HIP_RIGHT Right hip
NUI_SKELETON_POSITION_KNEE_RIGHT Right knee
NUI_SKELETON_POSITION_ANKLE_RIGHT Right ankle
NUI_SKELETON_POSITION_FOOT_RIGHT Right foot
NUI_SKELETON_POSITION_COUNT Used as an index to terminate a loop. Not used as a position index.
Kinect Skeleton
SC24 WG9 & Web3D Meetings January 11-14, 2016 24
Definition using X3D Interpolator (1)
Conversion of motion capture data (e.g. bvh) to X3D Interpolator
Bvh to X3D interpolator conversion
Requires conversion between motion capture rotation angles such as euler angles
and SFRotation angles
Requires conversion between the orders of parameter values
Definition of a Motion Data Component (2)
Joint node (update)
Define additional fields for motion capture parameters
Motion node (new)
Define motion captured data for an H-Anim character model
Define the motion node after adjusting the center of each joint to the H-Anim
character model
Motion Definition Using Motion Capture Data
January 11-14, 2016 SC24 WG9 & Web3D Meetings 25
Procedure
Step1: Joint mapping between an H-Anim model and a motion capture file
Set a sensor flag in each H-Anim joint for denoting a motion capture sensor
at a motion capture joint corresponding to the H-Anim model
Step2: Conversion of motion parameters from a motion capture file
1. Compute cycle time
2. Compute interpolator key array
3. Compute time sensor key array
4. Compute key value array
5. Convert each motion capture angle to X3D SFRotation
Process different rotation orders between X3D interpolators and the
motion capture file:
BVH: order specified
X3D: x-y-z order
Motion capture animation using X3D interpolators
January 11-14, 2016 SC24 WG9 & Web3D Meetings 26
Mocap data example: bvh (1)
HIERARCHY
ROOT Hips
{
OFFSET 0.000000 0.000000 0.000000
CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation
JOINT Chest
{
OFFSET 0.000000 5.613096 0.000000
CHANNELS 3 Zrotation Xrotation Yrotation
JOINT LeftCollar
{
OFFSET 0.003804 10.354579 1.025227
CHANNELS 3 Zrotation Xrotation Yrotation
JOINT LeftShoulder
{
OFFSET 3.922637 0.000000 0.000000
CHANNELS 3 Zrotation Xrotation Yrotation
JOINT LeftElbow
…
SC24 WG9 & Web3D Meetings January 11-14, 2016 27
Mocap data example: bvh (2)
MOTION
Frames: 482
Frame Time: 0.016667
1.662 31.427 60.304 -1.249 -4.859 -3.582 4.463 1.354 0.075 -13.732 3.052 3.999 95.677 1.705 -1.512 5.541 -3.491
0.339 1.259 -3.022 1.790 6.765 2.405 -4.446 -91.027 -7.187 4.910 -3.633 0.867 0.043 -2.879 0.120 -5.688 -
1.132 -1.858 0.809 -2.969 -8.472 1.461 -1.304 3.919 -2.045 1.054 9.006 -0.191 2.695 -1.341 -0.615 0.361 4.452
4.756 0.484 8.095 0.193 -6.340 -0.815 1.224
1.659 31.427 60.307 -1.268 -4.835 -3.588 4.487 1.352 0.080 -13.802 3.059 3.999 95.651 1.737 -1.609 5.541 -3.521
0.340 1.298 -3.030 1.974 6.795 2.410 -4.418 -90.999 -7.145 4.917 -3.633 0.825 0.043 -2.862 0.151 -5.736 -
1.141 -1.863 0.806 -2.988 -8.482 1.455 -1.283 3.890 -2.157 1.055 9.042 -0.193 2.781 -1.350 -0.638 0.381 4.430
4.692 0.484 8.102 0.194 -6.266 -0.819 1.206
1.657 31.428 60.310 -1.287 -4.811 -3.594 4.512 1.349 0.085 -13.872 3.066 3.999 95.624 1.769 -1.707 5.542 -3.550
0.341 1.337 -3.038 2.159 6.824 2.417 -4.389 -90.970 -7.101 4.924 -3.633 0.783 0.043 -2.845 0.182 -5.783 -
1.150 -1.869 0.804 -3.006 -8.492 1.449 -1.262 3.860 -2.270 1.055 9.077 -0.196 2.867 -1.358 -0.661 0.401 4.408
4.628 0.484 8.108 0.194 -6.189 -0.824 1.187
1.654 31.428 60.313 -1.306 -4.787 -3.599 4.536 1.347 0.090 -13.942 3.073 3.999 95.596 1.801 -1.804 5.542 -3.580
0.342 1.376 -3.046 2.343 6.853 2.423 -4.360 -90.940 -7.056 4.931 -3.633 0.741 0.042 -2.827 0.212 -5.831 -
1.159 -1.874 0.801 -3.024 -8.503 1.442 -1.241 3.831 -2.383 1.056 9.113 -0.198 2.953 -1.367 -0.684 0.420 4.386
4.565 0.484 8.113 0.194 -6.107 -0.831 1.167
…
SC24 WG9 & Web3D Meetings January 11-14, 2016 28
Step 1: Process joint mapping of motion capture joints to the joints of an H-Anim model, and set a
motion sensor flag at each H-Anim model joint. The following shows a section of an H-Anim model. A
motion flag at each H-Anim joint references if there is a corresponding joint in the motion capture file.
<HAnimHumanoid DEF="hanim_HAnim" info="humanoidVersion=2.0" containerField="skeleton" name="HAnim"
scale="0.0225 0.0225 0.0225" version="2.0">
<HAnimJoint DEF="hanim_HumanoidRoot" center="0.000000 44.070000 -1.034000" name="HumanoidRoot"
containerField="children">
<HAnimSegment DEF="hanim_sacrum" name="sacrum" containerField="children">
<Transform translation="0.000000 44.070000 -1.034000" rotation="0.000000 0.000000 0.000000 0.000000"
scale="1.000000 1.000000 1.000000" scaleOrientation="0.000000 0.000000 0.000000 0.000000">
<Shape>
<Appearance>
<Material diffuseColor="0.588000 0.588000 0.588000"/>
<ImageTexture DEF="ChulTextureAtlas" url="Chul.png"/>
</Appearance>
<IndexedFaceSet creaseAngle="3.14159" coordIndex="0, 1, 2, -1, 0, 2, 3, -1, 0, 3, 4, -1, 0, 4, 5, -1, 0, 5, 6, -1, 0, 6, 7, -1, 0,
7, 8, -1, 0, 8, 9, -1, 0, 9, 10, -1,
…
Motion Definition Using X3D Interpolator (1)
January 11-14, 2016 SC24 WG9 & Web3D Meetings 29
Step 2: Define motion parameter values with X3D interpolators using a motion capture file. TimeSensor,
PositionalInterpolator, and OrientationalInterpolator nodes, and Route statements are used.
<Group DEF='WalkAnimation'>
<TimeSensor DEF='WalkTimer' cycleInterval='1.73' enabled='true' loop='true'/>
<OrientationInterpolator DEF='Walk_r_metatarsalPitch' key='0 0.2 0.4 0.6 0.7 1' keyValue='0 0 1 0 0 0 1 0 0 0 1 0 0 0
1 0 0 0 1 0 0 0 1 0'/>
<OrientationInterpolator DEF='Walk_r_ankleRotInterp' key='0 0.125 0.2083 0.375 0.4583 0.5 0.6667 0.75 0.7917
0.9167 1' keyValue='0 0 1 0 -1 0 0 0.3533 -1 0 0 0.1072 1 0 0 0.2612 1 0 0 0.1268 -1 0 0 0.01793 -1 0 0 0.05824 -1 0 0
0.2398 -1 0 0 0.35 -1 0 0 0.3322 0 0 1 0'/>
<OrientationInterpolator DEF='Walk_r_kneeRotInterp' key='0 0.125 0.2083 0.2917 0.375 0.5 0.6667 0.7917 0.9167 1'
keyValue='1 0 0 0.8573 1 0 0 0.8926 1 0 0 0.5351 1 0 0 0.1756 1 0 0 0.1194 1 0 0 0.3153 1 0 0 0.09354 1 0 0 0.08558 1
0 0 0.2475 1 0 0 0.8573'/>
...
<ROUTE fromField='fraction_changed' fromNode='WalkTimer' toField='set_fraction'
toNode='Walk_r_ankleRotInterp'/>
<ROUTE fromField='fraction_changed' fromNode='WalkTimer' toField='set_fraction'
toNode='Walk_r_kneeRotInterp'/>
...
<ROUTE fromField='value_changed' fromNode='Walk_r_ankleRotInterp' toField='rotation' toNode='hanim_r_ankle'/>
<ROUTE fromField='value_changed' fromNode='Walk_r_kneeRotInterp' toField='rotation' toNode='hanim_r_knee'/>
...
</Group>
Motion Definition Using X3D Interpolator (2)
January 11-14, 2016 SC24 WG9 & Web3D Meetings 30
Step 1: Process joint mapping of motion capture joints to the joints of an H-Anim model, and set a
motion sensor flag at each H-Anim model joint. The following shows a section of an H-Anim model. A
motion flag at each H-Anim joint references if there is a corresponding joint in the motion capture file.
In addition, number of channels and channels are specified in each joint node of the H-Anim model
using a motion capture file.
<HAnimHumanoid DEF="hanim_HAnim" info="humanoidVersion=2.0" containerField="skeleton" name="HAnim"
scale="0.0225 0.0225 0.0225" version="2.0">
<HAnimJoint DEF="hanim_HumanoidRoot" center="0.000000 44.070000 -1.034000" name="HumanoidRoot"
containerField="children">
<HAnimSegment DEF="hanim_sacrum" name="sacrum" containerField="children">
<Transform translation="0.000000 44.070000 -1.034000" rotation="0.000000 0.000000 0.000000 0.000000"
scale="1.000000 1.000000 1.000000" scaleOrientation="0.000000 0.000000 0.000000 0.000000">
<Shape>
<Appearance>
<Material diffuseColor="0.588000 0.588000 0.588000"/>
<ImageTexture DEF="ChulTextureAtlas" url="Chul.png"/>
</Appearance>
<IndexedFaceSet creaseAngle="3.14159" coordIndex="0, 1, 2, -1, 0, 2, 3, -1, 0, 3, 4, -1, 0, 4, 5, -1, 0, 5, 6, -1, 0, 6, 7, -1, 0,
7, 8, -1, 0, 8, 9, -1, 0, 9, 10, -1,
Motion Definition Using new Motion nodes (1)
January 11-14, 2016 SC24 WG9 & Web3D Meetings 31
Interface Joint {
// the same as the existing joint node
float[3] bboxCenter 0 0 0
float[3] bboxSize -1 -1 -1
float[3] center 0 0 0
sequence<Object> children []
sequence<Object> displacers []
sequence<float[3]> llimit []
float[4] limitOrientation 0 0 1 0
string name ""
float[4] rotation 0 0 1 0
float[3] scale 1 1 1
float[4] scaleOrientation 0 0 1 0
float[3] translation 0 0 0
sequence<float[3]> ulimit []
// define additional fields
int[2] ChannelsNumber
sequence<string> Channels
bool Sensor
}
Fields for receiving motion capture data
ChannelsNumber Channels Sensor
Joint Node Update (1)
SC24 WG9 & Web3D Meetings January 11-14, 2016 32
Define additional fields: Channels, ChannelsNumber, Sensor (new fields)
ChannelsNumber: Number of channels at a joint
Channels: Identifiers for channels
Sensor: Flag for joint mapping with motion capture data
Example
Interface Joint {
…
int[2] ChannelsNumber
sequence<string> Channels
bool Sensor
}
Joint {
…
ChannelsNumber [ 1, 3 ]
Channels “ Xrotate Yrotate Zrotate”
Sensor [false, true]
}
Joint Node Update (2)
SC24 WG9 & Web3D Meetings January 11-14, 2016 33
Step 2: Define motion parameter values with Motion nodes using a motion capture file. Frame number,
Frame time, and Transformation parameter values are defined from the motion capture file.
Frames: 482
Frame Time: 0.016667
1.662 31.427 60.304 -1.249 -4.859 -3.582 4.463 1.354 0.075 -13.732 3.052 3.999 95.677 1.705 -1.512 5.541 -3.491
0.339 1.259 -3.022 1.790 6.765 2.405 -4.446 -91.027 -7.187 4.910 -3.633 0.867 0.043 -2.879 0.120 -5.688 -
1.132 -1.858 0.809 -2.969 -8.472 1.461 -1.304 3.919 -2.045 1.054 9.006 -0.191 2.695 -1.341 -0.615 0.361 4.452
4.756 0.484 8.095 0.193 -6.340 -0.815 1.224
1.659 31.427 60.307 -1.268 -4.835 -3.588 4.487 1.352 0.080 -13.802 3.059 3.999 95.651 1.737 -1.609 5.541 -3.521
0.340 1.298 -3.030 1.974 6.795 2.410 -4.418 -90.999 -7.145 4.917 -3.633 0.825 0.043 -2.862 0.151 -5.736 -
1.141 -1.863 0.806 -2.988 -8.482 1.455 -1.283 3.890 -2.157 1.055 9.042 -0.193 2.781 -1.350 -0.638 0.381 4.430
4.692 0.484 8.102 0.194 -6.266 -0.819 1.206
……
Motion Definition Using new Motion nodes (2)
January 11-14, 2016 SC24 WG9 & Web3D Meetings 34
Definition of Motion Node (a new node)
Define fields: Frames, FrameTime, Transformation Channels
Frames: Number of frames for an animation sequence
FrameTime: Specifies a sampling rate
Transformation: Transformation values of a joint for each frame
Example
Interface Motion {
int Frames
float FrameTime
sequence<float> Transformation
}
Motion {
Frames 601
Frametime 0.033333
transformation [ 11.623, 31.312, 64.121, -0.700, -4.023, ……
11.616, 31.313, 64.107, -0.696, -3.954, ……
…… ]
}
Motion Node
SC24 WG9 & Web3D Meetings January 11-14, 2016 35
<Scene> <NavigationInfo speed='1.5' type='"EXAMINE" "ANY"'/> <Viewpoint centerOfRotation='0 1 0' description='Jin' position='0 1 3'/> <HAnimHumanoid DEF='Jin' info='"humanoidVersion=2.0"' name='Jin' scale='0.0225 0.0225 0.0225' version='2.0'> <HAnimJoint DEF='Jin_HumanoidRoot' containerField='skeleton' name='HumanoidRoot'> <HAnimJoint DEF='Jin_sacroiliac' center='0.000000 35.830002 -0.707600' name='sacroiliac‘ ChannelsNumber=‘6’ Channels=“Xposition, Yposition, Zposition, Zrotation, Xrotation, Yrotation“ Sensor=‘true’> <HAnimSegment DEF='Jin_pelvis' name='pelvis‘containerField="children"> <Transform translation='0.000000 35.830002 -0.707600'> <Shape> <Appearance> <Material diffuseColor='0.588000 0.588000 0.588000'/> <ImageTexture DEF='JinTextureAtlas' url='"images/Jin.bmp" "images/Jin.png" </Appearance> <IndexedFaceSet coordIndex='0 1 2 -1 0 2 3 -1 0 3 4 -1 0 4 5 -1 … <Coordinate point='0.0000 5.4970 0.1424 0.0000 4.7610 -2.8250 … <TextureCoordinate point='0.6211 0.5754 0.7851 0.5720 0.7614 …
Joint node update example
SC24 WG9 & Web3D Meetings January 11-14, 2016 36
0.0221,0.4411 0.0186,0.5487 0.0186,0.4637 0.1714,0.4708 0.0186"/> </IndexedFaceSet> </Shape> </Transform> </HAnimSegment> </HAnimJoint> </HAnimJoint> </HAnimJoint> </HAnimJoint> … <HAnimMotion Frames ="482" FrameTime = "0.016667"> <Frame Transformation = " 2.1834 41.2869 79.2237 -1.2490 -4.8590 -3.5820 4.4630 1.3540 0.0750 -13.7320 3.0520 3.9990 95.6770 1.7050 -1.5120 5.5410 -3.4910 0.3390 1.2590 -3.0220 … … 5.8060 -2.2160 1.0610 9.5740 -0.2270 0.2010 -3.8550 0.3020 -2.8020 7.0420 4.4960 0.4940 6.9090 0.1390 -8.4900 -0.2720 1.8800 "/> </HAnimMotion> </Scene> </X3D>
Motion node example
SC24 WG9 & Web3D Meetings January 11-14, 2016 37
A Procedure for generating H-Anim motion capture animation
SC24 WG9 & Web3D Meetings January 11-14, 2016 38
H-Anim
modeling
definition
part
H-Anim
motion
definition
part
H-Anim model and motion
Data format
(X3D Interpolators OR
Motion node def)
H-Anim
modeling
using a 3D
tool
LOA modeling
guideline
General 3D tools WRL X3D
(H-Anim)
WRL-to-
X3D H-
Anim
conversion
H-Anim model
Data format
LOA1, LOA2, LOA3
H-Anim
modeling
directly
Motion capture data
H-Anim Character Modeling Using a General Graphics Tool (1)
All joints of an H-Anim figure must be represented as a tree hierarchy starting with the
HumanoidRoot joint. Each joint may or may not have a segment. The figure shows an LOA1
example hierarchy used for the H-Anim character. 18 joints and 18 segments are used.
SC24 WG9 & Web3D Meetings January 11-14, 2016 39
Humanoid Root
sacroiliac vl1
L_hip r_hip l_acromioclavicular r_acromioclavicular skullbase
L_knee r_knee Llshoukder r_shoulder
L_ankle r-ankle l_elbow r_elbow
l_wrist r_wrist
Step 1: Units are set up before modeling of the H-Anim character.
H-Anim Character Modeling Using a General Graphics Tool (2)
SC24 WG9 & Web3D Meetings January 11-14, 2016 40
H-Anim Character Modeling Using a General Graphics Tool (3)
Step 2: Uniquely identify each segment according to the naming scheme of H-Anim.
SC24 WG9 & Web3D Meetings January 11-14, 2016 41
skullbase
r_wrist
r_shoulder
vl5
sacroiliac
r_hip
r_knee
r_ankle
r_elbow
HumanoidRoot
r_midtarsal
l_shoulder
l_elbow
l_wrist
l_hip
l_knee
l_ankle
l_midtarsal
When a segment is designed, its H-Anim joint name must be provided as defined in the
hierarchy of an LOA, i.e. LOA1 in this example
H-Anim Character Modeling Using a General Graphics Tool (4)
SC24 WG9 & Web3D Meetings January 11-14, 2016 42
Step 3. Each segment with its joint is modelled. The figure shows a segment with an H-Anim
skullbase joint. All segments are modelled in this way.
H-Anim Character Modeling Using a General Graphics Tool (5)
SC24 WG9 & Web3D Meetings January 11-14, 2016 43
Step 4:. Integrate all the segments to form a complete character.
H-Anim Character Modeling Using a General Graphics Tool (6)
SC24 WG9 & Web3D Meetings January 11-14, 2016 44
In the front view, the character is looking forward and the origin of the coordinate system is
located between the two feet
H-Anim Character Modeling Using a General Graphics Tool (7)
SC24 WG9 & Web3D Meetings January 11-14, 2016 45
Step 5.: For each segment, a pivot point, which is initially located at the center of an
object, must be moved to the H-Anim joint center. The pivot point becomes the center
value for each H-Anim joint. In the figure, an arrow denotes each pivot point at a joint.
H-Anim Character Modeling Using a General Graphics Tool (8)
SC24 WG9 & Web3D Meetings January 11-14, 2016 46
This figure shows the pivot point of the H-Anim skullbase joint
H-Anim Character Modeling Using a General Graphics Tool (9)
SC24 WG9 & Web3D Meetings January 11-14, 2016 47
The figure shows the pivot points of the H-Anim vc4 and vl1 joints.
H-Anim Character Modeling Using a General Graphics Tool (10)
SC24 WG9 & Web3D Meetings January 11-14, 2016 48
Step 6: If necessary, the H-Anim figure can be scaled, at this stage considering real length.
Otherwise, the real length of each segment can be taken into consideration at the beginning,
when each segment is modelled, if exact length is required. In this example, the character was
initially designed considering only the length ratio of each segment then, at this stage, the
segments were scaled according to real length (e.g. the height of the character).
H-Anim Character Modeling Using a General Graphics Tool (11)
SC24 WG9 & Web3D Meetings January 11-14, 2016 49
Step 7: In the final stage, all Transform nodes are initialized after being applied to all segments.
Now, the H-Anim character has final modelling coordinates for his geometry.
H-Anim Character Modeling Using a General Graphics Tool (12)
SC24 WG9 & Web3D Meetings January 11-14, 2016 50
Step 8: Store the designed H-Anim character as a wrl file if the general graphics tool has this
capability, which is usually the case. The wrl file can be converted to an x3d H-Anim file
using a converter program.
H-Anim Character Modeling Using a General Graphics Tool (13)
WRL X3D H-Anim
SC24 WG9 & Web3D Meetings January 11-14, 2016 51
Conversion Program from WRL to X3D H-Anim (1)
WRL File
Definition of H-Anim
structure with joint hierarchy, joint center, and segment
geometry
X3D H-Anim File
DEF skullbase Transform { … Shape { … } DEF vc4 Transform { … Shape { … } DEF vl1 Transform { … Shape { … } DEF humanoidRoot Transform { … Shape { … }
<Scene> … <HAnimHumanoid DEF=“Ru” …> <HAnimJoint DEF=“Ru_HumanoidRoot” center= … <HAnimJoint DEF=“Ru_sacroiliac” center=… <HAnimSegment DEF=“Ru_pelvis”, … <Transform …> <Shape> … </Shape> </Transform> </HAnimSegment> <HAnimJoint DEF=“Ru_l_hip” center=…
SC24 WG9 & Web3D Meetings January 11-14, 2016 52
Conversion Program from WRL to X3D H-Anim (2)
DEF skullbase Transform { translation 0 58.72 -1.829 children [ Shape { appearance Appearance { material Material { diffuseColor 0.588 0.588 0.588 ambientIntensity 1.0 specularColor 0 0 0 shininess 0.145 transparency 0 } texture ImageTexture { url "Ru.bmp" } } geometry DEF skullbase-FACES IndexedFaceSet { ccw TRUE solid TRUE coord DEF skullbase-COORD Coordinate { point [ 1.079 -0.6566 1.833, 5.073 7.007 2.332, 5.209
5.705 0.1804, ... coordIndex [ 56, 57, 19, -1, 19, 8, 56, -1, 7, 6, 18, -1, 18, 16, 7, -
1, ... texCoordIndex [ 1, 2, 3, -1, 3, 0, 1, -1, 4, 5, 6, -1, 6, 7, 4, -1, 8, 9, 6, -
1, ...
WRL
DEF vc4 Transform { translation 0 56 -1.959 children [ Shape { appearance Appearance { material Material { diffuseColor 0.588 0.588 0.588 ambientIntensity 1.0 specularColor 0 0 0 shininess 0.145 transparency 0 } texture ImageTexture { url "Ru.bmp" } } geometry DEF vc4-FACES IndexedFaceSet {
SC24 WG9 & Web3D Meetings January 11-14, 2016 53
Conversion Program from WRL to X3D H-Anim (3) <Scene> ... <HAnimHumanoid DEF=“Ru" info="humanoidVersion=2.0" name=“Ru" scale="0.0225 0.0225 0.0225" version="2.0"> <HAnimJoint DEF=“Ru_HumanoidRoot" center="0.0 0.0 0.0" containerField="skeleton" name="HumanoidRoot"> <HAnimJoint DEF=“Ru_sacroiliac" center="0.000000 35.950001 -2.058000" name="sacroiliac" containerField="children"> <HAnimSegment DEF=“Ru_pelvis" name="pelvis" containerField="children"> <Transform translation="0.000000 35.950001 -2.058000" rotation="0.000000 0.000000 0.000000 0.000000" scale="1.000000 1.000000 1.000000"
scaleOrientation="0.000000 0.000000 0.000000 0.000000"> <Shape> <Appearance> <Material diffuseColor="0.588000 0.588000 0.588000"/> <ImageTexture url="Ru.bmp"/> </Appearance> <IndexedFaceSet coordIndex="18, 0, 6, -1, 6, 8, 18, -1, 30, 31, 10, -1, 10, 12, ... <Coordinate point="5.1340 1.8530 -2.3070, 5.5760 1.8490 2.3010, 1.8320 1.8550 ... <TextureCoordinate point="0.9018 0.4063,0.9014 0.4179,0.8749 0.4137,0.8692 ... </IndexedFaceSet> </Shape> </Transform> </HAnimSegment> <HAnimJoint DEF=“Ru_l_hip" center="3.602000 34.000000 -1.967000" name="l_hip" containerField="children"> <HAnimSegment DEF=“Ru_l_thigh" name="l_thigh" containerField="children"> <Transform translation="3.602000 34.000000 -1.967000" rotation="0.000000 0.000000 0.000000 0.000000" scale="1.000000 1.000000 1.000000"
scaleOrientation="0.000000 0.000000 0.000000 0.000000"> <Shape> <Appearance> <Material diffuseColor="0.588000 0.588000 0.588000"/> <ImageTexture url="Ru.bmp"/> </Appearance> <IndexedFaceSet coordIndex="47, 16, 2, -1, 2, 48, 47, -1, 1, 18, 15, -1, 15, 0, 1, ... <Coordinate point="2.8500 0.1840 -0.0877, 2.0230 -0.0169 -2.6250, 0.2938 - <TextureCoordinate point="0.6973 0.5621,0.6732 0.5606,0.6756 0.5519,0.6995 … </IndexedFaceSet> </Shape> </Transform> </HAnimSegment> <HAnimJoint DEF=“Ru_l_knee" center="3.931000 18.500000 -1.935000" name="l_knee" containerField="children"> <HAnimSegment DEF=“Ru_l_calf" name="l_calf" containerField="children"> <Transform translation="3.931000 18.500000 -1.935000" rotation="0.000000 0.000000 0.000000 0.000000" scale="1.000000 1.000000 1.000000"
scaleOrientation="0.000000 0.000000 0.000000 0.000000">
X3D H-Anim
SC24 WG9 & Web3D Meetings January 11-14, 2016 54
H-Anim modeling file
<Scene> <NavigationInfo speed='1.5' type='"EXAMINE" "ANY"'/> <Viewpoint centerOfRotation='0 1 0' description='Jin' position='0 1 3'/> <HAnimHumanoid DEF='Jin' info='"humanoidVersion=2.0"' name='Jin' scale='0.0225 0.0225 0.0225' version='2.0'> <HAnimJoint DEF='Jin_HumanoidRoot' containerField='skeleton' name='HumanoidRoot'> <HAnimJoint DEF='Jin_sacroiliac' center='0.000000 35.830002 -0.707600' name='sacroiliac'> <HAnimSegment DEF='Jin_pelvis' name='pelvis'> <Transform translation='0.000000 35.830002 -0.707600'> <Shape> <Appearance> <Material diffuseColor='0.588000 0.588000 0.588000'/> <ImageTexture DEF='JinTextureAtlas' url='"images/Jin.bmp" "images/Jin.png" "http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/images/Jin.bmp" "http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/images/Jin.png"'/> </Appearance> <IndexedFaceSet coordIndex='0 1 2 -1 0 2 3 -1 0 3 4 -1 0 4 5 -1 0 5 ... <Coordinate point='0.0000 5.4970 0.1424 0.0000 4.7610 -2.8250 -2.2830 4.7610 -2.4280 -3.9540 4.7610 -1.9480 -4.5660 4.7610 0.1424 -3.9540
H-Anim Character Modeling File (X3D H-Anim)
SC24 WG9 & Web3D Meetings January 11-14, 2016 55
1.Jin 2.Chul 3.Hyun 4.Young 5.Ju 6.Ga
7.No 8.Da 9.Ru 10.Mi 11.Min 12.Sun
LOA1 H-Anim Characters (X3D H-Anim)
SC24 WG9 & Web3D Meetings January 11-14, 2016 56
LOA2 H-Anim Characters (X3D H-Anim)
1.Jin 2.Chul 4.Young 11.Min
SC24 WG9 & Web3D Meetings January 11-14, 2016 57
LOA3 H-Anim Characters (X3D H-Anim)
1.Jin
SC24 WG9 & Web3D Meetings January 11-14, 2016 58
H-Anim Behavior Animation Using X3D Interpolators
SC24 WG9 & Web3D Meetings January 11-14, 2016 59
X3D H-Anim
Model
BVH MOCAP
Behaviors
BVH Converter
H-Anim Editor
C++ JAVA
X3D Edit
Interpolator
Generator
Interpolator
Generator
Jin.x3d Dance.bvh
Inspect
Edit
View
Inspect
Edit
View BVH
Controller
Coordinate Interpolators
Orientation Interpolators
EXPORT statements
Script Controller
Inline “Jin.x3d”
Inline “Dance.x3d”
ROUTE statements
Behavior file
Composed
Character behavior file
EXPORT EXPORT
Load Load
Load
Jin dancing
Jin dancing
H-Anim
Motion Capture
using
X3D Interpolators
BVH
Converter
SC24 WG9 & Web3D Meetings January 11-14, 2016 60
In the H-Anim motion viewer:
1. Read a motion capture file and apply it to the H-Anim character model. Our H-
Anim motion viewer includes the function of motion retargeting which applies the
motion of each joint of the motion captured model to the motion of a corresponding
joint of the designed H-Anim character model. The motion viewer displays an
animation sequence from the H-Anim character model and a motion capture file.
2. The motion viewer can save the animation sequence as an animation file which has
the format of the H-Anim motion data definition. The following figure shows the
interface to save an H-Anim animation file after generating an animation sequence.
H-Anim Motion Capture Using Motion Node (1)
SC24 WG9 & Web3D Meetings January 11-14, 2016 61
H-Anim Character Animation Using Motion Capture (2)
SC24 WG9 & Web3D Meetings January 11-14, 2016 62
Hanim.x3d
Motion.bvh
H-Anim Viewer NewHanim.hanim
Save
H-Anim Motion Viewer
Read an H-Anim character model and motion captured data
Map joints between the mocap character and the H-Anim character
Apply motion parameters at each joint of the mocap character to its
corresponding joint of the H-Anim character
Generate and display the motion captured animation for the H-Anim character
Generate an H-Anim animation file including the H-Anim character model
with the motion captured data
Load
H-Anim Animation File Generation
SC24 WG9 & Web3D Meetings January 11-14, 2016 63
LOA2 H-Anim Motion Capture Animation
SC24 WG9 & Web3D Meetings January 11-14, 2016 64
H-Anim Motion Viewer
SC24 WG9 & Web3D Meetings January 11-14, 2016 65
H-Anim Motion Editor
SC24 WG9 & Web3D Meetings January 11-14, 2016 66
H-Anim Editor
• Objectives
• Generate human animation using motion capture data
• Edit motion capture animation
• Edit model hierarchy
• Edit joint
• Edit segment
• Edit motion capture data
• Joint mapping between an H-Anim model and motion
capture data
SC24 WG9 & Web3D Meetings January 11-14, 2016 67
Joint Mapping Tool
① ②
③ ④ ⑤
1. H-Anim hierarchy
2. BVH hierarchy 3. BVH load 4. H-Anim save 5. Close
SC24 WG9 & Web3D Meetings January 11-14, 2016 68
H-Anim character animation & music 3D
• http://www.youtube.com/watch?v=hFz3WU3JIU0
• H-Anim Music 3D Animation - Sundays
• http://www.youtube.com/watch?v=0gsRNN9gfSM
• H-Anim Music 3D Animation - Sundays
• http://www.youtube.com/watch?v=TiUskQ0DTqY
• H-Anim Music 3D Animation - Travels
• http://www.youtube.com/watch?v=MlGwSHI8aGA
• H-Anim Music 3D Animation - The Present
• http://www.youtube.com/watch?v=R_uYm-pyJxU
• H-Anim LOA2 Music 3D Animation - Trevels
SC24 WG9 & Web3D Meetings January 11-14, 2016 69
In-progress and future work
• LOA1, LOA2, LOA3 model converter: wrl-to-x3d hanim
• LOA1, LOA2, LOA3 Joint mapping tool
• LOA1, LOA2, LOA3 Bvh to X3D interpolator conversion
• LOA1, LOA2, LOA3 H-Anim motion viewer
• LOA1, LOA2, LOA3 H-Anim motion editor
• LOA4 modeling and animation
SC24 WG9 & Web3D Meetings January 11-14, 2016 70