Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy...

86
  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    284
  • download

    0

Transcript of Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy...

Page 1: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:
Page 2: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Animation From Motion Animation From Motion CaptureCapture

Animation From Motion Animation From Motion CaptureCapture

Motion Capture Assisted Motion Capture Assisted Animation: Texturing and Animation: Texturing and

SynthesisSynthesis

Kathy PullenKathy Pullen

Chris BreglerChris Bregler

Motion Capture Assisted Motion Capture Assisted Animation: Texturing and Animation: Texturing and

SynthesisSynthesis

Kathy PullenKathy Pullen

Chris BreglerChris Bregler

Page 3: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Related Work:Related Work:Animation With StyleAnimation With StyleRelated Work:Related Work:Animation With StyleAnimation With Style

M. Brand and A. Hertzmann. Proc. SIGGRAPH 2000, pp 183-192M. Brand and A. Hertzmann. Proc. SIGGRAPH 2000, pp 183-192

D. Chi, M. Costa, L. Zhao and N. Badler. Proc. SIGGRAPH 2000, pp 173-182D. Chi, M. Costa, L. Zhao and N. Badler. Proc. SIGGRAPH 2000, pp 173-182

M. Gleicher. 1997 Symposium on Interactive 3D Graphics, pp 139-148M. Gleicher. 1997 Symposium on Interactive 3D Graphics, pp 139-148

J. Hodgins, W. L. Wooten, D. C. Broogan and J. F. O’Brien. AnimatingJ. Hodgins, W. L. Wooten, D. C. Broogan and J. F. O’Brien. AnimatingHuman athletics. Proc. SIGGRAPH 1995, PP 229-238Human athletics. Proc. SIGGRAPH 1995, PP 229-238

K. Perlin and A. Goldberg. Proc. SIGGRAPH 1996, PP 205-216K. Perlin and A. Goldberg. Proc. SIGGRAPH 1996, PP 205-216

Z. Popovic and A. Witkin. Proc. SIGGRAPH 1999, pp 159-168Z. Popovic and A. Witkin. Proc. SIGGRAPH 1999, pp 159-168

A. Witkin and M. Kass. Computer Graphics, 22:159-168, 1988A. Witkin and M. Kass. Computer Graphics, 22:159-168, 1988

Page 4: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Related Work:Related Work:Signal ProcessingSignal ProcessingRelated Work:Related Work:Signal ProcessingSignal Processing

A. Bruderlin and L. Williams. Proc. SIGGRAPH 1995, pp 97-104A. Bruderlin and L. Williams. Proc. SIGGRAPH 1995, pp 97-104

J. S. De Bodnet. Proc. SIGGRAPH 1999, pp 21-28J. S. De Bodnet. Proc. SIGGRAPH 1999, pp 21-28

D. J. Heeger and J. R. Bergen. Proc. SIGGRAPH 1995, pp 229-238D. J. Heeger and J. R. Bergen. Proc. SIGGRAPH 1995, pp 229-238

Z. Popovic and A. Witkin. Proc. SIGGRAPH 1999, pp 159-168Z. Popovic and A. Witkin. Proc. SIGGRAPH 1999, pp 159-168

U. Unuma, K. Anjyo and R. Tekeuchi. Proc. SIGGRAPH 1995, pp 91-96U. Unuma, K. Anjyo and R. Tekeuchi. Proc. SIGGRAPH 1995, pp 91-96

A. Witkin and Z. Popovic. Proc. SIGGRAPH 1995, PP 105-108A. Witkin and Z. Popovic. Proc. SIGGRAPH 1995, PP 105-108

Page 5: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Related Work:Related Work:Animation from MocapAnimation from MocapRelated Work:Related Work:Animation from MocapAnimation from Mocap

O. Arikan and D. A. Forsyth. Interactive motion generation O. Arikan and D. A. Forsyth. Interactive motion generation from examples. Proc. SIGGRAPH 2002from examples. Proc. SIGGRAPH 2002

L. Kovar, M. Gleicher, and F. Pighin. Motion Graphs. L. Kovar, M. Gleicher, and F. Pighin. Motion Graphs. Proc. SIGGRAPH 2002Proc. SIGGRAPH 2002

J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and N. S. Pollard. Interactive control of avatars animated N. S. Pollard. Interactive control of avatars animated with human motion data. Proc. SIGGRAPH 2002with human motion data. Proc. SIGGRAPH 2002

Y. Li, T. Wang, and H. Shum. Motion Texture: A two-level Y. Li, T. Wang, and H. Shum. Motion Texture: A two-level statistical model for character motion synthesisstatistical model for character motion synthesis

Page 6: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

• Create a method that allows an Create a method that allows an artist low-level control of the motionartist low-level control of the motion

• Combine the strengths of keyframe Combine the strengths of keyframe animation with those of mocapanimation with those of mocap

• Create a method that allows an Create a method that allows an artist low-level control of the motionartist low-level control of the motion

• Combine the strengths of keyframe Combine the strengths of keyframe animation with those of mocapanimation with those of mocap

Page 7: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

““Sketch” an animation by keyframingSketch” an animation by keyframing““Sketch” an animation by keyframingSketch” an animation by keyframing

Page 8: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

Page 9: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

Page 10: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Enhance” the result with mocap dataEnhance” the result with mocap data““Enhance” the result with mocap dataEnhance” the result with mocap data

Page 11: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Enhance” the result with mocap dataEnhance” the result with mocap data• SynthesizeSynthesize missing degrees of freedom missing degrees of freedom

““Enhance” the result with mocap dataEnhance” the result with mocap data• SynthesizeSynthesize missing degrees of freedom missing degrees of freedom

Page 12: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Sketch” an animation by keyframingSketch” an animation by keyframing• Animate only a few degrees of freedomAnimate only a few degrees of freedom

• Set few keyframes Set few keyframes

““Enhance” the result with mocap dataEnhance” the result with mocap data• SynthesizeSynthesize missing degrees of freedom missing degrees of freedom

• TextureTexture keyframed degrees of freedom keyframed degrees of freedom

““Enhance” the result with mocap dataEnhance” the result with mocap data• SynthesizeSynthesize missing degrees of freedom missing degrees of freedom

• TextureTexture keyframed degrees of freedom keyframed degrees of freedom

Page 13: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Goal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted AnimationGoal: Motion Capture Goal: Motion Capture Assisted AnimationAssisted Animation

Blue = Keyframed

Purple = Textured/Synthesized

Page 14: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

What is a Motion Texture?What is a Motion Texture?

Every individual’s movement is Every individual’s movement is uniqueunique

• “Motion texture” was coined by Ken Perlin

Every individual’s movement is Every individual’s movement is uniqueunique

• “Motion texture” was coined by Ken Perlin

Page 15: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

What is a Motion Texture?What is a Motion Texture?

Every individual’s movement is Every individual’s movement is uniqueunique

• “Motion texture” was coined by Ken Perlin

• Dance! Acrobatics!Dance! Acrobatics!

Every individual’s movement is Every individual’s movement is uniqueunique

• “Motion texture” was coined by Ken Perlin

• Dance! Acrobatics!Dance! Acrobatics!

Page 16: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

What is a Motion Texture?What is a Motion Texture?

Every individual’s movement is Every individual’s movement is uniqueunique

• “Motion texture” was coined by Ken Perlin

• Dance! Acrobatics!Dance! Acrobatics!

• Everyone walks, but not the same wayEveryone walks, but not the same way

Every individual’s movement is Every individual’s movement is uniqueunique

• “Motion texture” was coined by Ken Perlin

• Dance! Acrobatics!Dance! Acrobatics!

• Everyone walks, but not the same wayEveryone walks, but not the same way

Page 17: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Animating With Motion TextureAnimating With Motion Texture

Every individual’s movement is Every individual’s movement is uniqueunique

• Synthetic motion should capture the Synthetic motion should capture the texturetexture

Every individual’s movement is Every individual’s movement is uniqueunique

• Synthetic motion should capture the Synthetic motion should capture the texturetexture

Page 18: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Animating With Motion TextureAnimating With Motion Texture

Every individual’s movement is uniqueEvery individual’s movement is unique

• Synthetic motion should capture the textureSynthetic motion should capture the texture

• To “texture” means to add style to a pre-To “texture” means to add style to a pre-existing motionexisting motion

Every individual’s movement is uniqueEvery individual’s movement is unique

• Synthetic motion should capture the textureSynthetic motion should capture the texture

• To “texture” means to add style to a pre-To “texture” means to add style to a pre-existing motionexisting motion

Page 19: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Animating With Motion TextureAnimating With Motion Texture

Every individual’s movement is Every individual’s movement is uniqueunique

• Synthetic motion should capture the Synthetic motion should capture the texturetexture

• To “texture” means to add style to a pre-To “texture” means to add style to a pre-existing motionexisting motion

• Technically, texturing is a special case of Technically, texturing is a special case of synthesissynthesis

Every individual’s movement is Every individual’s movement is uniqueunique

• Synthetic motion should capture the Synthetic motion should capture the texturetexture

• To “texture” means to add style to a pre-To “texture” means to add style to a pre-existing motionexisting motion

• Technically, texturing is a special case of Technically, texturing is a special case of synthesissynthesis

Page 20: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control

Page 21: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

Page 22: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

•Detail hard

Page 23: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

•Detail hard•Many DOF

Page 24: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

•Detail hard•Many DOF

•Detail easy

Page 25: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

•Detail hard•Many DOF

•Detail easy•All DOF

Page 26: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

•No control

•Detail hard•Many DOF

•Detail easy•All DOF

Page 27: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control•Intuitive

•No control•Not intuitive

•Detail hard•Many DOF

•Detail easy•All DOF

Page 28: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

How an Animator How an Animator WorksWorksHow an Animator How an Animator WorksWorks

• A few degrees of freedom at firstA few degrees of freedom at first

• Not in detailNot in detail

• Fill in detail with more keyframes Fill in detail with more keyframes laterlater

• A few degrees of freedom at firstA few degrees of freedom at first

• Not in detailNot in detail

• Fill in detail with more keyframes Fill in detail with more keyframes laterlater

Page 29: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

The Method in WordsThe Method in WordsThe Method in WordsThe Method in Words

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those regions

• Put that data onto the keyframed animation

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those regions

• Put that data onto the keyframed animation

Page 30: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Before Beginning:Before Beginning:Choose Matching Choose Matching AnglesAngles

Before Beginning:Before Beginning:Choose Matching Choose Matching AnglesAngles

Left Hip xLeft Hip yLeft Hip zLeft Knee xLeft Knee yLeft Knee zLeft Ankle xLeft Ankle yLeft Ankle zLeft Ball xLeft Ball yLeft Ball zRight Hip xRight Hip yRight Hip zRight Knee xRight Knee yRight Knee zRight Ankle xRight Ankle yRight Ankle zRight Ball xRight Ball yRight Ball z

Root x transRoot y transRoot z transRoot x rotRoot y rotRoot z rotSpine1 xSpine1 ySpine1 zSpine2 xSpine2 ySpine2 zSpine3 xSpine3 ySpine3 zNeck xNeck yNeck zHead xHead yHead zHead Aim xHead Aim yHead Aim z

Left Clavicle xLeft Clavicle yLeft Clavicle zLeft Shoulder xLeft Shoulder yLeft Shoulder zLeft Elbow xLeft Elbow yLeft Elbow zLeft Wrist xLeft Wrist yLeft Wrist zRight Clavicle xRight Clavicle yRight Clavicle zRight Shoulder xRight Shoulder yRight Shoulder zRight Elbow xRight Elbow yRight Elbow zRight Wrist xRight Wrist yRight Wrist z

Time TimeTime

Page 31: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching Angles Matching Angles Drive the SynthesisDrive the SynthesisMatching Angles Matching Angles Drive the SynthesisDrive the Synthesis

Left Hip xLeft Hip yLeft Hip zLeft Knee xLeft Knee yLeft Knee zLeft Ankle xLeft Ankle yLeft Ankle zLeft Ball xLeft Ball yLeft Ball zRight Hip xRight Hip yRight Hip zRight Knee xRight Knee yRight Knee zRight Ankle xRight Ankle yRight Ankle zRight Ball xRight Ball yRight Ball z

Root x transRoot y transRoot z transRoot x rotRoot y rotRoot z rotSpine1 xSpine1 ySpine1 zSpine2 xSpine2 ySpine2 zSpine3 xSpine3 ySpine3 zNeck xNeck yNeck zHead xHead yHead zHead Aim xHead Aim yHead Aim z

Left Clavicle xLeft Clavicle yLeft Clavicle zLeft Shoulder xLeft Shoulder yLeft Shoulder zLeft Elbow xLeft Elbow yLeft Elbow zLeft Wrist xLeft Wrist yLeft Wrist zRight Clavicle xRight Clavicle yRight Clavicle zRight Shoulder xRight Shoulder yRight Shoulder zRight Elbow xRight Elbow yRight Elbow zRight Wrist xRight Wrist yRight Wrist z

Time TimeTime

Page 32: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Motion Capture DataMotion Capture DataMotion Capture DataMotion Capture Data

Left Hip xLeft Hip yLeft Hip zLeft Knee xLeft Knee yLeft Knee zLeft Ankle xLeft Ankle yLeft Ankle zLeft Ball xLeft Ball yLeft Ball zRight Hip xRight Hip yRight Hip zRight Knee xRight Knee yRight Knee zRight Ankle xRight Ankle yRight Ankle zRight Ball xRight Ball yRight Ball z

Root x transRoot y transRoot z transRoot x rotRoot y rotRoot z rotSpine1 xSpine1 ySpine1 zSpine2 xSpine2 ySpine2 zSpine3 xSpine3 ySpine3 zNeck xNeck yNeck zHead xHead yHead zHead Aim xHead Aim yHead Aim z

Left Clavicle xLeft Clavicle yLeft Clavicle zLeft Shoulder xLeft Shoulder yLeft Shoulder zLeft Elbow xLeft Elbow yLeft Elbow zLeft Wrist xLeft Wrist yLeft Wrist zRight Clavicle xRight Clavicle yRight Clavicle zRight Shoulder xRight Shoulder yRight Shoulder zRight Elbow xRight Elbow yRight Elbow zRight Wrist xRight Wrist yRight Wrist z

Time TimeTime

Page 33: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

OverviewOverviewOverviewOverview

Steps in texture/synthesis methodSteps in texture/synthesis method

• Frequency analysisFrequency analysis

• MatchingMatching

• Path findingPath finding

• JoiningJoining

Steps in texture/synthesis methodSteps in texture/synthesis method

• Frequency analysisFrequency analysis

• MatchingMatching

• Path findingPath finding

• JoiningJoining

Page 34: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

ExampleExample

Page 35: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Frequency Analysis:Break into BandsFrequency Analysis:Break into Bands

Page 36: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Fre

quen

cy

Time

Band-pass decomposition of matching angles

Keyframed Data Motion Capture Data

Frequency AnalysisFrequency Analysis

Page 37: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Fre

quen

cy

Time

Keyframed Data Motion Capture Data

Chosen low frequency band

Frequency AnalysisFrequency Analysis

Page 38: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframed Data Motion Capture Data

Hip angle data (a matching angle)

Fre

quen

cy

Time

Chosen Low Frequency BandChosen Low Frequency Band

Page 39: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframed Data Motion Capture Data

Fre

quen

cy

Time

Making FragmentsMaking Fragments

Break where first derivative changes sign

Page 40: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframed Data Motion Capture Data

Fre

quen

cy

Time

Making FragmentsMaking Fragments

Step through fragments one by one

Page 41: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

KeyframedFragment

Page 42: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

KeyframedFragment

Motion Capture Data

Page 43: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

KeyframedFragment

Motion Capture Data

Page 44: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

Compare to all motion capture fragmentsA

ngle

in d

egre

es

Time

KeyframedMocap

Page 45: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

Resample mocap fragments to be same lengthA

ngle

in d

egre

es

Time

KeyframedMocap

Page 46: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

Keep the K closest matchesA

ngle

in d

egre

es

Time

KeyframedMocap

Page 47: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

KeyframedFragment

Motion Capture Data

Page 48: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

KeyframedFragment

Motion Capture Data

CloseMatches

Page 49: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

MatchingMatching

Hip Angle (Matching Angle)

Spine Angle (For Synthesis)

Page 50: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching and SynthesisMatching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Page 51: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching and SynthesisMatching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Page 52: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching and SynthesisMatching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Page 53: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching and SynthesisMatching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Page 54: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching and SynthesisMatching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Page 55: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Matching and SynthesisMatching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Page 56: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Possible Synthetic Spine Angle DataPossible Synthetic Spine Angle Data

Ang

le in

deg

rees

Time

Page 57: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Path FindingPath FindingA

ngle

in d

egre

es

Time

Page 58: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Path FindingPath FindingA

ngle

in d

egre

es

Time

Page 59: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Path FindingPath FindingA

ngle

in d

egre

es

Time

Page 60: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Path FindingPath FindingA

ngle

in d

egre

es

Time

Page 61: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Path FindingPath FindingA

ngle

in d

egre

es

Time

Page 62: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

JoiningJoiningA

ngle

in d

egre

es

Time

Page 63: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

TexturingTexturing

Band-pass decomposition of keyframed dataF

requ

ency

Time

Page 64: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

TexturingTexturing

Synthesize upper frequency bandsF

requ

ency

Time

Page 65: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframed SketchKeyframed Sketch

Page 66: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Motion Capture DataMotion Capture Data

Two different styles of walk

Page 67: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Enhanced AnimationEnhanced Animation

Upper body is synthesized

Lower body is textured

Page 68: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframed Sketch With More DetailKeyframed Sketch With More Detail

Page 69: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Motion Capture DataMotion Capture Data

Page 70: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Textured AnimationTextured Animation

Blue = Keyframed

Purple = Textured

Page 71: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Lazy Keyframed SketchLazy Keyframed Sketch

Page 72: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Enhanced AnimationEnhanced Animation

Blue = Keyframed

Purple = Textured/Synthesized

Page 73: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Different PathsDifferent Paths

Page 74: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Summary of the MethodSummary of the Method

Keyframed data

Mocap Data

Keyframed Data

Mocap Data Possible Synthetic Data

Matching Angles

Sketch + Mocap

Frequency Analysis Matching

Path Finding JoiningEnhanced Animation

Page 75: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Conclusions and ApplicationsConclusions and Applications

• For more automatic generation with high For more automatic generation with high level control, the previous methods are more level control, the previous methods are more appropriate appropriate

• Appropriate for an artist interested in a very Appropriate for an artist interested in a very particular style of motionparticular style of motion

• The artist may have a relatively small motion The artist may have a relatively small motion capture set of that stylecapture set of that style

• The artist may want precise control over The artist may want precise control over parts of the motionparts of the motion

• For more automatic generation with high For more automatic generation with high level control, the previous methods are more level control, the previous methods are more appropriate appropriate

• Appropriate for an artist interested in a very Appropriate for an artist interested in a very particular style of motionparticular style of motion

• The artist may have a relatively small motion The artist may have a relatively small motion capture set of that stylecapture set of that style

• The artist may want precise control over The artist may want precise control over parts of the motionparts of the motion

Page 76: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

For more info. . . For more info. . .

http://graphics.stanford.edu/~pullen

Special Thanks to:Reardon Steele, Electronic Arts

Page 77: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Choices the Animator Must MakeChoices the Animator Must Make

1.1. Which DOF to use as matching anglesWhich DOF to use as matching angles

2.2. Which DOF to texture, which to synthesizeWhich DOF to texture, which to synthesize

3.3. Which frequency band to use in matchingWhich frequency band to use in matching

4.4. How many frequency bands to use in How many frequency bands to use in

texturingtexturing

5.5. How many matches to keep How many matches to keep

1.1. Which DOF to use as matching anglesWhich DOF to use as matching angles

2.2. Which DOF to texture, which to synthesizeWhich DOF to texture, which to synthesize

3.3. Which frequency band to use in matchingWhich frequency band to use in matching

4.4. How many frequency bands to use in How many frequency bands to use in

texturingtexturing

5.5. How many matches to keep How many matches to keep

Page 78: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Conclusions and Further WorkConclusions and Further Work

• Direct incorporation of hard Direct incorporation of hard constraintsconstraints

• Fundamental units of motionFundamental units of motion

• Direct incorporation of hard Direct incorporation of hard constraintsconstraints

• Fundamental units of motionFundamental units of motion

Page 79: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframe Data vs. Keyframe Data vs. Motion Capture DataMotion Capture DataKeyframe Data vs. Keyframe Data vs. Motion Capture DataMotion Capture Data

Page 80: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

TexturingTexturing

Synthesize upper frequency bands

Page 81: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Ang

le in

deg

rees

Time

MatchingMatching

Keyframed

Page 82: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Keyframed

Page 83: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Keyframed

Textured

Page 84: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Keyframed

Textured

Not keyframed

Page 85: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Enhancing Animations:Enhancing Animations:Texturing and Texturing and SynthesisSynthesis

Keyframed

Textured Synthesized

Not keyframed

Page 86: Animation From Motion Capture Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler Motion Capture Assisted Animation:

Keyframing vs. MocapKeyframing vs. MocapKeyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control