Reach Out and Touch Space (Motion Learning)
description
Transcript of Reach Out and Touch Space (Motion Learning)
![Page 1: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/1.jpg)
Reach Out and Touch Space(Motion Learning)
Luis Goncalves, Enrico Di Bernardo, Pietro Perona
California Institute of Technology
![Page 2: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/2.jpg)
Motion is an important cue
![Page 3: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/3.jpg)
A system for body tracking(Goncalves-Di Bernardo-Perona, ICCV ‘95)
• Single camera
• Real-time estimate of body pose
• 3D Model-based
• No markers required
• Able to track at frame rate (30 fr/sec)
• 8% max error (along the line of sight)
• No loose clothing• Calibration on the user • Loses track for fast movements
BUT
![Page 4: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/4.jpg)
A system for body tracking(Goncalves-Di Bernardo-Perona, ICCV ‘95)
Arm silhouette generation 3D Model
BackgroundSubtraction
Camera Errorvector
Estimated arm position and velocity
RecursiveEstimator
Calibrationparameters
Arm modelparameters
DynamicalModel
![Page 5: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/5.jpg)
Current model: Random Walk
Dynamical model equations:
),0(, Νwwv
v
![Page 6: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/6.jpg)
Why Models for Human Motion ?
• Locomotion (biomechanics, robotics)
• Brain motor control (neuroscience)
• Human/Machine perception of biological
motion (neuroscience, psychophysics, computer vision)
• Realistic animation (computer graphics)
![Page 7: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/7.jpg)
Invariant properties of ballistic point to point movements
• The path is approximately straight
• The tangential velocity profile has a smooth bell shape
• These properties are invariant wrt subject, execution time, load carried
(Hollerbach, Viviani, Flash-Hogan, Bizzi)
Far from the neuro-muscular limits and after practice:
No predictive power for general motion
![Page 8: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/8.jpg)
Human figure animation in CG
• Keyframing, manual editing (Perlin, animation software)
• Physics-based (Hodgins et al.)
• Constraint optimization (Witkin and Kass, Badler et al.)
• Human motion capture (Bruderlin, Rose et al.)
![Page 9: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/9.jpg)
The proposed method
• Acquire sample human motions
• Label each individual motion with a high level description (e.g. goal of motion)
• Learn a function that maps labels to motions
![Page 10: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/10.jpg)
The 2D Motion Capture System
• 14 dots on the main body joints• single camera• UV lighting• real-time detection with sub-pixel accuracy
Title:/home2/dibe/matlab/MLD/reports/MLDibe.psCreator:XV Version 3.10 Rev: 12/16/94 - by John BradleyPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 11: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/11.jpg)
Example: Reaching motions
Start from a fixed initial pose.
Reach to various locations in space.
![Page 12: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/12.jpg)
Pictorial representationLabel space:
Trajectory space:
LNLlTN*28Mm
![Page 13: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/13.jpg)
The Functional Space
NLdN
ddj L
ll 10
11)( l1. Polynomial Basis Functions
2. Radial Basis Functions
T
jjj
ej
)()(21 1
)(μlΣμl
l
![Page 14: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/14.jpg)
Learning the function
ki
ik
ik
wjk mfw
jk ,
2
}{))((arg min l
ΦMW
N
jjjkkk wfm
0)()(ˆ ll
![Page 15: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/15.jpg)
Experiments: Reaching motions
• NL = 2
• NT = 120
Picking up apples in30 different locations
90 sample motions
![Page 16: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/16.jpg)
Experiments: Drawing motionsDrawing strokes on a blackboard• NL = 8
• NT = 60110 sample strokes
![Page 17: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/17.jpg)
How to evaluate performance?
• RMS error
• Perceptual evaluation by subjects
• other methods ...
![Page 18: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/18.jpg)
Results: %RMS error for Reaching
![Page 19: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/19.jpg)
Results: Visual DiscriminabilityReaching using 3rd order polynomial functional space
![Page 20: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/20.jpg)
Results: %RMS error for Drawing
![Page 21: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/21.jpg)
Results: Visual DiscriminabilityDrawing using 1st order polynomial functional space
![Page 22: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/22.jpg)
Combining Reaching and Drawing
Blend out trajectory discontinuities at merge points of different motions.
![Page 23: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/23.jpg)
Conclusions
• The method generates realistic synthetic reaching and drawing motions • The method can generate motions from a high level description
• The technique can be used for animation
![Page 24: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/24.jpg)
Future work• Obtaining and running experiments with 3D captured data
• Develop a perceptually motivated metric
• Experiment with other high level labels such as speed, emotional state or gender
• Use models for tracking/prediction
![Page 25: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/25.jpg)
Minimum jerk trajectories in reaching movements (Flash-Hogan ‘85)
Explains the experimental evidence for both straight and via point reaching motions
The trajectory minimizes:
Tt
t
dttC0
2)(x
![Page 26: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/26.jpg)
The equilibrium point trajectory (Bizzi ‘91)
The muscular system has a spring-like behavior
Brain signals activate entire muscle groups
The activation signals depend only on the ideal (minimum jerk) trajectory
![Page 27: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/27.jpg)
Motion is a powerful cue
From the trajectories of 12 dots attached to the main joints in the body, a subject could• distinguish human motion from other objects motion• identify gender• identify action and mood• perceive 3D structure from 2D trajectories(Johansson ‘73, Mather ‘94, Dittrich et al. ‘96)
![Page 28: Reach Out and Touch Space (Motion Learning)](https://reader036.fdocuments.net/reader036/viewer/2022062322/56814e4e550346895dbbdd43/html5/thumbnails/28.jpg)
Labeling the data