Rowing Motion Capture SystemSimon Fothergill

Ph.D. student, Digital Technology Group, Computer Laboratory

Jesus College graduate conference May 2009

Overview

• The Bigger Picture

• Previous work

• Problem

• Process

• Data Capture System

• Results

• Future work

The Bigger Picture

• Sentient Computing!

• Computer Vision

• Pattern Recognition & Machine learning

• A long way to go!

The Bigger Picture – Watching Humans

• Physical Performances

• Heath care

• What are they doing?

• How well are they doing it?

• How should be improved?

• How should they be told?

Previous Work - Activity / Gesture recognition

• Motion capture methods have included:

• Blob tracking

• Point trajectories

• Recognition techniques have included:

• Single frame

• Multiple frame

• Parametric

Learn the quality of a performance from body part trajectories

• Minimise markers using redundancy

• Complex trajectories, continuous score

• Flexible rubrics require learning

• Different types of expert labelling:

• Explanations

• Non-specific / specific

• Different granularities of quality

• Which sections of the trajectory are how relevant?

• One section of a can depend on many aspects

Process

Learning

Judging

Performance

Capture motion

Expert coach labels with their judgement

Trajectories

Inference modelLearn

Video

Performance

Capture motion

Trajectories Inference model

FeaturesExtract and select features

Features

Extract and select features

Judgement

Evaluate

Capture video

Data Capture System

Erg

PowerControl

Motion sensitive LED markers

ECS

Wii controllers

Data Capture System - Architecture

Nintendo Wii controller

 

Bluetooth

IR 1024x768 camera(100Hz)

Nintendo Wii controller

 IR 1024x768 camera

(100Hz)

PC

Wii libraryBluetooth library

C server

Bluetooth

PCJava / C client

Video camera(30Hz)

Fire wire

TCP/IP

C server

Buffer Wii controller Wii controller

Data Capture System – Calibration and operation

Server

Triangulation

Stereo calibration

Client

4 x 2D coordinates

4 x 3D coordinates

Erg calibration

Label markers

Transform to ECS

Update ECS if necessary ECS

Detect strokes

Log data Log files

Save picture

Encodes video

Calibrate labeller

Calibrate WMCS

StorageBatch

Display on GUI

Calculate stats

Control camera

Cal

ibra

tion

Live

ope

ratio

n

Data Capture System

• Example video

Preliminary Results

• Preliminary results have been obtained using a dataset of 6 rowers and the complete trajectory of the erg handle only. Binary classification over stroke quality was done using tempo-spatial features of the trajectory and a neural network. Two training methods were compared.

60 70 80 90 100

Quick hands (2)

Early open back (2)

Separate arms/legs (3)

Overreaching (4)

Percent of correctly classified strokes

Gradient descent training

Moore-Penrose training

Classification accuracy across given number of performers, for quality of individual aspects

of technique.

Summary and Further Work

• Data capture system and how it fits into the bigger picture

• More information is available on the feature extraction & selection and inference algorithms.

• A larger data set would allow conclusive results to be obtained

• Feature extraction and selection methods that address using the relevant segments of the relevant trajectories

• More sophisticated modelling based on particle filters

• Supports multiple body parts and labelling methods

• Uses a distribution of motion vectors to probabilistically track the “quality so far” as the stroke evolves.

In Conclusion

• Advertisement!

• Acknowledgements

• Professor Andy Hopper, Dr Sean Holden, Dr Robert Harle

• Members of the DTG and Rainbow groups, Computer Laboratory

• Jesus College, JCBC and the Graduate society

• References

• Optical tracking using commodity hardware, Hay, S.; Newman, J.; Harle, R.; ISMAR 2008. Page(s):159 - 160

Thank you!

Questions?

Please come down to the boathouse and use the data capture system!