Repetition Counting With Microsoft Kinect Presented by: Jonathan Gurary Dai Jun.
-
Upload
marcia-stevens -
Category
Documents
-
view
217 -
download
3
Transcript of Repetition Counting With Microsoft Kinect Presented by: Jonathan Gurary Dai Jun.
Repetition Counting With Microsoft Kinect
Presented by:Jonathan Gurary
Dai Jun
Proposal
Design a Kinect-based system for physical therapy activities
Usable by the end-user in their home No professional staff or facility needed Simple UI anyone can use Similar in performance to expensive 8-
camera setups
Identify and count a variety of motions
Implementation
Hardware: Microsoft Kinect
Software: OpenNI API, executed through Processing
Designed for visual software, communicating with vision sensors
OpenNI
Cross-platform support
-Linux, Mac, Windows Older mod community
-Less updates = more working samples
-Issues are better documented And there's books
-First Kinect SDK books just hit shelves
-OpenNI has a couple books already
Limiting Kinect Problems
Bad guesses Lost a limb? It must be straight at your
side! Mismatched limbs
Crossed legs or arms can “swap” Jitter
Shaky and unstable reads These can happen with no effect on
Confidence
Effects on Repetition Counting
Angles and positions will be collected in error Almost certainly out-of-bounds for the
exercise Best case: Happens during non-critical
portion of motion, no effect Worst case: miss flag, false count,
reset the motion
Our Solution to Erroneous data
Make sure the user knows what the Kinect is seeing
Jerky skeletons, mismatched or lost limbs, all very obvious. Just stop and try again.
Stop looking at data when confidence drops Allows for the split-second guessing
errors common in some motions. Use a tolerance
Mitigate the effects of jittery reads
Flag-Based Motion Identification
Each motion or exercise is broken down and split into five intermediate steps
Example: waving your hand Change in elbow angle of about 50
degrees Step 1: elbow angle reduces 10
degrees Step 2: elbow angle reduces 10 more
degrees
Flag-Based Motion Identification
Every step becomes a flag to be passed Must be passed in order All flags reset (exercise failed) if a body
part goes out of bounds If every flag is passed correctly, count
the action
How Are Flags Determined
Flags are manually input by the coder Can use angles of all major limbs, height of
torso, and distance of hands and feet Potentially (future feature) scale to
user's height
Recording and Playback
Main program records user's entire session automatically
Saved during execution to a .onni file Separate program available for playback of
session. Skeleton filtered out Can pause, fast forward, slow down, or
seek through recording
User Interface Design
User Friendly, easy to use -no programming knowledge required -motion icons to choose -works like a button, but beautiful!
Replay mode -control bar -works like a real media player!
User Interface Design
User Interface Design
Dynamic Flag Setup
Identifying flags by examining Kinect output is tedious
Dynamic program allows the user to “input” flags with their body
Allows any almost any motion to be input on the fly
Data is output to a text file, allowing coder to turn the input into a featured exercise easily
Dynamic Flag Limitations
Impossible to tell what tolerance will be needed
Whole body is captured All limbs must be visible and not
guessed. Exercise is limited to the position
(distance wise) that it was started at Counting isn't always accurate,
especially for small or complex motions
Demo Presentation
Summary
Developed a relatively accurate, user friendly physical therapy program
Count repetitions of exercises Exercises can be set by a professional, added
to the code Dynamic flag setter to identify motion,
then potentially added to exercises by coder
User can view raw footage of exercises
Future Features
Allow the dynamic flag set to ignore some parts of the body
Adjust for big height differences between users
Allow a health professional to easily add a dynamically created exercise to code