Languages
Pages
Legal
Increasing Kinect Application Development Productivityby an Enhanced Hardware Abstraction
Bernardo Reis ([email protected])
Virtual Reality and Multimedia Research Group
Federal University of Pernambuco, Computer Science Center
João Marcelo Teixeira ([email protected])
Felipe Breyer ([email protected])
Luis Arthur Vasconcelos ([email protected])
Aline Cavalcanti ([email protected])
André Ferreira ([email protected])
Judith Kelner ([email protected])
Development Difficulties
Necessity to have enough space
• At least 2m from the device
Physical weariness during tests
• Due to repetition
Specific knowledge for the right movements
• Necessity to have the support of a specialist during development and testing
Necessity to own the accessory
• One for each developer
2/19
Solution
A tool that enables playback of previously recorded information from Kinect (as if being streamed directly
from the device), supplying the developer with sufficient mechanisms for a productive development environment.
3/19
Existing Solutions
Cons
Can’t access skeleton data
Can’t perform depth/color registration
Based on a very low level library
Pros
Open-source
Avoids changes in code
4/19
(FAKENECT)
Existing Solutions
Cons
No access to audio or elevation motor
Changes in code are required
Pros
Open-source
Performs depth/color registration
Can record while running the user application
5/19
(BUILT-IN)
Existing Solutions
Cons
Requires to have a Kinect connected
Pros
Saves the dump transparently
Access directly the driver
Can manipulate the input at runtime
6/19
(KINECT STUDIO)
Identified Problems
Good specific solutions, such as:
• No changes in code
• Record while running user application
But they all lack a sense of a complete and productive environment, missing important features that we believe are important.
7/19
KINA Toolkit
A toolkit for reducing the effort of the programmer on the development of body-tracking based applications that make use of Microsoft Kinect SDK. Its design principles are:
Clean and simple API
Require the minimum possible knowledge
Support a productive development model
8/19
KINA Toolkit
9/19
Recorders
Simple Recorder
• Small application for recording Kinect data
• Independent from the user application
• Should be used for initial specifications of movements
Pass-through Recorder
• Library that intercepts user streams and records them
• Enables feedback from the user application
• Should be used for later testing
10/19
Recorders
Very low storage overhead!
Can store:
• RGB
• Depth/Depth + Player
• Skeleton
Compressed:
• JPEG (color), 90% quality
• PNG (depth)
• 93% reduction in size
11/19
Movement Database
12
Editor
Allows cropping of dump files
• Adjusting them to your needs
• Generating multiple dumps from one record
• In the future, will also combine dumps
13/19
Emulator
Playback mechanism of KINA
A C++ library that can be used without any change in code
Maintains temporal coherence with the original input
Currently, only supports pooling mode of capturing/reading
Small RAM overhead (11MB against 158MB of the MS SDK)
14/19
Development Model
15/19
In Advanced Topics in Media and Interaction class:
• 12 CS students, 20 to 27 years old, 1 woman
• No previous knowledge of natural interaction
• 3 groups, based on acquaintance between them
Experiment
Functional training program Art experiment wherethe body is a brush
Bug-smashing game
16/19
Experiment Feedback
Suggestions:
• Possibility to dynamically change the dump during execution
• GUI for the simple recorder
“Indispensable during test phase”
“it reduced the overall time of development”
17/19
The toolkit was positively evaluated by the students.
It is essential to have similar solutions for multi-dimensional interface devices.
The development model was shown to be well suited for Kinect based applications.
Conclusions and Future Works
Future work:
• Combining dumps in the editor
• Event-based playback support
• C# API
18/19
Increasing Kinect Application Development Productivityby an Enhanced Hardware Abstraction
Bernardo Reis ([email protected])
Virtual Reality and Multimedia Research Group
Federal University of Pernambuco, Computer Science Center
João Marcelo Teixeira ([email protected])
Felipe Breyer ([email protected])
Luis Arthur Vasconcelos ([email protected])
Aline Cavalcanti ([email protected])
André Ferreira ([email protected])
Judith Kelner ([email protected])
Top Related