EICS2012: Increasing Kinect Application Development Productivity by an Enhanced Hardware Abstraction
19
Increasing Kinect Application Development Productivity by an Enhanced Hardware Abstraction Bernardo Reis ([email protected]) Virtual Reality and Multimedia Research Group Federal University of Pernambuco, Computer Science Center João Marcelo Teixeira ([email protected]) Felipe Breyer ([email protected]) Luis Arthur Vasconcelos ([email protected]) Aline Cavalcanti ([email protected]) André Ferreira ([email protected]) Judith Kelner ([email protected])
-
Upload
joao-marcelo-teixeira -
Category
Technology
-
view
412 -
download
2
description
Designing and implementing the interaction behavior for body tracking capable systems requires complex modeling of actions and extensive calibration. Being the most recent and successful device for robust interactive body tracking, Microsoft’s Kinect has enabled natural interaction by the use of consumer hardware, providing detailed and powerful information to designers and developers, but little tooling. To fulfill this lack of adequate tools for helping developers in the prototyping and implementation of such interfaces, we present Kina. It works in the form of a toolkit to be used together with Microsoft Kinect for Windows SDK and provides playback capabilities and in-use recording with low effort for developers. An online movement database is also made available and provides samples that can be employed when performing initial tests using the toolkit.
Transcript of EICS2012: Increasing Kinect Application Development Productivity by an Enhanced Hardware Abstraction
Increasing Kinect Application Development Productivityby an Enhanced Hardware Abstraction
Bernardo Reis ([email protected])
Virtual Reality and Multimedia Research Group
Federal University of Pernambuco, Computer Science Center
João Marcelo Teixeira ([email protected])
Felipe Breyer ([email protected])
Luis Arthur Vasconcelos ([email protected])
Aline Cavalcanti ([email protected])
André Ferreira ([email protected])
Judith Kelner ([email protected])
Development Difficulties
Necessity to have enough space
• At least 2m from the device
Physical weariness during tests
• Due to repetition
Specific knowledge for the right movements
• Necessity to have the support of a specialist during development and testing
Necessity to own the accessory
• One for each developer
2/19
Solution
A tool that enables playback of previously recorded information from Kinect (as if being streamed directly
from the device), supplying the developer with sufficient mechanisms for a productive development environment.
3/19
Existing Solutions
Cons
Can’t access skeleton data
Can’t perform depth/color registration
Based on a very low level library
Pros
Open-source
Avoids changes in code
4/19
(FAKENECT)
Existing Solutions
Cons
No access to audio or elevation motor
Changes in code are required
Pros
Open-source
Performs depth/color registration
Can record while running the user application
5/19
(BUILT-IN)
Existing Solutions
Cons
Requires to have a Kinect connected
Pros
Saves the dump transparently
Access directly the driver
Can manipulate the input at runtime
6/19
(KINECT STUDIO)
Identified Problems
Good specific solutions, such as:
• No changes in code
• Record while running user application
But they all lack a sense of a complete and productive environment, missing important features that we believe are important.
7/19
KINA Toolkit
A toolkit for reducing the effort of the programmer on the development of body-tracking based applications that make use of Microsoft Kinect SDK. Its design principles are:
Clean and simple API
Require the minimum possible knowledge
Support a productive development model
8/19
KINA Toolkit
9/19
Recorders
Simple Recorder
• Small application for recording Kinect data
• Independent from the user application
• Should be used for initial specifications of movements
Pass-through Recorder
• Library that intercepts user streams and records them
• Enables feedback from the user application
• Should be used for later testing
10/19
Recorders
Very low storage overhead!
Can store:
• RGB
• Depth/Depth + Player
• Skeleton
Compressed:
• JPEG (color), 90% quality
• PNG (depth)
• 93% reduction in size
11/19
Movement Database
12
Editor
Allows cropping of dump files
• Adjusting them to your needs
• Generating multiple dumps from one record
• In the future, will also combine dumps
13/19
Emulator
Playback mechanism of KINA
A C++ library that can be used without any change in code
Maintains temporal coherence with the original input
Currently, only supports pooling mode of capturing/reading
Small RAM overhead (11MB against 158MB of the MS SDK)
14/19
Development Model
15/19
In Advanced Topics in Media and Interaction class:
• 12 CS students, 20 to 27 years old, 1 woman
• No previous knowledge of natural interaction
• 3 groups, based on acquaintance between them
Experiment
Functional training program Art experiment wherethe body is a brush
Bug-smashing game
16/19
Experiment Feedback
Suggestions:
• Possibility to dynamically change the dump during execution
• GUI for the simple recorder
“Indispensable during test phase”
“it reduced the overall time of development”
17/19
The toolkit was positively evaluated by the students.
It is essential to have similar solutions for multi-dimensional interface devices.
The development model was shown to be well suited for Kinect based applications.
Conclusions and Future Works
Future work:
• Combining dumps in the editor
• Event-based playback support
• C# API
18/19
Increasing Kinect Application Development Productivityby an Enhanced Hardware Abstraction
Bernardo Reis ([email protected])
Virtual Reality and Multimedia Research Group
Federal University of Pernambuco, Computer Science Center
João Marcelo Teixeira ([email protected])
Felipe Breyer ([email protected])
Luis Arthur Vasconcelos ([email protected])
Aline Cavalcanti ([email protected])
André Ferreira ([email protected])
Judith Kelner ([email protected])
