EEC-492/592 Kinect Application Development Lecture 15 Wenbing Zhao [email protected].
1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao [email protected].
Transcript of 1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao [email protected].
1
EEC-492/592EEC-492/592Kinect Application Kinect Application
DevelopmentDevelopmentLecture 2Lecture 2
Wenbing ZhaoWenbing Zhao
[email protected]@ieee.org
22
Outline
Reminder No class next Monday: Martin Luther King Day
Components of Kinect sensor Kinect for Windows SDK Building first Kinect application
33
Components of Kinect for Windows Color camera Infrared (IR) emitter IR depth sensor Tilt motor Microphone array LED
44
The Color Camera
Red, blue, green color images (RGB)
Frame rate: 30 FPS at 640x480 12 FPS at 1280x960
Viewing range: 43 degrees vertical by
57 degrees horizontal
55
IR Emitter & IR Depth Sensor IR emitter emits IR light in a pseudo-random dot pattern Dotted light reflects off different objects IR depth sensor reads reflected light => depth info Depth stream resolution
640x480 320x240 80x60
66
Depth Data Processing
77
Tilt Motor Used to change the camera and sensor’s angles to
get the correct position of the human skeleton Can tilt vertically (upwards/downwards) up to 27
degrees
88
Microphone Array 4 different microphones: locate the direction of
the audio wave, not just capture the sound Enhanced noise suppression, echo cancellation,
beam-forming
99
LED
Placed between camera and IR emitter Used for indicating the status of the Kinect Green indicates Kinect drivers loaded
properly
1010
Kinect for Xbox vs. Kinect for Windows Kinect for Windows
Near mode: as close as 40cm away from Kinect Can be used for commercial applications Can be used in a Win7 virtual machine: HyperV,
VMWare, Parallels For both
Track motion up to 12 feet (4 meters) away Identical resolution
1111
Kinect for Windows SDK Supported operating systems
Windows 7, Windows 8, Windows Embedded 7 Hardware requirements
Dual core 2.66 GHz or faster CPU Dedicated USB 2.0 bus 2GB or higher RAM
Software requirement Microsoft Visual Studio 2010 or higher Kinect for Windows SDK: current version 1.8
1212
Kinect for Windows SDK
1313
Kinect for Windows SDK
1414
How Applications Interact with Kinect
1515
Classification of Kinect SDK APIs
1616
Kinect Driver The Kinect driver controls the camera, depth
sensor, audio microphone array, and motion Data passes between the sensor and the app in
the form of data streams Color data stream Depth data stream Audio data stream Infrared data stream (for low light environment) Accelerameter data
1717
The Near Mode Track a human upper body as close as 40 cm Only Kinect for Windows supports the near mode
1818
Tracking Human Skeleton and Joint Movement Kinect SDK does the magic to
extract the skeleton and joint positions from the data streams received
Can track up to two users at the same time with full joint positions
Each skeleton consists of 20 joints
1919
Building First Kinect App Modified KinectInfoBox app Steps
Create a new project: C# WPF application Adding Kinect reference Draw the GUI Modify MainWindow.xaml Adding code
2020
Creating a New Project
2121
Adding Kinect Libraries
2222
Create User Interface
2323
Modify MainWindow.xaml
2424
Adding Code (MainWindow.xaml.cs) Setting up name space
using Microsoft.Kinect;
Setting up state variable for the Kinect sensor An instance of Microsoft.Kinect.KinectSensor class It represents the complete runtime pipeline for the Kinect sensor
during life cycle of the apppublic partial class MainWindow : Window
{KinectSensor sensor;
// remaining code goes here
}
2525
Adding Code private void WindowLoaded(object sender, RoutedEventArgs e) {
if (KinectSensor.KinectSensors.Count > 0) {
this.sensor = KinectSensor.KinectSensors[0];
if (this.sensor != null && !this.sensor.IsRunning) {
this.sensor.Start();
displayInfo();
}
}
else {
MessageBox.Show("No device is connected with system!");
this.Close();
}
}
private void WindowClosing(object sender, System.ComponentModel.CancelEventArgs e) {
if (this.sensor != null && this.sensor.IsRunning) {
this.sensor.Stop();
}
2626
Adding Code
Double click “Increase” button, the template for the button1_Click() method will be automatically created
Double click “Decrease” button, the template for the button2_Click() method will be automatically created
private void displayInfo() { this.textBlock1.Text = this.sensor.DeviceConnectionId; this.textBlock2.Text = this.sensor.Status.ToString(); this.textBlock3.Text = this.sensor.ElevationAngle.ToString(); }
private void button1_Click(object sender, RoutedEventArgs e) { this.sensor.ElevationAngle = this.sensor.ElevationAngle + 1; displayInfo(); }
private void button2_Click(object sender, RoutedEventArgs e) { this.sensor.ElevationAngle = this.sensor.ElevationAngle - 1; displayInfo(); }
2727
More In-depth Stuff Starting Kinect sensor: sensor.Start() In the Start() method
First check the status of Kinect, if no sensor connected, runtime will throw an InvalidOperationException object with the KinectNotReady message
Init the sensor if connected
Init options None: default option UseDepthAndPlayerIndex UseColor UseSkeletonTracking UseDepth UseAudio
2828
More In-depth Stuff Open data streams
ColorImageStream: this.sensor.ColorStream.Enable(); DepthImageStream: this.sensor.ColorStream.Enable(); SkeletonStream: this.sensor.SkeletonStream.Enable();
Stopping the Kinect sensor this.sensor.Stop(); Stops the color, depth, and skeleton data stream Stops audio source, if open Kills all threads spawned Shuts down Kinect sensor and sets init option to None