Intel® Perceptual Computing SDK 2013 GOLD Release Notes

Intel® Perceptual Computing SDK 2013 GOLD

Release Notes (Version 13842)

Overview

The Intel® Perceptual Computing SDK is a library of pattern detection and recognition

algorithm implementations exposed through standardized interfaces. The library aims at

lowering barriers to using these algorithms and shifting the application developers’ focus

from coding the algorithm details to innovating on the usage of these algorithms for next

generation human computer experience.

Features – Change Logs

The SDK sets the environment variable PCSDK_DIR to the installation location. By default,

PCSDK_DIR points to C:\Program Files (x86)\Intel\PCSDK.

The SDK provides the following documentation under $(PCSDK_DIR)\doc:

Name Reference Manual Description

CHM/sdkhelp.chm The CHM version comprehensive

documentation about developer guide,

reference manuals, samples and tutorials.

CHM/sdkmanuals.chm The CHM version reference manuals.

HTML/index.html The HTML version comprehensive

documentation about developer guide,

reference manuals, samples and tutorials.

PDF/sdkdevguide.pdf The PDF version developer guide.

PDF/sdkmanuals.pdf The PDF version reference manuals.

PDF/sdkprivacy.pdf The PDF version privacy notification guide.

PDF/sdksamples.pdf The PDF version sample guide.

PDF/sdktutorial-cpp.pdf The PDF version C++ tutorial.

PDF/sdktutorial-csharp.pdf The PDF version C# tutorial.

PDF/sdktutorial-open.pdf The PDF version openFrameworks* tutorial.

PDF/sdktutorial-processing.pdf The PDF version Processing* tutorial.

PDF/sdktutorial-java.pdf The PDF version Java* tutorial.

PDF/sdktutorial-unity.pdf The PDF version Unity* tutorial.

PDF/sdktutorial-havok.pdf The PDF version Havok* tutorial.

Release 13842 (GOLD R8):

This release provides the following major changes:

o Pulse Estimator Preview module

o Emotient* emotion detection Preview module

o Support for Nuance* Genie* Command And Control, Dictation, Text To Speech

(This is supported only for Windows* 8)

o Additional projection interface Color to depth projection



o Capture Viewer enhancements for 3D



o Tracing capability is enabled and the trace information can be viewed via sdk_info

o C++ and C# sample raw_streams which unifies raw stream playback and

projection visualization



o A DepthBall demo application has been added to showing development with

Microsoft XNA and 3D data. This needs MS .NET FW 4.0 installed on the machine.

o Enabled UtilCaptureFile for real-time recording.



o Changes to address the Command & Control interoperability issue with Dragon

Assistant application; Dictation will be fixed in later release.

o Added documentation which captures Best Known methods for Voice Recognition

– to tune audio volume and confidence thresholds for improved recognition.

o Camera Calibration tool: The Intel® Perceptual Computing SDK 2013 R3 (version

9302) shows a misalignment in the UV map and color image with earlier versions

of the Creative camera developer kits. The camera calibration tool is a part of R4

and is a short-term solution to compensate certain camera offset mapping depth

to color coordinates. This is a developer tool only. For end users, the cameras are

calibrated during the manufacturing process.


o This release provides the following major features:

o Background Segmentation/Removal module

o Support for multiple languages in Speech

deDE German (Germany) enGB English (Great Britain) enUS English (United States) frFR French (France) itIT Italian (Italy) jaJP Japanese (Japan) ptBR Portuguese (Brazil) esUS Spanish (United States)

zhCN Chinese (PRC)



o Support Java application development.

o Enable projection support in framework porting libraries.

o The voice_recognition and voice_synthesis samples are rewritten to be GUI

applications for UNICODE input/output.

o Rewrote the gesture_viewer sample to be GUI based and include the simple

(UtilPipeline-based) and advanced pipeline (UtilCapture-based). Removed the

gesture_viewere_simple sample.

o Rewrote the gesture_viewer_simple.cs (C#) sample to be GUI based and renamed

it to be gesture_viewer.cs.

o Added a face_analysis.cs sample (C#) to demonstrate face detection and

landmark tracking.

o Added a tutorial to support using the SDK under Havok Vision* SDK.

o The face attribution detection feature is deprecated. Applications built with the SDK

GOLD version will be still functional in SDK GOLD R2.

o Major revisions to documentation including C# function definitions and samples

Release 7383 (Gold):


o Native Microsoft* Visual Studio* 2008, 2010 and 2012 project files for samples.

o Two usage demo applications:

The AugmentedFarm sample application displays an augmented book, using the

object tracking technology.

The RemoteControl sample application demonstrates driving non-SDK

applications using SDK enabled finger, gesture and voice events.

o With the GOLD license, the developers can create commercial SDK applications.

Please refer to the SDK developer guide $(PCSDK_DIR)/doc/PDF/sdkdevguide.pdf

for application deployment recommendations and requirements, the redistributable

files $(PCSDK_DIR)/redist.txt, and the license terms $(PCSDK_DIR)/Intel PCSDK

EULA.rtf.

Release 6765 (Beta3):

This release provides the following major features:

o The Creative Senz 3D* I/O module exposes the PXCProjection interface for

mapping RGB, depth and world coordinates. See sdkmanual-core.pdf for how to

use the interface. The camera_uvmap sample illustrates the usage.

o The finger tracking module exposes a new pose BIG5 and three new gestures

CIRCLE, SWIPE_UP, and SWIPE_DOWN.

o The Unity and Processing framework ports now support voice recognition and

dictation. See sdkmanual-framework.pdf for details.

o The finger tracking module and the face tracking module use Intel® AVX instructions

set for performance optimization. As a result, the SDK requires Microsoft Windows 7 SP1 or later.

o The SDK provides an experimental runtime installer for end-user system installation.

The installer is a silent installer designed to be integrated as part of the application

package. Please report any issues or request additional features.

o The SDK includes two usability sample applications under the demo directory:

The GlassBalls sample application shows how to use shadow hands to position

and interact with physical objects.

The HeadCoupledPerspective sample application shows how to use face

detection to provide a different view perspective in 3D object visualization.



o Nuance* speech recognition and speech synthesis. Download the required voice

add-on package (non-redistributable and for development only) from

http://www.intel.com/software/perceptual. The following samples demonstrate the

usages:

voice_recognition Command control and dictation in C++

voice_recognition.cs Command control and dictation in C#

voice_synthesis Speech synthesis in C++

o Depthmap and the Creative Senz 3D* camera I/O module. See the depth_smoothing

sample for comparison of raw and smoothed depthmap streams.

o Improved finger tracking and gesture recognition for tracking stability and

pose/gesture recognition accuracy.

o Improved framework and game engine support for multithreaded applications.

o Improved privacy notification tools for better visualization of applications that access

to the I/O devices.



o Color, depth and audio stream capturing; file recording and playback.

o Finger tracking and pose/gesture recognition.

o Face tracking and facial landmark tracking.

o 2D/3D markerless object tracking.

o This release provides the following experimental features for feedbacks:

Privacy monitoring utility that visualizes certain data access related to personal

identifiable information.

http://www.intel.com/software/perceptual

Depth stream filtering capability in the depth_smoothing sample.

System Requirements

Hardware

2nd and 3rd generation IA-32 or Intel® 64 Intel® Core™ architecture processors (with Intel® AVX support).

1GB free hard disk space.

The Creative Senz 3D* camera (firmware version 2028 or later.)

The Creative Senz 3D* Interactive Gesture camera KIT is meant for application developers to use

with the Perceptual Computing SDK 2013. The Creative Senz 3D* Interactive Gesture camera KIT may

have known defects and errata which will be published along the product.

USB-2 or USB-3 port.

Software

Microsoft Windows 7 (with SP1) or Microsoft Windows 8.

Microsoft Visual Studio C++ 2008, 2010 or 2012 with Service Pack 1 or later versions.

Microsoft .NET* 4.0 framework for C# development.

SoftKinetic* DepthSense* SDK runtime 1.0.4 or later.

Unity PRO 3.5.1f2 or later for Unity 3D development.

Processing 1.5.1 or 2.0 beta7 for Processing framework development.

Java JDK 1.7.0_11 for Java development.

openFrameworks* v0071 or later for openFrameworks development.

Nuance* Dragon Assistant* Core Engine 1.1.7 or later & Nuance Language Pack 1.1.1 or later.

For development purpose, these Nuance components are a part of the Intel® Perceptual

Computing SDK developer package. There is no redistribution rights granted for these.

These Nuance components are available only on qualified platforms. The application developer can negotiate directly with Nuance for redistribution rights on other platforms.

Havok* Vision* SDK v2012.2.1 or later for Havok development.

Installation

Remove any previous SDK packages. Cleanup destination directory if needed.

Run the SDK web-based or off-line installer also from http://www.intel.com/software/perceptual.

Reboot your computer. This step is critical to ensure environmental parameters

propagate to all applications.

API Change Logs

Change with Release 12492 (GOLD R7):

Added pxcpulseestimator.h

Added pxcemotion.h


None


None


Added the following device properties. See the SDK reference manuals for details. o DEVICE_PROPERTY_DEPTH_UNIT

o DEVICE_PROPERTY_CAMERA_MODEL

Added the following functions to UtilCapture and UtilPipeline for directly accessing

color formats: o UtilCapture::QueryImageByFormat

o UtilPipeline::QueryImageByFormat

Added property info and deprecated property imageInfo in C# PXCMImage. The

property imageInfo is not CLS compliant.

Added property info and deprecated property audioInfo in C# PXCMAudio. The

property audioInfo is not CLS compliant.

Added Util[M]Capture::SetFilter(PXC[M]Capture::DeviceInfo) for locating a

specific device.

Added the grammar field in PXC[M]VoiceRecognition::Recognition to distinguish

between different grammars in the command and control mode.

Added the confidence field in PXC[M]VoiceRecognition::ProfileInfo to set

confidence threshold for voice recognition. The argument threshold in the

PXC[M]VoiceRecognition::SubscribeRecognition function is deprecated.


None

http://www.intel.com/software/perceptual


A new API for Background Removal pxcsegmentation.h has been added. See

reference manual for details


The face attributes detection API PXCFaceAnalysis::Attribute is deprecated in C++

and C#. Applications compiled with Gold release are still compatible with Gold R2.

The voice parameter is deprecated in the PXCVoiceSynthesis::ProfileInfo interface.

Currently there is only a single voice (some male and some female depending on the

language setting) for each language.

The following C# functions are added to help access the ImageData buffers. The

PXCMImage.QueryBitmap function is deprecated (but still available).

o PXCMImage.ImageData.ToByteArray

o PXCMImage.ImageData.ToShortArray

o PXCMImage.ImageData.ToFloatArray

o PXCMImage.ImageData.ToBitmap

o PXCMImage.ImageData.ToWriteableBitmap

Added ProcessAudioEOS() in the C++/C# voice interface to signal end of stream,

replacing the C# function ProcessAudioAsync(), which is deprecated but still available.

The QueryLandmarkData function in Processing relaxes the requirement that the array

structure be fully initialized. The red line below is now optional.

PXCMFaceAnalysis.Landmark.LandmarkData[] data=new

PXCMFaceAnalysis.Landmark.LandmarkData[6];

// for (int i=0;i<data.length;i++) data[i]=new

PXCMFaceLandmark.Landmark.LandmarkData();

pp.QueryLandmarkData(face_id,PXCMFaceAnalysis.Landmark.LABEL_6POINTS,da

ta);

The following functions are added to the framework support:

o ProjectImageToRealWorld

o ProjectRealWorldToImage

o MapDepthToColorCoordinates

o SetVoiceLanguage

The following enhancements are made to access the 60fps QVGA depth stream:

o The C++ UtilCapture::SetFilter or C# UtilMCapture.SetFilter

functions can now set filters to specify size and frame rates.

o Unity, Processing and Java programs can use the mode DEPTH_QVGA_60FPS to

request the 60fps depth stream.

The Unity API adds QueryRGB(Int32[]) to retrieve raw RGB data in an integer array.

The PXCMCapture.AudioStream.ReadStreamAsync function protocol is fixed as follows:

pxcmStatus ReadStreamAsync(out PXCMAudio audio, out PXCMScheduler.SyncPoint sp);

The previously definition mistakenly used “ref” on the audio parameter.

Change with Release 7383 (Gold):

None

Change with Release 6765 (Beta3):

The new PXCProjection interface exposes functions to map RGB, depth and world

coordinates. The Creative Senz 3D* I/O module exposes such interface. See the sdkmanual-core.pdf for details.

The SDK session implements the PXCMetaData interface to provide centralized

configuration storage. The application can derive the PXCMetaData instance from a

PXCSession instance.

The PXCMetadata interface exposes additional functions, AttachSerializable and

CreateSerializable to save or recreate serialization instances such as the

PXCProjection interface.

The module instance creation function CreateInstance in the PXCSessionService::

DLLExportTable structure is extended to take more input parameters for future

extension. The old interface is deprecated but still supported in this release.

The UtilPipeline utility classes added the following functions to start/stop modular

operations: PauseFaceLocation, PauseFaceLandmark, PauseGesture,

PauseVoiceRecognition.

The SetVoiceCommands function in the UtilPipeline utility classes is redesigned to take

a list of voice commands as input. The application can use this function during

recognition to change the active grammar.

The SetVoiceDictation function is added in the UtilPipeline utility classes to set the

dictation mode.

The Unity and Processing framework functions are aligned on behaviors for better

scalability. See the sdkmanual-framework.pdf for details.

Change with Release 5831 (Beta2):

Introduce the new PXCProjection interface which exposes the following methods

PROPERTY_DEPTH_CONFIDENCE_

THRESHOLD Set the depthmap flooring threshold.

PROPERTY_DEPTH_SMOOTHING Set the depthmap smoothing mode.

See sdkmanual-core.pdf for details.

The PXCCapture::VideoStream::DataDesc::DeviceCap structure is deprecated and

redefined as PXCCapture::Device::DeviceCap. See sdkmanual-core.pdf for details.

The application no longer needs to derive from the PXCBaseImpl template to implement

any callback functions. The application can implement the callback functions by directly

deriving from the corresponding callback interfaces. This change does not impact backward compatibility. See sdkmanual-core.pdf for details.

The PXCGesture interface exposes a parameter to configure the activation distance in

the PXCGesture::ProfileInfo structure. The default value is 40 centimeters. Objects

within this distance to the camera trigger hand tracking. It is however not recommended

that the application changes the default value, unless absolutely necessary. Different activation distances in different applications confuse users. See sdkmanual-gesture.pdf

for details. The PXCSmartPtr template exposes a member function IsValid for testing whether the

internal pointer is valid or not. See sdkmanual-core.pdf for details.

The depthmap and vertices visualization is improved in the PXCImage::AcquireAccess

function not to use any confidence values. This change reduces flickering. The depthmap and vertices are now visualized as gray images. See sdkmanual-core.pdf for details.

The UtilCapture utility class expands the SetFilter set of functions to preset device

properties. The function sets the device properties before delivering any device

properties to the algorithm modules. See sdkmanual-core.pdf for details.

The UtilCapture utility class expands the LocateStreams set of functions to search for

an I/O device that can serve both audio and video module data streaming needs. See

sdkmanual-core.pdf for details.

The UtilCaptureFile utility class now supports playing back WAV files. See


The UtilPipeline utility class is completely rewritten to be modular for supporting

more modules. The current implementation consists of UtilPipelineRaw,

UtilPipelineFace, UtilPipelineGesture, and UtilPipelineVoice that cover raw

stream capture, face detection and location, finger tracking, and voice recognition. The

UtilPipeline utility class exposes all member functions in above modular classes. See


The PXCUPipeline set of functions and all framework and game engine wrappers are

rewritten to be thread-safe. See sdkmanual-framework.pdf for details.

Changes with Release 5057 (Beta1):

This is the first publicly available release of the Intel® Perceptual Computing SDK 2013

Beta.

Known Issues

This section captures the currently known issues, which are targeted to be fixed in

subsequent GOLD update releases.

The Creative Senz 3D* camera, camera driver, and the I/O module

Issue Description Release

Lower depth and/or color frame rate may be observed when running some Creative cameras > 8 hours

12492 (Gold 7)

A saturated first frame (black frame) may be randomly encountered when recording

with gesture viewer (seen only on a few cameras)

9302

(Gold 3)

The camera’s automatic gain leveling can’t be disabled. 8779 (Gold 2)

The camera’s white balance settings are limited to only three settings. 8779 (Gold 2)

The camera’s balance auto-leveling settings persist after application close. 8779 (Gold 2)

White balance auto-leveling settings persist after the application closes. 8779 (Gold 2)

One application setting the RGB resolution for the camera may prevent another application simultaneously accessing the RGB with a different resolution

8779 (Gold 2)

The MapColorCoordinatesToDepth function of the PXCProjection interface is not

implemented.

6765 (Beta3)

The camera may not be able to stream data at the same time as any integrated

camera, if they are on the same USB bus, due to the USB bandwidth issue.

5057

(Beta1)

The camera and the camera driver do not yet support system suspension. The driver may occasionally hang or crash. The camera laser may be kept on during system suspension. To work around, set the computer in the high-performance mode and use only the AC mode.

5057 (Beta1)

The camera driver may not get any valid color or depth samples if an SDK application tries to access to the camera, when the camera has already been exclusively occupied by any Microsoft Media Foundation or DirectShow* capturing filters.

5057 (Beta1)

There is a slight delay every 10-12 frames in retrieving color and depth streams, due to buffering in the USB software stack. The RGB and depth streams are not

gen-locked. A future version of the camera will resolve this issue. The color and depth pictures are generally in sync within 3 frames.

5057 (Beta1)

The I/O capturing module cannot serve more than a single user on a multiple user system. Multiple accesses from applications of the same user are allowed.

5057 (Beta1)

If the developer enables C++ Exceptions in the Microsoft Visual Studio debugging 5057

mode, the application may receive first chance exceptions from the I/O capturing

module during the module startup or shutdown.

Workaround: disable the C++ exception (DebugExceptionsC++ Exceptions under

Microsoft Visual Studio 2010).

(Beta1)

The applications may take some time between 5 and 30 seconds to resume after the computer is resumed from the sleep/hibernate state. This timing is platform specific.

5057 (Beta1)

The camera is not compliant with the USB 2.0 High Speed Certification. A new revision of the camera is required to solve this issue.

5057 (Beta1)

The camera does not support the snapshot mode for the depth stream. A future version of the camera will resolve this issue.

5057 (Beta1)

When the camera is detected and then plugged out and back in within a period of less

than 5 seconds, the detection may fail to happen. To work around, wait for more than 5 seconds before plugging the camera back in.

5057

(Beta1)

Video recording is not working with Amcap* when recording in uncompressed mode. When starting the recording, the preview window is black. The speed of the recorded movie is four times slower. When using Amcap but recording in the .wmv mode, the preview windows is still black, but the recorded movie can be played back correctly. The same issue is not detected with other webcam recording tools.

5057 (Beta1)

The reflectivity and opacity of some materials as seen by the 3D camera can be different than seen in the visual spectrum. To work around, do not visualize depth map or confidence map.

5057 (Beta1)

Other I/O modules


The Microsoft Media Foundation* camera module does not support dynamic profile change. The application should avoid calling the SetProfile function more than once.

5057 (Beta1)

Gesture module


Possible increase in memory consumption when moving hand and fingers extremely close to the camera for extended periods of time.

8779 (Gold 2)

Incorrect geonodes reported for peace pose. The peace pose is correctly requiring that the index and middle finger be present in order to be identified but the geonodes

being reported as active during a peace pose are always the thumb and some other

finger (often the index finger).

8779 (Gold 2)

Finger tracking module


The alert LABEL_FOV_BLOCKED will not work when depthmap data is filtered. The SDK

by default filters depthmap data for better quality. To work around, set the device

5831

property PROPERTY_DEPTH_SMOOTHING to false to disable depthmap data smoothing. (Beta2)

The finger tracking module does not distinguish left and right hands. Use LABEL_BODY_HAND_PRIMARY/LABEL_BODY_HAND_SECONDARY for identifying the

first/second tracked hand, and the side field of the data structure to determine the

sidedness of the hand.

5057

(Beta1)

The finger tracking module does not label fingers accurately (i.e. thumb, index, middle, ring and pinky), unless all fingers are visible at least once to the camera.

5057 (Beta1)

The finger tracking module treats two merged fingers as one and tracks the middle of the two fingers.

5057 (Beta1)

The finger tracking module will confuse hand(s) and head if they are closer than 8 inches. To work around, design your application scenarios so that hands are naturally

away from one’s head at least 8 inches.

5057 (Beta1)

The CPU utilization may peak at 50% on 2nd generation Intel® Core™ processors. The average CPU utilization should be below 10%.

5057 (Beta1)

Face tracking module


When face is detected, and the face is moved up slowly (or the camera angle down) until the top of the face comes into contact with the top edge - the detected region becomes highly unstable. The detection rectangle flickers, and re-acquires about

every frame. This happens to a lesser degree against the edges/bottom of the detection area.

9870 (Gold 4)

VIEW_ANGLE_FRONTAL_ROLL option not working for face detector – was only an experimental feature. Plans to deprecate it. Contact SDK team via developer Forum in case this is needed for your app.

9870 (Gold 4)

Landmark detection too jittery to be used for gaming 8779 (Gold 2)

Facial detection very jittery at times even on still faces 8779 (Gold 2)

Face detection drops fairly frequently even when user is just sitting normally 8779 (Gold 2)

Face landmark detection noise from frame to frame is higher in some cases 8779 (Gold 2)

Head tracking too inaccurate for game use 8779

(Gold 2)

Head-coupled display sample is too jittery and unstable sometimes 8779 (Gold 2)

Face recognition component not adequate for face login 8779 (Gold 2)

Facial landmark detection rate drops when head is tilted more than ~10% left/right 8779 (Gold 2)

Facial recognition and detection rates drop when user is wearing glasses 8779 (Gold 2)

Landmark detection in the face tracking module does not support pose estimation. The function QueryPoseData always return PXC_STATUS_ITEM_UNAVAILABLE.

5831 (Beta2)

The face tracking module will lose tracking if the object moves too fast in front of the camera. The algorithm is designed for still or slow-motion image sequences.

5057 (Beta1)

Face size does not update as subject moves close to and further from camera when using the face_detection.exe sample. This effect is more evident when streaming

with higher resolutions.

6765 (Beta3)

Face analysis may perform very slowly at higher resolutions (1920x1080) (~10FPS) 6765 (Beta3)

Landmark detection doesn’t respond well to open mouths. With open mouth, you may see landmark detector switch mouth corner nodes to nose.

6765 (Beta3)

Motion/jitter correction threshold may be set too high when streaming high resolution. 6765 (Beta3)

If a face to be detected is tilted upward, the face_detection sample may not identify

the face to be the same person and rapidly update face identifier.

6765 (Beta3)

Nuance* voice recognition/synthesis modules


Each language model for dictation needs >500M memory. As the number of simultaneous dictation apps increase, the number of instances of the language model in memory too increases, thereby increasing the memory footprint.

12029 (Gold 6)

Slow voice dictation recognition may be observed when running for the first time after fresh install

10898 (Gold 5)

Voice dictation quality for other languages is not as good as US English 9302

(Gold 3)

If recording volume/level set too high or too low then voice recognition rates may

significantly drop. Refer to the Voice Best Known Methods in the documentation to tune tis for improved recognition.

8779 (Gold 2)

The voice recognition module supports multiword grammars when working in the

command and control mode. The application must set the endOfSentence

configuration parameter longer than the expected silence duration between words.

5831

(Beta2)

The voice recognition module recognizes short sentence up to 30 seconds when working in the dictation mode. To work around, always dictate sentences shorter than 30 seconds.

5831 (Beta2)

The voice synthesis module supports only a female or male voice output (varies by

language). The configuration parameter to choose different voices is ignored. 5831 (Beta2)

Total Immersion* object tracking module


Open D'Fusion Studio Select Menu -> ? -> Help

You will get error message stating help pdf file can't be found. But, the file in included in the packaging.

13842

(Gold 8)

The D’fusion studio tool switches between the classic and lite mode depending

on how it is started. The tool appears to start in lite mode by default but if a

new session is selected, the tool switches to classic mode. There is no flag in

the tool to switch between the modes. The outcome may change if the targets

are trained in the wrong mode.

12029

(Gold 6)

The Total Immersion Studio application may show motion glitches when

visualizing rotating objects in front of the Creative Camera. 10898

(Gold 5)

The object tracking module does not track very well with reflective materials or under

poor lighting conditions.

7383

(Gold)

Pulse Estimator:


The Algorithm imposes the following restrictions:

Keep as still as possible, and face the camera fully. Excess motion can disrupt estimation.

Keep a light source in front of you. Bright lights behind you may interfere with detection.

If possible, disable your camera's automatic white balance and exposure settings.

Experiment with profile settings until you find a combination that works well for you.

13842

(Gold 8)

Utilities and tools:


During recording, the UtilCaptureFile utility caches all images in memory until they

are successfully written to the disk. In some cases, the application may see constant memory increase if the disk I/O speed cannot catch up with the image recording speed. To work around, record only a small number of frames or use faster SSD disks.

9870 (GOLD R5)

The sdk_info (camera_info in beta2) cannot reliably retrieve information while other

applications are accessing the camera. Run this tool without any other applications

accessing the camera.

5057

(Beta1)

The sdk_info (camera_info in beta2) may not retrieve the TOF controller version and

the USB backend version reliably in some cases. In such case, the TOF controller version shows 65535. Use the rescan button to rescan the device to retrieve the right values.

5057 (Beta1)

Language ports, frameworks and game engines


Java process is still running in the background if the user clicks the Processing IDE stop

button (as opposite to closing the Processing rendering window.) This is a known Processing issue but not a SDK defect. The user should close the Processing rendering window.

6765

(Beta3)

Installer/Uninstaller

Description Release

DepthSenseSDK doesn't uninstall automatically – need manual removal from Add/Remove programs

12492 (Gold 7)

Random failures of language pack installation. This seems to happen sometimes when

all languages are selected for install. A reinstall works fine. Issue is being investigated. 9870

(Gold 4)

Installer may fail with the following error. Module Name DepthSense SDK 1.2.1.840 Win64-VS2010-Release redist signed.exe. Application returned Error Code 2. This is a Depth Sense SDK driver installer issue and is being looked into by SoftKinetic.

9302 (Gold 3)

Installation of SDK package without DA language pack can be accomplished by

following these steps: Deselect the DA Core will cause language packs to be deselected. Selecting DA Core again will cause it to be selected alone.

8779

(Gold 2)

Installation may fail if common registry keys are locked. A system reboot may be required.

8779 (Gold 2)

The web-based installer does not support proxy authentication for all existing

methods. If the installer doesn’t work for your proxy settings by default, you can try

the Internet Explorer settings by adding the “--system-engine-download” switch

(without quotes) to the installer command line, or use the offline installer.

7383

(Gold)

Text on splash screen could be cut off for Medium / Larger font system settings 7883 (Gold)

The installer does not remove registry keys with settings created by Unity demos 7883

(Gold)

The uninstaller does not remove user created files and registry keys. Remove them manually if needed.

5057 (Beta1)

Samples / Demos


These sample/demo apps are not compile-able under VS 2012 Express: capture_viewer, Remote Control.

These samples require the Microsoft MFC. 9870 (Gold 4)

RemoteContol demo does nothing when using any sensor (e.g.: voice and gesture) for

executing any action on "explorer" group (tried windows navigation and key press) on Ultrabook™ system with Win8 x64. The same steps on notebook with Win7 x64 works as expected. If any_active_application group option or non group application name like "My Computer" (explorer) is chosen, it works. Fails only group named "explorer"

9870 (Gold

4)

Head Coupled Perspective demo shows solid color instead of picture when the head is at the top

9870 (Gold 4)

The following message may appear when running a console application: “WARNING: Invalid Operation Exception: The main event loop is not running.” This message is harmless.

8779 (Gold 2)

Simple module vs2008 solution generates simple_module.exe instead of simple_module_d.exe when compiling in debug mode.

8779 (Gold 2)

The GlassBalls and AugmentedFarm demo applications may exhibit high CPU

utilization up to 75% under the DC mode.

7383 (Gold)

The AugmentedFarm demo requires Unity Pro 4.x or higher 7383 (Gold)

The HeadCoupledDisplay demo application may momentarily run slowly or pause on

some systems when moving the head too quickly

6765 (Beta3)

SDK Core


Illegal instruction issue may be observed on CPU which do not support Intel® Advanced Vector Extensions 13842 (Gold 8)

PerC SDK Application may see a memory leak if it creates and deletes PXCSession object many times during

execution, the leak is about <10Kb per session 12029 (Gold 6)

The following issues are fixed since 13842 (Gold R8)

The Creative Senz 3D* camera, camera driver, and the I/O module


A Timeout exception may be seen if Creative camera left plugged in for extended periods of time

13842 (Gold 8)

GUI samples Menu->Device does not work if multiple depthsense cameras are

connected -- only the first camera in list streams 13842

(Gold 8)

Older cameras show an offset in the alignment which affects the uvmapping and

projection mapping. This can be alleviated by adjusting the offset using the calibration tool in this release. However, If we move color principal point, some area of the color frame could lack depth data.

9870

(Gold 4)

When converting RGB24 to RGB32, the A value always hardcoded to 128 9302 (Gold 3)

If accessed through Microsoft Media Foundation*, the camera exposes two color streams. The second stream is not playable. (It does not deliver any data sample.) The SDK temporarily disables the second stream in the SDK Media Foundation I/O module

7383 (Gold)

and in the capture_viewer tool. Recommended to always access the camera through

the SDK capture module.

Setting device property DEVICE_PROPERTY_COLOR_GAMMA always returns

PXC_STATUS_DEVICE_FAILED.

5831

(Beta2)

When using a laptop in balanced power mode running on battery, the detection of the camera on the USB port may fail or only detect partial features. To work around, set the power option to the performance mode.

5057 (Beta1)

The camera depth stream may interfere with the color stream in low-light conditions, when the application retrieves both color and depth streams. There will be horizontal strips in the color images. To work around, use the camera in normal-light conditions, or use only the color or depth stream.

5057 (Beta1)

The camera LED indicator is partially working. A future version of the camera will solve

this issue.

5057

(Beta1)

The camera only supports audio sampling at 44.1KHz. Other sample rate may not work properly. To work around, always use the 44.1KHz sampling rate.

5057 (Beta1)

The RGB device properties such as brightness, hue and exposure are read only. Future improvement will enable setting these parameters.

5057 (Beta1)

32-bit applications and 64-bit applications cannot stream color/depth data at the same time. The camera data can be shared among 32-bit applications or 64-bit applications.

5057 (Beta1)

Finger tracking module


Incorrect geonode coordinates (duplicates in the coordinates fixed) 9302 (Gold 3)

When two hands are overlapping, the finger tracking module may incorrectly calculate the palm center position to be close to the forearm position. To reduce impact, design

use cases where two hands never overlap.

5057 (Beta1)

The position of the grabbing points (LABEL_HAND_UPPER, LABEL_HAND_MIDDLE,

LABEL_HAND_LOWER) may vary frame by frame.

5057 (Beta1)

If the Creative Senz 3D* camera is positioned on a table, the finger tracking module

may mistake the table or any additional unrelated objects as valid hands. To avoid this, always put the camera on top of the laptop lid. This is the only designed usage position.

5057 (Beta1)

The pose LABEL_POSE_THUMB_UP is not reliably recognized if performed outside the

camera’s tracking activation zone, about 40cm away from the camera. To work around, first move inside the activation zone so that the SDK module starts to track the fingers, then perform the thumb up pose.

5057 (Beta1)

Face Analysis

While setting face feature profiles on a different thread, the app may run into a race condition and experience

a crash 9302

(Gold 3)

Background Segmentation/Removal Module:

Older cameras show an offset in the alignment which affects the uvmapping and projection mapping. This

affects the quality of the segmented image. This will result in the segmented object being incorrectly offset

from the original image when displayed. This issue can be addressed by using the Calibration tool in the R4

release of the SDK.

However, If we move color principal point, some area of the color frame could lack depth

data.

9870 (Gold 4)

Nuance Voice Recognition/Synthesis module:


When using two instances of voice recognition sample in dictation mode, recognition results may not be identical

13842

(Gold 8)

Playback of long recorded files in 32bit app may freeze. The workaround is to record

by specifying the number of frames in the sample app. The playback works fine with 64 bit version of apps.

12029

(Gold 6)

Voice recognition[cs] samples in dictation mode do not report fails if someone else uses dictation (module busy).

12029 (Gold 6)

The Nuance core engine works in single instance of Dictation. If the application creates

multiple instances of the voice recognition (in Dictation) or voice synthesis module, only the first instance will function. Voice recognition and voice synthesis can function at the same time.

12029

(Gold 6)

Voice Recognition application may crash when there are multiple pipelines or multiple sessions of voice recognition

9302 (Gold 3)

The voice_recognition.cs sample may fail to return to a functional state after a few

times of resuming from S3/S4 power states. 6765 (Beta3)

The recognition rate of the voice recognition module may drop on systems in full load. To achieve the best recognition results, always use voice recognition when the CPU utilization is below 50%.

5831

(Beta2)

The voice recognition or synthesis module may hang an application if the user presses CTRL-C (CTRL-BREAK) to exit the application. To work around, do not interrupt the

Nuance module process.

5831 (Beta2)

The voice_recognition module may pop up a DOS window during exit. 6765 (Beta3)

Starting and stopping the voice recognition module may take up to 30 seconds. 5831

(Beta2)

The recognition latency may increase if the application delivers a large audio buffer at input. For responsiveness, it is recommended that the application streams audio data with an interval of 24-40 milliseconds.

5831 (Beta2)

The voice recognition module leaks memory. For about an hour, the memory increases from 11 MB to 40 MB.

5831 (Beta2)

Total Immersion* Object Tracking Module:


Possible unhandled exception may occur during a user callback when running DFusionStudioCV_IntelPCSDK.exe. This seems to occur when using an integrated

webcam instead of the Creative Senz 3D* Camera.

6765 (Beta3)

Utilities and Tools:


The trace feature in sdk_info is functional in Gold R6. The tool now shows preview of tracing controls and tracing information can be viewed as well.

12029 (Gold 6)

The Privacy Notification Tool (PNT) does not display info about applications that were launched prior to the launch of the PNT.

7383 (Gold)

The PNT may not correctly display the displaying activities in the capture_viewer tool. 6765 (Beta3)

The UtilCaptureFile recording utility implementation currently has a 2GB limitation

on 32-bit systems.

5057 (Beta1)

The source under sample/common is prebuilt in libpxcutils.lib against Microsoft

Visual Studio 2008 and Microsoft Visual Studio 2010 projects. For other compiler versions, the developer will have to rebuild them against the source.

5057 (Beta1)

The PNT must be launched with run as administrator privilege so it can insert itself in

the taskbar tray.

5057 (Beta1)

Language ports, frameworks and game engines


The precompiled library ipppxcu_l.lib (used to support Unity, Processing and other frameworks) lacked the

Intel® AVX2 code path, which resulted in crash on Intel® Third Generation Intel processors. The fix added

the Intel® AVX2 code path.

9302

(Gold 3)

Java process is still running in the background if the user closes the Processing IDE window (as opposite to closing the Processing rendering window.) This is a known issue but not a SDK defect. The user should close the Processing rendering window

then the Processing IDE window.

5057 (Beta1)

The Unity sample function PXCUPipeline.Init is hard-coded to initialize only once.

This is necessary as the Unity engine calls the function multiple times during

initialization. To change to a different configuration, restart Unity.

5057 (Beta1)

Installer/Uninstaller:

Description Release

The SDK installer does not correctly detect Microsoft Windows XP 64-bit as an unsupported OS. During installation, some SDK components will refuse to install. To

6765 (Beta3)

work around, do not install SDK on Microsoft Windows XP.

The Intel64 runtime may not be installed by developer package if newer version of

IA32 runtime exists on system.

6765

(Beta3)

Uninstallation of developer package may not work if newer runtime is installed on top of it.

6765 (Beta3)

In certain cases, the SDK installer/uninstaller may not be able to cleanly exit the

privacy notification utility, which runs as a taskbar service. The installation or uninstallation process will terminate. To work around, manually exit the privacy notification utility before installing a new SDK version or uninstalling an existing SDK version.

5831

(Beta2)

To fully uninstall the Nuance Dragon Assistant Core Recognition Service and Dragon Assistant Language Data modules, you must select each one ands right click uninstall

in the Program and Features application in the control panel or you can just double click the intel_pc_sdk_voice_ia32_11.exe installer

5831 (Beta2)

Samples and Demos:


camera_viewer sample description is documented in CHM and PDF, is yet

needs to be added in HTML documentation. 13842 (Gold 8)

Compile error may be seen in gesture_viewer and other samples under VS2012, need to reference windows.h

instead of afxres.h for addressing this. PCSDK (Gold 6)

Capture Viewer app hangs while streaming from more than one camera. 9302 (Gold 3)

The installer package missed including prebuilt executable for the AugmentedFarm

demo and the GlassBalls demo. To work around, run the demos in the Unity Pro

editor 3.5.1 (the GlassBalls demo) and 4.1 (the AugmentedFarm demo.)

7383 (Gold)

The audio_recorder.exe sample may crash when unplugging USB camera

w/microphone during live streaming.

6765 (Beta3)

The depth_smoothing.exe sample may crash when unplugging USB camera

w/microphone during live streaming.

6765 (Beta3)

Possible memory leaks when face and landmark detection samples. The rate may vary from 5MB-15MB per hour of working set memory.

6765 (Beta3)

The gesture_viewer.exe sample may detect hand points on the face particularly if the

face is too close hands.

6765

(Beta3)

Intel® Perceptual Computing SDK 2013 GOLD Release Notes

Documents

Transcript of Intel® Perceptual Computing SDK 2013 GOLD Release Notes