Team IRALAR Breanna Heidenburg -- Michael Lenisa -- Daniel Wentzel Advisor: Dr. Malinowski.
-
Upload
jesse-green -
Category
Documents
-
view
228 -
download
0
Transcript of Team IRALAR Breanna Heidenburg -- Michael Lenisa -- Daniel Wentzel Advisor: Dr. Malinowski.
Team IRALARBreanna Heidenburg -- Michael Lenisa -- Daniel Wentzel
Advisor: Dr. Malinowski
The Project◦ Why is it important
The Goals◦ System breakdown
Image recognition Point transformation User Interface
The Results
What is our project?
Track a user’s eye and use the information to control a computer cursor
Enhances Human Computer Interaction
◦Speed of use◦Hands-free use
3 Part System◦ Image Processing Application◦ Calibration and Mapping system◦ GUI designed for gaze-based interaction
Systems developed concurrently and independently
Separate Applications at run-time
Hardware and Image Processing Application
Hardware◦ Camera
QuickCam Pro for Notebooks Visible Spectrum Camera
◦ Polarizer Tiffen 25mm polarizing filter Removes glare from eye reflections
◦ Lighting Diffuse LEDs Slightly distracting to the user, but necessary to provide
light for the camera
LitEye LE-500◦ High resolution (SVGA)◦ Color Display◦ Translucent or opaque operation◦ Stationary relative to user’s eye
Real time pupil tracking system◦ Developed in C using OpenCV image processing libraries◦ Traditional image processing and blob tracking
Capabilities◦ Locate and determine center of pupil in image◦ Low light and high reflection environments◦ All eye colors◦ Data logging and static test modes◦ Packaged into self contained Windows installer for easy
deployment onto any computer
The Process◦ Capture Image◦ Extract Red Channel◦ Smooth image◦ Apply binary threshold◦ Locate blobs◦ Reject false positives◦ Determine center of pupil blob◦ Adapt threshold◦ Repeat
Summary◦The Good
Dynamically adapts to changing lighting conditions and eye types
Maintains performance in low-light and specularly noisy conditions
◦The Bad Still relies on Logitech camera drivers Extreme reflections still cause problems
Raw Capture
Pre-Processed Image
Completed Recognition
Examples of performance in poor conditions
Low Light Difficult False Positive
Calibration and point mapping
System for mapping the location of the center of the pupil to a pixel on a computer screen
Reasons◦ Geometry
The eye is not flat but a screen is not
◦ User Customization All eyes are different Everyone wears the HMD differently
◦ User Training Calibration system also acts as a quick tutorial
3 dimensional best fit plane◦ Currently using a 4th degree best fit
Xpix = A1 + Xeye*B1 + Yeye*C1
Ypix = A2 + Xeye*B2 + Yeye*C2
Calibration sub-system determines these coefficients
How do we solve the problem?
◦ Multiple Variable Linear Regression – Least Squares
Y = B0 + B1x1 + … + Bkxk
◦ Uses matrix algebra to obtain a coefficient matrixB[] = (X’X)-1X’Y
Results◦ pixelX = 224.9 + 4.8*eyeX – 2.9*eyeY◦ pixelY = 1612.5 + 2.0*eyeX – 5.6*eyeY
B[] B[] == ((XX’’XX))-1-1XX’’YY
How do we click?◦ Monitor eye movements◦ Identify pauses
◦ When eye position is within a small percentage for a certain amount of time
◦ Currently set at 5 frames (~200 mS)◦ Generally, it takes 230 mS for a hand to click a
mouse.
Communications and Custom GUI interface
Why is it important?◦ Allows multiple processes to communicate◦ Allows relay of time sensitive information
UDP vs. TCP◦ UDP -> ‘Connectionless’◦ TCP -> ‘Connection oriented’
Multi-threading◦ Necessary for running multiple pieces of code in a
single process
Multi-threading issues present
Multi-threading◦ Issues with public variable usage◦ Solutions we are using:
Critical Section Raises thread priority (does not allow for interrupts)
Receive request for
data (over UDP)
Raise Thread priority
Lower Thread Priority
Reply to
request
Read Variable
Custom GUI for Gaze Tracking Applications
Why?◦ Gaze tracking accuracy limited by inherent
properties of human vision system◦ Traditional GUI too small and intrusive for use
with transparent HMD◦ Demonstrate applications of gaze tracking
Achieved via a combination of Hardware and Software
Software:WPF & C#◦ XAML
(eXtensible Application Markup Language)◦ Very similar to XML◦ Uses ‘code-behind’ in a similar style to javascript
‘code-behind’ is in C#
Multiple applications have been created within the interface
Multiple screens for functionality testing◦ even games
Additional ability to minimize interface and interact with external applications is present
Calibration Screen Main Screen
What did our test results show?
Who helped us out?
Northrop Grumman LitEye
Dr. Malinowski and the EE faculty Mr. Mattus & Mr. Schmidt