Gaze detection with Virtual Reality and Unity 3d

13
Interactive Gaze Detection with VR & Unity Dr James Birt

Transcript of Gaze detection with Virtual Reality and Unity 3d

Page 1: Gaze detection with Virtual Reality and Unity 3d

Interactive Gaze Detection with VR & Unity

Dr James Birt

Page 2: Gaze detection with Virtual Reality and Unity 3d

Requirements

• Unity 5.3 or higher

• Oculus runtime 0.8 installed

• Introductory VR in Unity 3d

Page 3: Gaze detection with Virtual Reality and Unity 3d

Gaze Detection• Detecting where the user is looking is very

important in VR

• This allows a user to interact with an object, trigger an animation etc

• Currently VR is limited – without eye tracking to accurately determine a users gaze

• We therefore need to estimate the user’s gaze

• This is typically determined by a mid point between the users two eyes (or the centre of the camera)

• In a program like Unity we would cast a ray forward from this centre point and find what the ray has collided with

Page 4: Gaze detection with Virtual Reality and Unity 3d

The Reticle

• Gains the name from a sighting device such as a telescopic lens

• Can help to indicate the center of the user’s vision

• The style of the reticlecould be a simple dot, or perhaps a crosshair

4

Page 5: Gaze detection with Virtual Reality and Unity 3d

Interactive Reticle

• Follow my Introductory VR in Unity 3d tutorial

• OR final package can be downloaded here

• Remember to set your build settings and quality settings as highlighted in tutorial

• Import the Reticle package - this has been modified from the Unity3d Interaction in VR sample scenes - http://tinyurl.com/z9cc8w8

5

Page 6: Gaze detection with Virtual Reality and Unity 3d

Creating a Reticle Camera

6

• Add MainCamera Prefab as a child to FPSController

• Copy the Transform details from FirstPersonCharacter to MainCamera

• Delete FirstPersonCharacter• Switch off Return to Main Menu Script

on MainCamera – Error will occur

Page 7: Gaze detection with Virtual Reality and Unity 3d

Test the Reticle by adding multiple objects to your scene

7

• Notice that your Reticle scales correctly to your objects

• Appears correctly in front and on top of the objects in your scene

Page 8: Gaze detection with Virtual Reality and Unity 3d

Interactive Objects

• Try dragging an Interactive Object into the scene from the Prefab folder

• You will notice the VR Interactive Item & Example Interactive Item Script(s) attached see what happens when you Gaze over this object

8

Page 9: Gaze detection with Virtual Reality and Unity 3d

9

Page 10: Gaze detection with Virtual Reality and Unity 3d

VRInteractiveItem.cs

10

Sets Boolean conditions if these Action events take place and calls the corresponding function

Page 11: Gaze detection with Virtual Reality and Unity 3d

ExampleInteractiveItem.cs

11

Here we are handling the outcome of the Events – You can do whatever you want in these functions the script(s) just needs to be placed on a gameObject

Page 12: Gaze detection with Virtual Reality and Unity 3d

Testing multiple objects and events

12

• When adding a new interactive gameObjectmake sure to include the VR Interactive Item Script and Example Interactive Item Script

• You need to add the Object to the Interactive Item and Renderer

• Notice we can have multiple instances of the objects and scripts and we know which object is being triggered at any time

Page 13: Gaze detection with Virtual Reality and Unity 3d

Final Scene

https://dl.dropboxusercontent.com/u/9011716/VRRETICLEDEMO.unitypackage