Unity & AR

Lab 07

Renato MainettiJacopo Essenziale

renato.mainetti@unimi.itjacopo.essenziale@unimi.it

From reality to VR(passing by AR)

2

video see-through deviceHUD

HMDNothing

Several AR Libraries:

3

How does AR typically works: (wiki)

• A key measure of AR systems is how realistically they integrate augmentations with the real world. The software must derive real world coordinates, independent from the camera, from camera images. That process is called image registration which uses different methods of computer vision, mostly related to video tracking.

4

Feature image recognition

5

Usually those methods consist of two parts:• The first stage is to detect interest points, fiducial markers or optical flow in the

camera images. This step can use feature detection methods like corner detection, blob detection, edge detection or thresholding and/or other image processing methods.

• The second stage restores a real world coordinate system from the data obtained in the first stage. Some methods assume objects with known geometry (or fiducial markers) are present in the scene. Mathematical methods used in the second stage include projective (epipolar) geometry, geometric algebra, rotation representation with exponential map, kalman and particle filters, nonlinear optimization, robust statistics.

Vuforia:

6

Hololens, Vuforia & Unity

7

Vuforia’s support for HoloLens is implemented in version 6.1 of the Vuforia Unity extension

Deploy to:

• Andorid

• iOs

• …

• Hololens

8

Hololens

9

Weight: 579 gramsCPU: quad-core Intel Atom x5-Z8100 1,04 GHz, GPU & HPU (Holographic Processing Unit)

Ram: 2 GB of RAMDisk: 64 GBBattery: 16.500 mWh (2 hours)

Audio surround (spatial sound)MemsLight sensor4 cameras (ambient analisys)1 Depht cam1 cam HD (2 megapixel) photo and videos.4 mic (speech recognition)

Hololens:

10

Vuforia target

11

Setup:• Create an account

• Register your app and obtain a valid key

• Add some image to the db and evaluate features

• Download the db

• Integrate in Unity importing the unity package

• Configure Vuforia with the app key and activate the dbimported

• Add an ARcamera prefab

• Add an imageTarget prefab, specifying the target

• Parent an object to the imageTarget

• Play

12

Texturing in blender:How to cheat the brain…

13

Change the light color in unity (match the natural colour of the image target) to obtain the best result.

Demo scenes and game prototype:

14