STEM and Virtual RealityA/Professor Tomasz Bednarz

QUT & CSIRO Data61

Prepared for QUT’s VC STEM Camp 2016

https://immersivelifeblog.files.wordpress.com/2015/04/vr_history.jpg

Mobile Virtual Reality on track to hit $861M in revenues in 2016

360-degree Panoramas

Immersive 360-degree images and videos

• Viewer can look around, observe different views

Amazon!

Approx. 35 hours

from Brisbane – Sydney –

Santiago, Chile –

Lima, Peru –

to Pucallpa, Peru

Approx. 12 hours

Pucallpa – Camito, Amazon, Peru

Technology to capture panoramas

360-degree cameras: Freedom 360 Explorer+ with 6x GoPro Hero 4

installed, 2x Ricoh Theta M, and 2x Kodak SP360.

Stereo camera: Dual Hero System with 2x GoPro Hero 3+ installed.

IR camera: Reconyx XR6 Ultrafire (infra-red, motion sensing

trap camera).

Sound: Zoom H2n Handy Recorder, Olympus VN-733PC.

Virtual Reality: Samsung GearVR, Samsung Galaxy S6, Google

Cardboard Glasses.

USB Hub: ORICO 10 Port-Hub.

GPS: Garmin 64ST GPS System.

Taking 360-degree photos

Processing with Autopano Giga, Autopano Video Pro

• To stitch one 360-degree panoramic image:• Drag and drop six images into Autopano Giga

Processing

Lightroom

Photoshop

After Effects

Premiere

Processed

Priceless!Beautiful & Beautiful Scenery

Warmth of village people, curious eyes and laugh of children

Daily canoeing in sunrise and sunset time

Time to truly engaging to wild life conservation issue

True power of VR in Conservation

• Essential aspects of VR technology are:• Presence

• Immersion

• Interactivity

• Empathy

[Chris Milk: How virtual reality can create the ultimate empathy machine]

VR technology can be used to provide people experience of wild life creating better connection between human and natures

Reaching Through the Looking Glass | Craig James15 |

Firing the Imagination

Milgram’s Reality-Virtuality continuum

The area between the completely real and completely virtual, consists of both augmented reality, where the virtual augments the real, and augmented virtuality, where the real augmented the virtual.

P. Milgram and A.F. Kishino, Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.

Virtual Reality

Hand gesturesto operate VR

• Enhance e-Learning

• Experience interactions = remember more

• Test in class-room situations

Use of iPhone’s touch screenImmersive e-book

Introduction to Unity

Unity as cross-platform game engine

Unity setup

• Download from https://unity3d.com/get-unity/download

Create new project

• Specify project name

• Specify project location

• Specify whether project is 3D or 2D

• Specify asset packages (pre-made content)

User Interface

Project Window

Hierarchy Window

Inspector Window

Toolbar

Scene View

Creating scenes

• GameObjects• Every object in your scene is a

GameObject• They are containers (for instance 3D

object, texture, animation)

• Object properties can be viewed and edited in Inspector

• Position of GameObjects, cameras, lights can be seen and adjusted in Scene view

• Interactions are added using Scripts attached to objects

Asset Workflow

• Asset from outside of Unity: a 3D model, an audio file, an image, or any of the other types of file that Unity supports.

• Asset created within Unity: an Animator Controller, an Audio Mixer, a Render Texture.

• For instance to add 3D model:• Add file to Assets folder in Project• Drag mesh from Project View into

Hierarchy or Scene View

GameObjects

• GameObject contain Components, for instance Transform, Camera, etc. (seen from the Inspector)

Positioning GameObjects

New Project

Change layout (Windows Layouts Tall)

Moving main camera

• Select Main Camera, select Translate Icon, Move camera

Creating immersive 360-degree viewer

• Create new project

• Add the Cardboard plugin VR SDKhttps://developers.google.com/vr/unity/

• Add 360-degree image asset

• Create a Skybox

• Add to the Scene

• Publish on the Android devicehttps://developer.android.com

Google Cardboard – see next slides

• Open Unity and create new 3D project

• Import the Unity package: Assets Import Packages Custom Package.• Select the GoogleVRForUnity package where you downloaded it and click on

Open.

• Check all the boxes and click Import.

• You can test whether it works by opening Assets GoogleVR DemoScenes

HeadsetDemo DemoScene.• Press Play (you should see a red cube).

• Press the Control key on your keyboard and move your mouse around to tilt the view. Pressing Alt will cause too pan around the view of the VR.

Import Google VR

• Import the Unity package: Assets Import Packages Custom Package.• Select the GoogleVRForUnity package where downloaded, and click on Open.

• Check all the boxes (uncheck iOS) and click Import.

Demo (Assets Google VR DemoScenes HeadsetDemo)

• Press Play (should see a red cube, or green when navigated to it)• Press the Control key and move mouse to tilt your view (inside SceneView)• Press the Alt key and move mouse to pan around (inside SceneView)

Adding Cardboard to Empty Project

• Drag GvrViewerMain from Assets GoogleVR Prefab to Hierarchy• Click on Play, should render stereo VR view (navigate using mouse and Control or Alt keys)

Import panoramic image (Disneyland, why not?)

Import panoramic image (Disneyland, why not?)

• Create folder Textures in Assets• Drag and drop 360-disney.jpg to Assets Textures• Click on the imported image: Inspector will show you properties of the texture

Change texture type

• Change Texture Type from Texture to Cubemap• Change Mapping o Latitude-Longitude Layout (Cylindrical)• Click on Apply

Create Skybox material

• Create folder Materials in Assets, and click right mouse button in that folder and Create Material• Name material Sky01 and click in inspector on Select• In Select Cubemap choose 360-disney earlier created

Create Skybox

• In Main Menu click on Window Lightening• Drag Sky01 material into Skybox field• Background of SceneView should now display panoramic image in proper projection

Install on Android

Change Bundle Identifier to something like au.com.qut.helloworld

Pushing to device

Thank youNow try it yourself :-D