Cardboard VR: Building Low Cost VR Experiences

112
CARDBOARD VR: BUILDING LOW COST VR EXPERIENCES Mark Billinghurst [email protected] April 14 th 2016 CHIuXiD

Transcript of Cardboard VR: Building Low Cost VR Experiences

Page 1: Cardboard VR: Building Low Cost VR Experiences

CARDBOARD VR: BUILDING LOW COST VR EXPERIENCES

Mark Billinghurst [email protected]

April 14th 2016 CHIuXiD

Page 2: Cardboard VR: Building Low Cost VR Experiences

What You Will Learn • Virtual Reality Fundamentals + History • How to Build Cardboard VR Viewer • Example Cardboard VR Applications • Basics of Unity Programming • How to Make Panorama VR Applications • How to Create VR Scenes • How to Add Interactivity to VR Applications • Cardboard Design guidelines

Page 3: Cardboard VR: Building Low Cost VR Experiences

VIRTUAL REALITY

Page 4: Cardboard VR: Building Low Cost VR Experiences

Virtual Reality

Computer generated multi-sensory simulation of an artificial environment that is interactive and immersive.

Page 5: Cardboard VR: Building Low Cost VR Experiences
Page 6: Cardboard VR: Building Low Cost VR Experiences

David Zeltzer’s AIP Cube

! Autonomy – User can to react to events and stimuli.

! Interaction – User can interact with objects and environment.

! Presence – User feels immersed through sensory input and output channels

Interaction

Autonomy

Presence

VR

Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators & Virtual Environments, 1(1), 127-132.

Page 7: Cardboard VR: Building Low Cost VR Experiences

Key Technologies • Autonomy

• Head tracking, body input •  Intelligent systems

•  Interaction • User input devices, HCI

• Presence • Graphics/audio/multisensory output • Multisensory displays

•  Visual, audio, haptic, olfactory, etc

Page 8: Cardboard VR: Building Low Cost VR Experiences

Early Experimenters (1950’s – 80’s)

Helig 1956

Sutherland 1965

Furness 1970’s

Page 9: Cardboard VR: Building Low Cost VR Experiences

The First Wave (1980’s – 90’s)

NASA 1989 VPL 1990’s

Virtuality 1990’s

Page 10: Cardboard VR: Building Low Cost VR Experiences

Jaron Lanier

•  Founded VPL, coined term “Virtual Reality”

Page 11: Cardboard VR: Building Low Cost VR Experiences

Desktop VR - 1995 • Expensive - $150,000+ •  2 million polys/sec • VGA HMD – 30 Hz • Magnetic tracking

Page 12: Cardboard VR: Building Low Cost VR Experiences

Second Wave (2010 - ) • Palmer Luckey

• HMD hacker • Mixed Reality Lab (MxR)

• Oculus Rift (2011 - ) •  2012 - $2.4 million kickstarter •  2014 - $2B acquisition FaceBook •  $350 USD, 110o FOV

Page 13: Cardboard VR: Building Low Cost VR Experiences

•  sddg

Page 14: Cardboard VR: Building Low Cost VR Experiences

Oculus Rift

Sony Morpheus

HTC/Valve Vive

2016 - Rise of Consumer HMDs

Page 15: Cardboard VR: Building Low Cost VR Experiences

Desktop VR 2016 • Graphics Desktop

•  $1,500 USD •  >4 Billion poly/sec

• $600 HMD •  1080x1200, 90Hz

• Optical tracking • Room scale

Page 16: Cardboard VR: Building Low Cost VR Experiences

https://immersivelifeblog.files.wordpress.com/2015/04/vr_history.jpg

Page 17: Cardboard VR: Building Low Cost VR Experiences

Market Size

Page 18: Cardboard VR: Building Low Cost VR Experiences

Computer Based vs. Mobile VR

Page 19: Cardboard VR: Building Low Cost VR Experiences

Mobile VR

CPU: 300 Mhz HDD; 9GB RAM: 512 mb Camera: VGA 30fps Graphics: 500K poly/sec

1998: SGI O2 2008: Nokia N95

CPU: 332 Mhz HDD; 8GB RAM: 128 mb Camera: VGA 30 fps Graphics: 2m poly/sec

Page 20: Cardboard VR: Building Low Cost VR Experiences

Mobile Phone AR & VR • Mobile Phone AR

• Mobile phone • Live camera view • Senor input (GPS, compass)

• Mobile Phone VR • Mobile phone • Senor input (compass) • Additional VR viewer

Page 21: Cardboard VR: Building Low Cost VR Experiences

VR2GO (2013)

• MxR Lab •  3D print VR viewer for mobiles • Open source hardware + software •  http://projects.ict.usc.edu/mxr/diy/vr2go/

Page 22: Cardboard VR: Building Low Cost VR Experiences

Multiple Mobile VR Viewers Available

Page 23: Cardboard VR: Building Low Cost VR Experiences

•  zxcvz

Page 24: Cardboard VR: Building Low Cost VR Experiences

CARDBOARD VR

Page 25: Cardboard VR: Building Low Cost VR Experiences

•  dsfsaf

Page 26: Cardboard VR: Building Low Cost VR Experiences

Google Cardboard

• Released 2014 (Google 20% project) • >5 million shipped/given away • Easy to use developer tools

+ =

Page 27: Cardboard VR: Building Low Cost VR Experiences

Cardboard ($2)

Lenses ($10)

Magnets ($6)

Velcro ($3)

Rubber Band (1¢)

Software

Components

Page 28: Cardboard VR: Building Low Cost VR Experiences

Assembling the Cardboard Viewer

Page 29: Cardboard VR: Building Low Cost VR Experiences

Version 1.0 vs Version 2.0

• Version 1.0 – Android focused, magnetic switch, small phone • Version 2.0 – Touch input, iOS/Android, fits many phones

Page 30: Cardboard VR: Building Low Cost VR Experiences

Many Different Cardboard Viewers

Page 31: Cardboard VR: Building Low Cost VR Experiences

SAMPLE CARDBOARD APPLICATIONS

Page 32: Cardboard VR: Building Low Cost VR Experiences

Cardboard App • 7 default experiences

• Earth: Fly on Google Earth

•  Tour Guide: Visit sites with guides

• YouTube: Watch popular videos

• Exhibit: Examine cultural artifacts

• Photo Sphere: Immersive photos

• Street View: Drive along a street

• Windy Day: Interactive short story

Page 33: Cardboard VR: Building Low Cost VR Experiences

100’s of Google Play Cardboard apps

Page 34: Cardboard VR: Building Low Cost VR Experiences

Sample Applications

Page 35: Cardboard VR: Building Low Cost VR Experiences

Cardboard Camera

• Capture 360 panoramas • Stitch together images on phone • View in VR on Cardboard

Page 36: Cardboard VR: Building Low Cost VR Experiences

Google Expeditions

• Teacher led VR experiences • https://www.google.com/edu/expeditions/

Page 37: Cardboard VR: Building Low Cost VR Experiences

Building Your Own Application • Cardboard Viewer

•  https://www.google.com/get/cardboard/

• Smart phone • Android/iOS

• Cardboard SDK •  iOS, Android, Unity •  https://developers.google.com/cardboard/

• Unity game engine (optional) •  https://unity3d.com

• Content

Page 38: Cardboard VR: Building Low Cost VR Experiences

Cardboard SDK

Features:1.  Lensdistor-oncorrec-on.2.  Headtracking.3.  3Dcalibra-on.4.  Side-by-siderendering.5.  Stereogeometryconfigura-on.

6.  Userinputeventhandling.

Unity Cardboard SDK

Page 39: Cardboard VR: Building Low Cost VR Experiences

INTRODUCTION TO UNITY

Page 40: Cardboard VR: Building Low Cost VR Experiences
Page 41: Cardboard VR: Building Low Cost VR Experiences

Unity 3D Game Editor

Page 42: Cardboard VR: Building Low Cost VR Experiences

SETUP

Page 43: Cardboard VR: Building Low Cost VR Experiences

Download and Install • Go to unity3d.com/download • Use Download Assistant – pick components you want

Page 44: Cardboard VR: Building Low Cost VR Experiences

Getting Started •  First time running Unity you’ll be asked to create a project • Specify project name and location • Can pick asset packages (pre-made content)

Page 45: Cardboard VR: Building Low Cost VR Experiences

Unity Interface •  Toolbar, Scene, Hierarchy, Project, Inspector

Page 46: Cardboard VR: Building Low Cost VR Experiences

Customizable Interface

Page 47: Cardboard VR: Building Low Cost VR Experiences

Building Scenes • Use GameObjects:

• Containers that hold different components •  Eg 3D model, texture, animation

• Use Inspector • View and edit object properties and other settings

• Use Scene View • Position objects, camera, lights, other GameObjects etc

• Scripting • Adding interaction, user input, events, etc

Page 48: Cardboard VR: Building Low Cost VR Experiences

GameObjects • Every object in Scene is a GameObject • GameObjects contain Components

•  Eg Transform Component, Camera Component

Page 49: Cardboard VR: Building Low Cost VR Experiences

Adding 3D Content

• Create 3D asset using modeling package, or download •  Fbx, Obj file format for 3D models

• Add file to Assets folder in Project • When project opened 3D model added to Project View • Drag mesh from Project View into Hierarchy or Scene View

•  Creates a game object

Page 50: Cardboard VR: Building Low Cost VR Experiences

Positioning/Scaling Objects

• Click on object and choose transform

Page 51: Cardboard VR: Building Low Cost VR Experiences

Unity Asset Store

• Download thousands models, scripts, animations, etc •  https://www.assetstore.unity3d.com/

Page 52: Cardboard VR: Building Low Cost VR Experiences

UNITY BASICS

Page 53: Cardboard VR: Building Low Cost VR Experiences

Making a Simple Scene 1.  Create New Project 2.  Create Game Object 3.  Moving main camera position 4.  Adding lights 5.  Adding more objects 6.  Adding physics 7.  Changing object materials 8.  Adding script behaviour

Page 54: Cardboard VR: Building Low Cost VR Experiences

CreateProject

• Create new folder and project

Page 55: Cardboard VR: Building Low Cost VR Experiences

New Empty Project

Page 56: Cardboard VR: Building Low Cost VR Experiences

Create GameObject

•  Load a Sphere into the scene • GameObject -> 3D Object -> Sphere

Page 57: Cardboard VR: Building Low Cost VR Experiences

Moving main camera

• Select Main Camera • Select translate icon • Move camera

Page 58: Cardboard VR: Building Low Cost VR Experiences

Add Light

• GameObject -> Light -> Directional Light • Use inspector to modify light properties (colour, intensity)

Page 59: Cardboard VR: Building Low Cost VR Experiences

Add Physics

•  Select Sphere •  Add Rigidbody component

•  Add Component -> Physics -> RigidBody •  or Component -> Physics -> RigidBody

•  Modify inspector properties (mass, drag, etc)

Page 60: Cardboard VR: Building Low Cost VR Experiences

Add More Objects

• Add several cubes •  GameObject -> 3D Object – Cube

• Move cube • Add Rigid Body component (uncheck gravity)

Page 61: Cardboard VR: Building Low Cost VR Experiences

Add Material

• Assets -> Create -> Material • Click Albedo colour box in inspector • Select colour • Drag asset onto object to apply

Page 62: Cardboard VR: Building Low Cost VR Experiences

Add Script

• Assets -> Create -> C# script • Edit script using Mono • Drag script onto Game Object

Page 63: Cardboard VR: Building Low Cost VR Experiences

Example C# Script GameObject Rotation

using UnityEngine; using System.Collections; public class spin : MonoBehaviour {     // Use this for initialization     void Start () {         }         // Update is called once per frame     void Update () {         this.gameObject.transform.Rotate(Vector3.up*10);     } } #

Page 64: Cardboard VR: Building Low Cost VR Experiences

Scripting C# Unity 3D •  void Awake():

•  Is called when the first scene is loaded and the game object is active

•  void Start(): •  Called on first frame update

•  void FixedUpdate(): •  Called before physics calculations are made

•  void Update(): •  Called every frame before rendering

•  void LateUpdate(): •  Once per frame after update finished

Page 65: Cardboard VR: Building Low Cost VR Experiences

Final Spinning Cube Scene

Page 66: Cardboard VR: Building Low Cost VR Experiences

Resources • Unity Main site

• http://www.unity3d.com/ • Holistic Development with Unity

• http://holistic3d.com • Official Unity Tutorials

• http://unity3d.com/learn/tutorials • Unity Coder Blog

• http://unitycoder.com

Page 67: Cardboard VR: Building Low Cost VR Experiences

IMMERSIVE PANORAMAS

Page 68: Cardboard VR: Building Low Cost VR Experiences

Steps 1.  Create a new project 2.  Load the Cardboard SDK 3.  Load a panorama image asset 4.  Create a Skymap 5.  Add to VR scene 6.  Deploy to mobile phone

Page 69: Cardboard VR: Building Low Cost VR Experiences

New Project

Page 70: Cardboard VR: Building Low Cost VR Experiences

Load Cardboard SDK

• Assets -> Import Package -> Custom Package •  Navigate to CardboardSDKForUnity.unitypackage

• Uncheck iOS (for Android build)

Page 71: Cardboard VR: Building Low Cost VR Experiences

Load Cardboard Main Camera

• Drag CardboardMain prefab into Hierarchy •  Assets -> Cardboard -> Prefab

• Delete CameraMain

Page 72: Cardboard VR: Building Low Cost VR Experiences

Panorama Image Asset

•  Find/create suitable panorama image •  Ideally 2K or higher resolution image

• Google “Panorama Image Cubemap”

Page 73: Cardboard VR: Building Low Cost VR Experiences

Add Image Asset to Project • Assets -> Import Asset

•  Select desired image

• Set Texture Type to Cubemap

• Set mapping to Latitude-Longitude (Cylindrical)

Page 74: Cardboard VR: Building Low Cost VR Experiences

Create Skybox Material

• Assets -> Create -> Material • Name material • Set Shader to Skybox -> Cubemap • Drag texture to cubemap

Page 75: Cardboard VR: Building Low Cost VR Experiences

Create Skybox • Window -> Lighting • Drag Skybox material into

Skypebox form

Page 76: Cardboard VR: Building Low Cost VR Experiences

Panorama Image in Unity

Page 77: Cardboard VR: Building Low Cost VR Experiences

One Last Thing..

• CardboardMain -> Head -> Main Camera • Set Clear Flags to Skybox

Page 78: Cardboard VR: Building Low Cost VR Experiences

Test It Out

• Hit play, use alt/option key + mouse to look around

Page 79: Cardboard VR: Building Low Cost VR Experiences

Deploy to Mobile (Android) 1.  Plug phone into USB

• make sure device in debug mode 2.  Set correct build settings 3.  Player settings

• Other settings • Set Bundle Idenitfier -> com.Company.ProductName

• Resolution and Presentation • Default Orientation -> Landscape Left

4.  Build and run

Page 80: Cardboard VR: Building Low Cost VR Experiences

Deploying to Phone 1.  Plug phone into USB 2.  Open Build Settings 3.  Change Target platform to Android 4.  Resolution and Presentation

•  Default Orientation -> Landscape Left

5.  Under Player Settings •  Edit Bundle Identifier – eg com.UniSA.cubeTest •  Minimum API level

6.  Build and Run •  Select .apk file name

Page 81: Cardboard VR: Building Low Cost VR Experiences

Running on Phone

• Droid@Screen View on Desktop

Page 82: Cardboard VR: Building Low Cost VR Experiences

CREATING 3D ENVIRONMENTS

Page 83: Cardboard VR: Building Low Cost VR Experiences

Key Steps 1.  Creating a new project 2.  Load Cardboard SDK 3.  Replace camera with CardboardMain 4.  Loading in 3D asset packages 5.  Loading a SkyDome 6.  Adding a plane floor

Page 84: Cardboard VR: Building Low Cost VR Experiences

New Project

• Camera replaced with CameraMain

Page 85: Cardboard VR: Building Low Cost VR Experiences

Download Model Package

• Magic Lamp from 3dFoin • Search on Asset store

Page 86: Cardboard VR: Building Low Cost VR Experiences

Load Asset + Add to Scene

• Assets -> Import Package -> Custom Package •  Look for MagicLamp.unitypackage (If not installed already)

• Drag MagicLamp_LOD0 to Hierarchy • Position and rotate

Page 87: Cardboard VR: Building Low Cost VR Experiences

Import SkySphere package

• SkySphere Volume1 on Asset store

Page 88: Cardboard VR: Building Low Cost VR Experiences

Add SkySphere to Scene

• Drag Skyball_WithoutCap into Hierarchy •  SkySphere_V1 -> Meshes

• Rotate and Scale as needed

Page 89: Cardboard VR: Building Low Cost VR Experiences

Add Ground Plane

• GameObject -> 3D Object -> Plane • Set Scale X to 2.0, Z to 2.0

Page 90: Cardboard VR: Building Low Cost VR Experiences

Testing View

• Use alt/option key plus mouse to rotate view

Page 91: Cardboard VR: Building Low Cost VR Experiences

Adding More Assets

•  Load from Asset store – look for free assets

Page 92: Cardboard VR: Building Low Cost VR Experiences

ADDING INTERACTIVITY

Page 93: Cardboard VR: Building Low Cost VR Experiences

Adding Movement Goal: Move in direction user looking when Cardboard Button pressed.

• Key Steps 1.  Start with static screen 2.  Create movement script 3.  Add movement script to Camera head 4.  Deploy to mobile

Page 94: Cardboard VR: Building Low Cost VR Experiences

Static Scene

Page 95: Cardboard VR: Building Low Cost VR Experiences

Create Movement Script • Add new script object

•  Assets -> Create -> C# Script

• Edit script in Mono

Page 96: Cardboard VR: Building Low Cost VR Experiences

Add Script to Scene

• Drag Script onto Head object •  CameraboardMain -> Head

• Uncheck Track Position Box • Adjust movement speed

Page 97: Cardboard VR: Building Low Cost VR Experiences

Adding More Interactivity

•  Load Cardboard Demo application •  Assets -> Import Package -> Custom Package •  Load CardboardDemoForUnity.unitypackage

•  Launch Demo Scene •  Assets -> Cardboard -> DemoScene

Page 98: Cardboard VR: Building Low Cost VR Experiences

Features Shown

• Gaze reticle + selection • Viewpoint teleportation • Menu panel overlay • Audio feedback • Event system

Page 99: Cardboard VR: Building Low Cost VR Experiences

DESIGN GUIDELINES

Page 100: Cardboard VR: Building Low Cost VR Experiences

Google Design Guidelines

• Google’s Guidelines for good VR experiences: • Physiological Considerations •  Interactive Patterns

•  Setup •  Controls •  Feedback •  Display Reticle

•  From http://www.google.com/design/spec-vr/designing-for-google-cardboard/a-new-dimension.html

Page 101: Cardboard VR: Building Low Cost VR Experiences

Physiological Considerations • Factors to Consider

• Head tracking • User control of movement • Use constant velocity • Grounding with fixed objects • Brightness changes

Page 102: Cardboard VR: Building Low Cost VR Experiences

Interactive Patterns - Setup • Setup factors to consider:

• Entering and exiting • Headset adaptation • Full Screen mode • API calls •  Indicating VR apps

Page 103: Cardboard VR: Building Low Cost VR Experiences

Interactive Patterns - Controls

• Use fuze buttons for selection in VR

Page 104: Cardboard VR: Building Low Cost VR Experiences

Interactive Patterns - Feedback

• Use audio and haptic feedback • Reduce visual overload • Audio alerts •  3D spatial sound • Phone vibrations

Page 105: Cardboard VR: Building Low Cost VR Experiences

Interactive Patterns - Display Reticle

• Easier for users to target objects with a display reticle • Can display reticle only when near target object • Highlight objects (e.g. with light source) that user can target

Page 106: Cardboard VR: Building Low Cost VR Experiences

Cardboard Design Lab Application

• Use Cardboard Design Lab app to explore design ideas

Page 107: Cardboard VR: Building Low Cost VR Experiences

CONCLUSION

Page 108: Cardboard VR: Building Low Cost VR Experiences
Page 109: Cardboard VR: Building Low Cost VR Experiences

Conclusion • Virtual Reality industry starting to boom again • Google Cardboard provides a great entry for VR

• Consumer hardware • Cheap/free viewer •  Industry standard game engine • High quality content • Many tutorials/learning resources

• The time to get started is now!

Page 110: Cardboard VR: Building Low Cost VR Experiences

RESOURCES

Page 111: Cardboard VR: Building Low Cost VR Experiences

Useful Resources • Google Cardboard main page

•  https://www.google.com/get/cardboard/

• Developer Website •  https://www.google.com/get/cardboard/developers/

• Building a VR app for Cardboard •  http://www.sitepoint.com/building-a-google-cardboard-vr-app-in-unity/

• Creating VR game for Cardboard •  http://danielborowski.com/posts/create-a-virtual-reality-game-for-

google-cardboard/

• Moving in VR space •  http://www.instructables.com/id/Prototyping-Interactive-Environments-

in-Virtual-Re/

Page 112: Cardboard VR: Building Low Cost VR Experiences

www.empathiccomputing.org

@marknb00

[email protected]