Luis Cataldi & Andy Hess - Epic Games

MAKINGVR GAMES & EXPERIENCES

IN UNREAL ENGINE

VR Learning Resources in the Learn Tab

VR Learning Resources in the Learn Tab

On-ramp to exploring VR in Unreal Engine

VR Learning Resources in the Learn Tab

The latest VR info and news is most often in the Release Notes - Link

New Features

Performance Improvements

Platform Additions

Bug Fixes

Enhancements

Changes

VR Learning Resources for Unreal Engine:Docs:

• Getting Started With VR• UE4 VR Index Page• VR Best Practices• VR Cheat Sheets• Oculus Quick Starts• GearVR Quick Starts

VR Learning Resources for Unreal Engine:Video:

• Integrating the Oculus Rift into UE4 • UE4 Support Stream - Developing for VR • 2015 UE4 - VR and Unreal Engine• Unreal Engine 4 Training Streams

VR Learning Resources for Unreal Engine:Presentations:

• Nick and Nick – Going Off the Rails: The Making of Bullet Train

• Lessons from Integrating the Oculus Rift into UE4

Links:• Tom Looman’s - Getting Started with VR in Unreal Engine 4• Sam Deiter - 10 VR tips for Unreal Engine

Education Community VR for UE4:

Mitchell McCaffrey’s - Mitch VR LabsMitch's VR Lab - an Introduction

Mitch's VR Lab - Look Based interaction

Mitch's VR Lab - Simple Teleportation Mechanic

Mitch's VR Lab - Introduction to SteamVR

Mitch's VR Lab - Simple Head IK

Mitch’s UE4 Forum Post

Education Community VR for UE4:

Carlos Coronado - VR Olive FPS ControllerCarlos’s UE4 Forum Post

Olive VR Locomotion: Movement

Olive VR Locomotion: Shooting

Olive VR Locomotion: Menus

Let’s take a look at Carlos’s Look Based Locomotion for Annie Amber

Before we get much deeper into Unreal Engine...

What are Mitch, Carlos, (and we) solving for?

One of the biggest issues for working in VR is Motion/Simulation Sickness.

How is it caused?

en.wikipedia.org/wiki/Virtual_reality_sickness

Sensory conflict theory believes that sickness will occur

when a user's perception of self-motion is based on

incongruent sensory inputs from the visual system,

vestibular system, and non-vestibular proprioceptors, and

particularly so when these inputs are at odds with the user's

expectation based on prior experience.

Five typical causes of Motion/Simulation Sickness in VR

Read more about it

1. Non-forward movements• No unnatural movements

2. Awareness of Vection• When a large part of the visual field moves, a viewer feels like

he has moved and that the world is stationary

3. The feeling of accelerations

4. Too much camera YAW

5. Helped by adding a static reference frame

Education Community Tips to Reduce Motion/Simulation Sickness

Extra Credits - Simulation Sickness

Offpeak Games - 5 Design Techniques to Reduce Simulator Sickness

GDC - Designing to Minimize Simulation Sickness in VR Games

VR Best Practices, Eliminating Motion Sickness - Power of Play 2015

Education Community VR for UE4:

Mitchell McCaffrey’s - Mitch’s VR Game Template

Jun 2014 UE Forum Post of VR Game Templates

Space Shooter Template

First Person Template

UE4 VR Locomotion Techniques

Teleport Mechanic by Mitchell McCaffrey

Mitch's VR Lab Ep03 - UE4 - Simple Teleportation Mechanic

Mitch's VR Lab Ep04 - UE4 - Advanced Teleportation Mechanic

UE4 VR Locomotion Techniques

Look -Based Locomotion/Interaction by Carlos Coronado

MIND: Path to Thalamus in the UE4 and VR. Annie Amber

Things we CAN DO in Unreal Engine to improve VR Games and Experiences

You MUST maintain framerate

For the VR experience to feel smooth, your game needs to run on 75 hz (Oculus DK2) or even 90 hz. (HTC Vive and Oculus CV1) depending on the device. To see the current framerate type in “stat fps” or “stat unit” (for more detailed breakdown) in your console when running the game.

Use UE4’s VR Performance Profiling Tools

To capture a single frame with GPU timings press Ctrl+Shift+, or type in “profilegpu” in the console. This command dumps accurate timings of the GPU, you will find that certain processes are a heavy burden on the framerate (Ambient Occlusion is one common example) when using VR.

The GPU Profiling & Performance and Profiling docs are a good place to learn about profiling your game.

VR Instanced Stereo Can HelpThe latest 4.11 release introduces Instanced Stereo Rendering, check the video below for a comparison video of how that works.

“Basically, we’re utilizing hardware instancing to draw both eyes simultaneously with a single draw call and pass through the render loop. This cuts down render thread CPU time significantly and also improves GPU performance. Bullet Train was seeing ~15 – 20% CPU improvement on the render thread and ~7 – 10% improvement on the GPU.” – Ryan Vance.

To enable this feature in 4.11 and above, go to your Project Settings and look for “Instanced Stereo” under the Rendering category.

Disable Heavy Post-ProcessorsDue to the demanding requirements of VR many of the advanced Post Processing features that you normally use should be disabled. To accomplish this you will need to do the following in your level.

•Add a Post Process(PP) volume to your level if there is not already one there.•Select the PP volume and in the Post Process Volume section enable the Unbound option so that the settings in the PP volume will be applied to the entire level.

•Expand the Settings of the Post Process Volume and then go through each section and disable any active PP settings by enabling that property by clicking on it and then set the value from the default, usually 1.0, to 0 to disable the feature.

•When doing this you will not need to hit every section and set all the properties to 0. Instead first disable the really heavy hitting features like Lens Flares, Screen Space reflections, Temporal AA, SSAO, and anything else that might have an impact on performance.•While a lot of the features are disabled by setting things in your .INI this ensures that nothing will happen to performance if someone deletes the .INI by mistake.

Things to keep in at the front of your mind:

LOD's and aggressive culling are a must to ensure that you

are hitting your VR performance targets.

Known issues and possible workarounds:

Parallax Mapping  

Parallax mapping takes Normal mapping to the next level by accounting for

depth cues, Normal mapping does not. A Parallax mapping shader can better

display depth information, making objects appear to have more detail than

they do. This is because no matter what angle you look at, a Parallax map will

always correct itself to show you the correct depth information from that view

point. The best use of a Parallax map would be for cobblestone pathways and

fine detail on surfaces.

UE4 – Lighting for VRDimmer lights & colors can help reduce simulation sickness.

Use Static Lighting over Stationary or Dynamic.

Make sure your Stationary / Dynamic Lights do not overlap.

Baked lighting is the best option for VR.

If using Dynamic Shadows only have one shadowing light.

Use Stat LightRendering to see current lighting cost.

Profile, Profile, Profile

Fake shadows Wherever You Can!!Using things like fake blob shadow drop to simulate dynamic shadows are a good general rule in order to keep VR project running at frame.

Blob shadow example. Image by Eric Chadwick

UE4 – Effects for VRMesh based VFX work the best for VR.

Camera Facing particles do not hold up well in VR on their own.

The Dither Temporal AA Material Function can make Opacity masked objects look

like Translucent ones.

Local Space rotation does not look correct in VR.

UE4 – Environments for VR

Reflection probes instead of screen space reflections.

Again… Texture Blob shadows are a cheap alternative to dynamic shadows.

The ** Merge Actor Tool ** can help cut down on Static Mesh draw call

without having to do work outside of UE4.

Some very important things we all need to know about Unreal Engine.

The Unreal Engine Framework

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

The GameMode is the definition of

the game.

● It should include things like

the game rules and win

conditions.

● It also holds important

information about:

○ Pawn

○ PlayerContoller

○ GameState

○ PlayerState

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

The Pawn class is the base class

of all Actors that can be controlled

by players or AI.

● The Pawn represents the

physical location, rotation,

etc. of a player or entity

within the game.

● A Character is a special type

of Pawn that has the ability to

walk around.

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

A PlayerController is the interface between the Pawn and the human player controlling it.

● The PlayerController decides what to do and then issues commands to the Pawn (e.g. "start crouching", "jump").

● Putting input handling or other functionality into the PlayerController is often necessary.

● The PlayerController persists throughout the game, while the Pawn can be transient.

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

The GameInstance is a class who’s

state persists switching of levels,

game modes, pawns etc. Where

classes like GameMode or

PlayerController are being reset

and data stored in those classes is

removed.

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

A PlayerState is the state of a

participant in the game, such as a

human player or a bot that is

simulating a player. Non-player AI

that exists as part of the game

would not have a PlayerState.

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

The GameState contains the state

of the game, which could include

things like the list of connected

players, the score, where the

pieces are in a chess game, or the

list of what missions you have

completed in an open world game.

The Unreal Engine Framework

GameInstance

GameMode

Pawn Class

HUD Class

PlayerController Class

GameState Class

PlayerState Class

The HUD is the base object for

displaying elements overlaid on the

screen. Every human-controlled

player in the game has their own

instance of the AHUD class which

draws to their individual Viewport.

Object

Actor

Pawn

Character

Base building blocks in the Unreal Engine

Any object that can be placed into a level

Subclass of Actor and serve as an in-game avatar

Subclass of a Pawn that is intended to be used as a player character

CharacterMovementComponent

Pawn

Character

Default Pawn

Wheeled Vehicle

CapsuleComponent

SkeletalMeshComponent

Etc.

VehicleMovementComponent

SkeletalMeshComponent

PhysicsHandle

Etc.

SpectatorPawn

DefaultPawnMovementComponent

StaticMeshComponent

CollisionComponent, Etc.

Controller1-to-1 Relationship

How about programming interactivity for VR?

Programming VR Interaction with Blueprints

Blueprints in Unreal Engine is a complete visual scripting system based on the concept of using a node-based interface to create interactions from within Unreal Editor.

Programming VR Interaction with Blueprints

Learning Blueprints through Content Examples

Hey!! We need AUDIO for VR too!!

UE4 – Audio for VRAmbient Sound Actors in VR

Ambient Sound Actor can be used for

many purposes such as ambient

looping sounds and non-looping

sounds. Generally, the Ambient

Sound Actor conforms to the real

world where the closer you are to a

sound, the louder it will appear.

UE4 – Audio for VRSound Properties

You can assign a sound asset from

the Details panel by selecting an

asset from the Sound settings drop-

down menu or by highlighting a

sound asset in the Content Browser

and clicking the button.

UE4 – Audio for VRAttenuation Properties

Attenuation is the ability of a sound

to decrease in volume as the player

moves away from it.

It is advisable to use Sound

Attenuation objects whenever

possible, if for no other reason than

to give broad control over the

settings for many Actors.

UE4 – Audio for VRNew: Stereo Spatialization

3D spatialization is now possible for

stereo audio assets.

The 3D Stereo spread parameter

defines the distance in game units

between the left and right channels

and along a vector perpendicular to

the listener-emitter vector.

UE4 – Audio for VRAudio Volume

Audio Volumes allow you to control

and apply various sounds in your

level as well as provide an avenue

to create compartmentalized audio

zones where you can control what

is heard inside and outside of the

volume.

Complete state of the art suite of AI Tools.

Additional toolsets in Unreal Engine to enhance VR and bring life to your experiences.

Additional toolsets in Unreal Engine to enhance VR and bring life to your experiences.

Complete set of tools for Animation Retargeting

So What’s Next?

Go download Unreal and start making your game today!