1

Enabling Mobile Virtual Reality Gaming

Know Your Hardware!

Nizar Romdan

Director of Developer Ecosystem

16th February 2016

2

About ARMApps Processor

Cortex-A5

Cortex-A9Cortex-

A15

Cortex-A7

Mali™-400

Mali-T760

Mali-450

Mali-T604

Cortex-

A17Mali-T720

Mali-V500Mali-

DP500

Power

MgmtCortex-M3

Cortex-M0BluetoothCortex-M3

Cortex-M0

Cellular

ModemCortex®-R4

Cortex-R5

Cortex-R7

WiFiCortex-M4

SIMSecurCore®

SC300™

GPS

Cortex-M3

Cortex-M0

Flash

ControllerCortex-M3

Touchscreen

& Sensor

HubCortex-M0

Cortex-M3

Cortex-M4

Sensor HubCortex-

M0+

Camera

Cortex-M3

3

The ARM Business Model

2-3 yearsARM research

and development

Cost

incurred

2-3

yearsPartner chip

development

License

revenu

e

$

20+ yearsMultiple applications

development and

sales

Royalty

revenu

e

$

4

Real presence scales emotions 10x

The feeling of actually being there

5

VR is not new – People were discussing VR the 90’s

Expensive and bulky hardware

Inadequate technology offered limited graphics capabilities

Virtuality – Sega VR – Nintendo Virtual Boy

VR is now accessible

Emergence of low-cost hardware based on existing platforms

Mobile platforms offering console-like performance offering

more realism

VR is about immersing users into Virtual worlds

All the virtual world is graphically rendered

Introducing VRVirtuality

Sega VR

6

VR Use Cases

Games Experiences Training

Education TV/Film

Advertising

7

ARM Ice Cave VR Demo

Frozen ice cave demo developed in Unity 5.0

High-end graphical showcase in Virtual Reality

Geomerics Enlighten

Reflections, refractions, soft shadows, …

OpenGL ES 3.0

Artwork from AAA art studio (RealtimeUK)

Optimised for Mali-T760 MP8 and above

Implemented camera streaming from GearVRdevice to another device (other people can see what the player is doing)

Alpha version shown at APM 2015

Final at Unite Boston 21st September 2015 Screenshots from Samsung Galaxy S6

8

9

10

11

VR Opportunities Could Hit $30B by 2020 – Likely to be mainly from Gaming & Hardware

12

VR Hardware Landscape Worldwide

13

Virtual Reality Segmentation

Samsung Gear VR Zeiss VR One Google Cardboard

Oculus Rift Sony Morpheus HTC Vive (Steam VR)

Mobile

Desktop/Console

14

Headset based on Smartphone

Partnership between Samsung Mobile & Oculus

First version for Galaxy Note 4 (Mali-T760)

Second version for Galaxy S6 (Mali-T760)

Resolution 1080p – 60 Hz

First VR appstore

Introducing GearVR

15

China VR Headsets – DeePoon Case Study

16

2020 Opportunity in Mobile Computing

Mid-Range

Smartphone

Entry-Level

Smartphones

Premium

Smartphone

650mdevices in 2020

450mdevices in 2020

1,000mdevices in 2020

Laptops

and 2-in-1s

250mdevices in 2020

Tablets

450mdevices in 2020

Source: Gartner and ARM

estimates

17

How Long Until Desktop GPU Compute Is In Mobile?

0

1,000

2,000

3,000

4,000

5,000

6,000

2006 2008 2010 2012 2014 2016 2018

GF

LO

PS

/Se

c

PS3

Xbox 360

Current mobile compute capability very near console

Mobile Today

PS4

XboxOne

18

Stereoscopic Display (via headset) – Providing wearer with a sense of depth

Render left eye and right eye independently

Barrel distortion effect applied to rendered output via post processing

Sensor input for head-tracking – To add realism to the output image

Sensor input for interaction (hands-tracking) – To provide an immersive

experience

How Virtual Reality Works

Left eye

Right eye

Eye gap (IPD)

Left eye Right eye

Lens

Lens

2xCPU

Processing

2xVertex

processing

IncreasedFragment

Processing

19

Virtual Reality Technical Challenges

20

ARM Graphics Developer Centre –

malideveloper.arm.com

Supporting Mali

developers with a

full range of

resources through

one, easy-access

portal

21

• API Trace & Debug

• OpenGL® ES, OpenCL™

• Debug and improve performance at frame level

Mali Graphics Debugger

Mali GPU

- Timeline

- HW Counters

- OpenCL visualizer

ARM DS-5 Streamline

• Analyze shader performance

• Generates binary shaders

• Command line tool

Mali Offline Compiler

• Emulate OpenGL ES 2.0 and 3.0

• Windows and Linux

• Khronos Conformant

OpenGL ES Emulator

• Command line and GUI

• ETC, ETC2, ASTC

• 3D textures

Texture Compression Tool

Integration with partners’ tools

Third Party Tools

Mali GPU Software ToolsPerformance Analysis, Debug, and Software Development

22

ARM Guide to The Unity Developers – Released

Available under http://malideveloper.arm.com/armunityguide

Released at Unite Seattle 2014, and regularly updated and expanded at each key

Unite event

ARM-specific and general advice

Optimization process

Profiling and Debugging

Advanced Graphics Techniques

23

ARM Mali VR SDK v0.1 available now under http://malideveloper.arm.com/resources/sdks/ Tutorial – Introduces the concepts of stereoscopic vision

Fundamentals of stereoscopic rendering Effects of lenses and methods for correcting lens distortion in software Calibration

Sample code Eye separation – Viewing distance – Size of screen – Lens separation

ARM Mali VR SDK v0.2 to be released mid Dec 2015 Support for using OVR_multiview extension Foveated rendering

ARM Mali VR SDK v1.0 planned for Mar16 @ GDC’16 Expose functionality as a user-friendly framework Add support for priority contexts

ARM Mali VR SDK under Development

24

Thank You!

malideveloper.arm.com