Developing Games Using OpenGL® ES

(part I, part II) Baback ElmiehSr. Software Engineer

Dave DurnilEngineering Manager

QUALCOMM Gaming Group & 3D Apps Research

June 2, 2005

2

Part I • Brief overview of using OpenGL ES• Mixing BREW® 2D with 3D APIs• The IJOYSTICK interface• Tweaking your .mif files for BREW 3.x• Demos

Part II• Detail overview of the QX Engine™ technology• OpenGL ES tools available for our platforms• Demos

3

Some OpenGL ES demos- What is possible today -

4

BREW OpenGL ESDevelopment Environment

Programming EnvironmentBREW SDK® simulator

Download to phone

www.brewdeveloper.com

5

Using BREW SDK with OpenGL ES

• In the BREW SDK modules directory:– C:\Program Files\BREW SDK v3.x\Bin\Modules– Must use the OpenGLESExtension.dll

• OpenGLESExtension.dll– QUALCOMM’s software implementation of OpenGL ES– Will work with BREW SDK 2.x,3.x & above

6

3D Gaming Software Architecture

• OpenGL ES APIs directly available to game developers• QX Engine™ APIs provides many gaming features• Highly optimized BREW libraries & extensions• Hardware acceleration available for OpenGL ES

7

BREW Gaming APIs• Current graphics interfaces available:

– IGRAPHICS– ISPRITE– IFONT– IIMAGE– IBITMAP / IDIB– ITRANSFORM– IDISPLAY– IGL (OpenGL ES APIs)– IEGL (standard EGL)

• Other useful APIs for gaming– IMEDIA (video & audio)– ISOUND – IPOSDET (gps)– ISOCKET– ICAMERA– IJOYSTICK

8

Steps for using OpenGL ES APIsInclude the IGL.h wrapper header file#include "IGL.h"

Create an instance of the IEGL interface & initialize the IEGL wrappersif (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLSID_EGL,

(void**)&pMe->m_pIEGL) == SUCCESS)IGL_Init(pMe->m_pIEGL);

elsereturn FALSE;

Create an instance of the IGL interface & initialize the IGL wrappersif (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLSID_GL,

(void**)&pMe->m_pIGL) == SUCCESS)IGL_Init(pMe->m_pIGL);

elsereturn FALSE;

// You can now use the standard OpenGL ES & EGL functions// Ex. glDrawArrays(); eglGetDisplay(EGL_DEFAULT_DISPLAY );

Initialize the gl wrappers

9

The gl.c fileProvides wrappers for the OpenGL ES & EGL standard API functions

For example:GLAPI void APIENTRY glActiveTexture(GLenum texture){

IGL_glActiveTexture(GPIGL,texture);}

GLAPI void APIENTRY glDrawArrays(GLenum mode, GLint first, GLsizeicount)

{IGL_glDrawArrays(GPIGL,mode,first,count);

}

GLAPI void APIENTRY glDrawElements(GLenum mode, GLsizei count, GLenum type, const GLvoid *indices)

{IGL_glDrawElements(GPIGL,mode, count, type, indices);

}

10

Mixing BREW 2D/3D Calls• QUALCOMM is providing an EGL extension for BREW

3.x which returns a pointer to the current color buffer• Typical usage of this extension will be

– combine 2D images on the color buffer, drawing skybox or HUDs for example

– post process the image to add special effects like night vision, etc…

• The extension name is EGL_QUALCOMM_COLOR_BUFFER• The extension should be called using

eglGetProcAddress(“eglGetColorBufferQUALCOMM”)

Blit 2D Skybox Night Vision

11

Current BREW Development PlatformsMultimedia Series

Platform• MSM6100™ - MSM6500™ + DSP acceleration

– MSM6xxx running at 143 MHz (ARM9)– Baseband capabilities– Runs BREW 2.x– On-target debugging

• 3D Gaming performance– 50K tri/sec, 500K pixels/sec

Enhanced Series Platform

MSM6500 + IMAGEON™ W2302 • MSM6500 + IMAGEON™ W2302– MSM6500 running at 143 MHz (ARM9)– IMAGEON™ W2302, running at 75MHz– Baseband capabilities– Runs BREW 2.x– On-target debugging

• 3D Gaming performance– 400K tri/sec, 50M pixels/sec

• MSM6550 + QUALCOMM 3D HW GRP– MSM6550 running at 225 MHz (ARM9)– 3D HW GRP, running at 37.5MHz– Baseband capabilities– Runs BREW 3.x– On-target debugging

• 3D Gaming performance– 100K tri/sec, 3.5M pixels/sec

12

Determining which platform & extensions are supported

glGetString(GL_EXTENSIONS)- Returns a space-separated list of supported extensions - See isExtensionSupported() routine (for reference only)

glGetString(GL_RENDERER)- Returns the name of the renderer- This name is typically specific to a particular configuration of a hardware platform - Best bet to check this value for hardware configuration

For example will return "Q3Dimension MSM6550“ for MSM6550 target

eglQueryString(dpy, EGL_EXTENSIONS )- Returns a space-separated list of supported extensions

eglQueryString(dpy,EGL_VENDOR)- Returns the company responsible for this GL implementation - MSM6550 vendor will be QUALCOMM- MSM6000 series with ATI external chip vendor will be ATI- MSM7500 series vendor will be QUALCOMM because of integrated ATI core with

QUALCOMM’s MSM chip

13

Using IJOYSTICK Interface

• About the joystick

• Including the joystick

• Starting the joystick

• Getting the coordinates

• Cleaning up

14

About the analog joystick• Joystick has a frequency of 40 Hz by default • The coordinates are passed to the application as two’s-

complement signed 16-bit integers• The coordinates range from -32768 to 32767 for each axis,

with a granularity of 256 between each unique joystick position

Frames per Second

# Events per Frame

10 4

15 2.667

20 2

30 1.334

15

Including the joystick• You need AEEVirtualJoystick.h in order to use the

Joystick extension under the SDK• You need AEEJoystick.h when using the joystick

on target• You can use the following code to automatically

do this:##ifdefifdef AEE_SIMULATORAEE_SIMULATOR

#include#include ""AEEVirtualJoystick.hAEEVirtualJoystick.h""#else#else

#include#include ""AEEJoystick.hAEEJoystick.h""##endifendif

• Need VirtualIJoystickExtension.dll for the BREW SDK– Placed in the ../modules directory

16

Starting the joystick• Use ISHELL_CreateInstance with AEECLSID_IJOYSTICK

// Create the IJoystick interfaceIJoystick* m_pJoy;

if ( SUCCESS != ISHELL_CreateInstance( pThis>m_App.m_pIShell, AEECLSID_IJOYSTICK, (void**)&pThis->m_pJoy) )

DBGPRINTF( “Error –Creating IJoystick”);else DBGPRINTF( “Created Joystick Interface”);

• After creating the joystick instance, you will receive joystick events through EVT_JOYSTICK_POS

17

Getting the coordinates• The X-coordinate in the upper 16-bits of the dwParam• The Y-coordinate in the lower 16-bits of the dwParam

#define HIWORD( ui ) ((uint16)(((ui)>>16)&0xFFFF))#define LOWORD( ui ) ((uint16)((ui)&0xFFFF))

boolean joystickdemo1_HandleEvent( IApplet *pApplet, AEEEventeCode, uint16 wParam, uint32 dwParam ) {

switch (eCode){

// … snip …case EVT_JOYSTICK_POS:int dx = (int)( (short)HIWORD( dwParam ) );int dy = (int)( (short)LOWORD( dwParam ) );break;

// … end snip …}

}

18

Polling Mode• The joystick co-ordinates can also be read using

the following function -

IJOYSTICK_Read(pwork->m_pIJoyStick,&dx,&dy);

• This function can be called with each rendering call

• Its been observed that the polling mode makes the joystick more responsive compared to the event mode

19

Cleaning up

• Call IJOYSTICK_Release to free the joystickIJOYSTICK_Release( pThis->m_pJoy );

• After the joystick object is released you will no longer receive any EVT_JOYSTICK_POS events in your applications’ event handler

20

How to make the annunciator bar disappear with BREW applications

1) Open the mif file in the BREW 3.x editor & click on the “Notifications, Flag , Settings…” button as shown below

2) Choose the “No” option in the annunciator option as shown

Annuciator bar

21

Some OpenGL ES Games

Z-Race Asphalt Urban GT NFS 2

22

Part II

The QX Engine™

Baback Elmieh

Sr. Software Engineer

QUALCOMM Gaming Group & 3D Apps Research

23

Outline• What is the QX Engine?

• Content Development Process

• Features

• API Overview

• How to build a simple QX Application

• Tools Overview

• Demos

• Q&A

24

QX EngineTM• What is QX?

– Low level API for manipulating Meshes & Animations– Toolset for exporting & optimizing content from

3D packages– Developed internally for 3DUIs & providing

reference optimizations• Focused on delivering performance

– Optimized for OpenGL ES on QUALCOMM Platforms– Provides content-side & engine-side optimizations– Utilizes all of the suggestions given during the

optimization talk• Usage

– Lowers prototyping time for game developers– Can be used in conjunction with existing engines– Optimization components available as libraries

25

Content Development ProcessUsing the QX Engine™ Environment

Download to phone

Create Animation

Preview Animation

Create Model

Preview Model

BREW SDK simulator

Visual Studio

26

FeaturesVertices: UVs, Normals, ColorsExportable to Strips or Lists

Hierarchical AnimationsMaterial AnimationsCompressed (on-the-fly decompression during playback)Instancing

UV BiasShading: Ambient, Diffuse, SpecularBlending: Incandescence, TransparencyAll attributes can be animated

Meshes

Animations

Materials

27

Features (Continued)

All OpenGL-ES formats (Compressed & Uncompressed)Lossless compression for smaller file-system footprintInstancing

Multiple CamerasApplication or Animation controlled updatesField of View, Aspect Ratio

Immediate/Batch modesMultiple culling options: Models, Materials, MeshesAlpha sortingFront-to-back sortingLandscape/Portrait

Textures

Cameras

Rendering

28

Features (Continued)

One-shot loading & object instantiationQuery functionality for accessing resources

MathData StructuresCommand ManagerDiagnosticsPrimitive Rendering

Resource Management

Utility Libraries

29

QX Engine API• Implemented on top of OpenGL ES• Applications can use QX Engine in tandem with standard

OpenGL• Application in control of initializing & driving the scene

30

API Overview• Top level API: Resource, Camera, Rendering• Low level API: Engine objects accessible & modifiable• Utilities: Math, Data Structures, Diagnostics

31

Render Manager• The default renderer• 2 Rendering modes:

– Batch Mode: Register models once & render them together– Individual Mode: Pass models to the manager individually

• Features– Multiple culling options: Model, Material, Mesh– Front to back sorting for opaque objects– Alpha sorting for transparent objects– Render state caching to reduce context changes– Landscape/Portrait mode

32

Resource Manager• Loads Meshes, Animations & Texture objects (file or memory)• Manages object dependencies• Manages OpenGL resource binding (VBOs, textures, etc…)• Optimized to minimize memory allocations• Application specific resource management possible through

lower level API

33

Other Systems• Camera Manager

– Container for active camera– FOV & Aspect Ratio can be modified– Used for Culling & distance sorting by Render

Manager• Command Manager

– Game-centric keypad abstraction API– Provides polling functionality on top of

BREW’s event based system– Supports KeyHeld, & KeyPress events– Key state history can be queried for

ticks/milliseconds-held– Useful for Combos!!

34

Model• Container for Materials, Meshes & Hierarchy• Application can modify any component

35

Material• Appearance settings for a mesh (Shader in Maya)• Container for meshes with same appearance• Texture, UV Bias• Shading parameters: Color, Specular, etc…• Blending parameters: Alpha, Glow• Render priority settings (Can be used for layers)

36

Animation• Can be attached to Models or other objects

(Cameras, Lights, etc…)• Can be shared or used partially (save memory)• Supports blending for transitional animations• Application determines playback rate• Compression: ~60-75% ratio• Hierarchical

– Translation– Rotation

• Material– UV Bias– Alpha– Color– Incandescence

37

Utilities• 3D Math Library

– Matrices, Vectors, Quaternion– Bounds & intersections: Spheres, Boxes,

Cylinders, Rays, Planes– Animation: LERP, SLERP, Ease/Acceleration

• Data Structure Library– Lists, Queues, Stacks, Dynamic Arrays– Spatial: Quad-Tree, Oct-Tree, Partitioning

Algorithms• Primitive Rendering

– Lines, Spheres, Boxes, Cylinders• Diagnostics System

– Logging, Memory reports, Timers

38

Engine Objects• Model

– Top level container, holds Materials & Meshes

• Material– Appearance of a mesh (color, texture, alpha, etc…)

• Mesh– Vertices, UVs, Normals, Colors

• Animation– Container for Hierarchical & Material animations

• Camera– Can be bound to an animation or controlled

programmatically

• Bounding Volumes– Can be calculated for Models, Materials, Meshes

39

Building a simple application• Steps

– Initialize OpenGL & EGL– Instantiate a QXState Object– Load Assets– Register Models with Manager– Call Manager to batch render

40

Initialization• Instantiate a new QXState using QXState_CreateState// initialize a qx state object

QXState *pState = QXState_CreateState(a.m_pIShell);

• Load resources from file using QXResourceMgr_GetResourceByName// load a model

QXResource *pModelRes = QXResourceMgr_GetResourceByName(pState, QX_RESOURCE_MODEL, “model.qxm");

// load an animation

QXResource *pAnimRes = QXResourceMgr_GetResourceByName(pState, QX_RESOURCE_ANIMATION, “animation.qxa”);

• Register the model using QXRenderMgr_RegisterModel// get the model from the resource

QXModel *pModel =(QXModel*)QXResource_GetDataPtr(pModelRes);

// register it

QXRenderMgr_RegisterModel(pState,pModel);

41

Rendering & Cleanup• Call QXRenderMgr_Render to render the registered models// render the registered models

QXRenderMgr_Render(pState);

• Call QXModel_UpdateWithAnimation to set an animation frame// get the model from the resource

QXModel *pModel =(QXModel*)QXResource_GetDataPtr(pModelRes);

// get the animation from the resource

QXAnimation *pAnimation=(QXAnimation*)QXResource_GetDataPtr(pAnimRes);

// set the model’s pose to frame 10

QXModel_UpdateWithAnimation(pModel, pAnimation, 10);

• Calling QXState_DeleteState will automatically release all loaded resources & instantiated objects

// release all QXEngine memory & assets

QXState_DeleteState(pState);

42

Tools• Maya™ Exporter

– Integrated directly into Maya• Multiple options for exporting optimized files: strips/lists

– conversion to optimized vertex formats for MSM6550, MSM7500• Animation & Texture compression• Artist centric: All assets can be completely generated in May

• Support for 3DS Max™ & SoftImage™ to follow

43

Tools (Continued)• Maya™ Pre-viewer

– Launch directly from Maya– Instantly verify Maya scenes using QX Engine™

rendering API– Supports all QX Engine™ functionality

• BREW viewer– View Maya scene files on the phone/BREW simulator– No coding necessary, just copy the exported files to a phone

loaded with the viewer

• Texture Viewer– View textures after applying OpenGL ES color conversion– Allows artists to anticipate any color degradation in their

textures as a result of conversion to OpenGL ES supported formats

44

Tools (Continued)• Polygon optimizer

– Analyzes & re-orders polygons for optimum performance on target platforms

– Currently supports MSM6550, will support future OpenGL ES targets

– Can be integrated into 3rd party asset tool-chains

• Visual Studio App Wizard– Builds new application templates with all QX Engine™ &

OpenGL-ES initialization code already in place– Can be used to prototype new QX Engine™ applications

quickly– Supported for Visual Studio .NET & 6.0

45

Demos

46

Where to get more information• Learn more about developing BREW applications or

to become an Authenticated BREW Developer visit: www.BREWdeveloper.com– Download (free) the latest version of the BREW SDK– Download the BREW SDK extension for OpenGL ES

• Contact BREW Developer Relations at: www.BREWdeveloperrelations.com

• Obtain the latest BREW market information at: www.BREWtoday.com

• Obtain information about QUALCOMM’s product line for OpenGL ES at: www.cdmatech.com/Q3D

• Visit the Khronos website for OPENGL ES specification & reference information at: www.opengles.org

47

Questions?

THE END