Android & OpenGL ES 2+from an app developers point of view

Droidcon Berlin, 2014

Andreas NilssonSoftware specialist @Jayway

!

Broad interest in graphics, both 2D and 3D !

www.jayway.com/blog

Inspiration

AndreasEarthBACKUP PLAN(ET)

Terminology

• Matrix - A NxN array of values

• Vertex - A point in space which act as the basic building block for shapes e.g. line, triangle, etc.

• Mesh - A structure compromised of triangles

• Shader - Small program running on the GPU in parallell.

Background - OpenGL

• An abstract graphics API - That is it

• Written in C but not limited to it

• Bindings for most programming languages

• Low level ⬌ Close to hardware

Better control of when things are rendered - Possible to make specific optimizations

Why OpenGL?Control

Control

Rendering of ambiguous 3d meshes

Models

Why OpenGL?

Control Models Effects

Why OpenGL?

Custom effects, that can only be achieved with a shader

Why not RenderScript ?

It is platform specific which limits its usage, etc.My preference: OpenGL + OpenCL

Android Wrapping of OpenGL

Model

float[…]

FloatBuffer

Drawing

OpenGL

GLES20

Abstract

Android

Native

Context

GLSurfaceView

EGL

EGL

• Platform independent abstract API for setting up an OpenGL context/surface

• Query graphics card for context capabilities

• GLSurfaceView wraps EGL

OpenGL ES Versions

OpenGL ES 1.x - Fixed pipeline

OpenGL ES 3.x- Extensions and Optimizations

OpenGL ES 2.x - Shader support

OpenGL

OpenGL ES⇎

⇕

OpenGL in AndroidGLSurfaceView

GLSurfaceView.Renderer

- A view, set-up for rendering with OpenGL - Runs in its own thread - OpenGL context can be lost onPause()

- Where the actual rendering with OpenGL happens

GLSurfaceView.Renderer - Quick Look

• onSurfaceCreated() - The OpenGL context is now created and we can start loading our resources

• onSurfaceChanged() - Called when the size has changed e.g. device rotation

• onDrawFrame() - Where the fun happens!

• Never mind the parameter GL10, for GLES20+

1. Create an OpenGL Context

2. Initial OpenGL Setup-clear color, cull mode, etc

3. Load Resources -textures, models, shaders, etc

4. Compile and link shaders

5. Upload Resources to GPU

6. Clear Screen

7. DrawDraw Loop

Rendering: Step-By-Step

GLSL ESThe OpenGL Shading Language

GLSL - OpenGL Shading Language

• A C-style language for writing OpenGL shaders

• OpenGL ES 2.0 has two types of shaders- Vertex shader- Fragment shader

• Vertex shader - A program that runs per vertex

• Fragment shader - A program that runs per fragment

Rendering Pipeline5.929688 4.125000 0.000000 5.832031 4.494141 0.000000 5.945313 0.000000 6.429688 4.125000 0.000000 5.387188 4.125000 2.747500 5.297100 4.494141

… x.xxxxxx

][

Vertex Shader Fragment Shader

Simplified!

GLSL Qualifiers

• Uniform - A value that is the same for all vertices and/or pixels

• Attribute - A value defined per vertex, only available in a vertex shader

• Varying - Data shared between shaders down the pipeline and since they don’t map 1:1 they are also interpolated

Simple Vertex Shaderattribute vec4 a_position;attribute vec3 a_color;uniform mat4 mvp;varying vec3 color;void main(){ color = a_color gl_Position = mvp * a_position;}

Calculations are done per vertex

Simple Fragment Shader

precision highp float; // ES Specificvarying vec3 color;void main(){ float alpha = 1.0; gl_FragColor = vec4(color, alpha);}

Calculations are done per fragment

We have a battery! Running @60 FPS for no reason drains the battery Solution: RENDERMODE_WHEN_DIRTY

General Tips & Tricks• Limited memory bandwidth - Offload memory bus with computations in shader

• If-statements - In shader is bad for parallelism, both branches are usually evaluated

• Method calls - In draw-loop may impact performance. Use direct access to data instead. Bad for encapsulation, good for performance

• Memory allocation - In draw loop should be avoided. Allocate everything on beforehand or on-demand

• Power of 2 textures - Good practice to use. Non-power-of-two textures has some limitations in OpenGL ES 2.0

Sample Project Demo

github.com/andreasnilsson/OGLHelloWorldExample Project:

Questions?

plus.google.com/+AndreasNilsson1985

se.linkedin.com/pub/andreas-nilsson/14/685/966

Droidcon attendees noticeI was unable to host the requested shaders for the earth demo. I might post a blogpost about it in the near future so follow me on google plus or the Jayway blog. You are feel to ask me questions about it though.

Finally, Thanks for your questions and interest in my presentation. Feel free to share any feedback to help me improve my presentations.