Seoul DevU December 2010 - Khronos Group...Developers creating user experiences need efficient...

© Copyright Khronos Group, 2010 -

Seoul DevUDecember 2010

Neil TrevettVice President Mobile Content, NVIDIA

President, The Khronos Group


Thank You for Coming!• Welcome Remarks & Review Agenda - Hwanyong Lee, Chief Technical Officer, Huone

• Khronos digital media ecosystem and WebGL - Neil Trevett, VP Mobile Content, NVIDIA

• OpenCL Overview - Justin Henley, Principal Member Technical staff, AMD

• OpenGL Overview - Dave Shreiner, Director of Graphics Technology, ARM

• Break - 10:45-11:00

• OpenGL SC - Nakhoon Baek - Professor Kyungpook Nat'l University

• OpenGL ES Applications on Bada - Sungyul Choi, Ph.Dm, Samsung

• OpenGL ES 2.0 Optimization and fragmentation - David Harold, Director of PR, Imagination Technologies

• Conference Lunch & Demos - 12:45 - 14:15

• GPU quality and application portability - Kalle Raita, Senior Software Architect, Drawelements

• OpenVG overview : implementations and applications - Hwanyong LEE, Chief Technical Officer, HUONE

• Key Advantages of OpenVG1.1 - Tommy Asanom, Executive Director, Advanced Technologies, Takumi

• Break - 16:00-16:15

• SMAPH series - Eisaku Ohbuchi, Director, Hardware Development, DMP

• OpenSL ES & OpenMAX AL, IL, DL - Erik Noreke, OpenSL ES Work Group Chairp OpenSL ES/OpenMAX

• Kanzi 3D User Interface Solution and COLLADA - Arto Ruotsalainen, Technical Project Manager, Rightware

• Rightware sponsored Beer!


Khronos Vision

“The best way to predict the future is to invent it.”Alan Kay

Billions of devices increasinggraphics, compute, video,

imaging and audio capabilities

Market growth driven by rich

user experiences

Developers creating user experiences need efficient silicon accelerator access

Standard APIs define silicon capabilities

and provide reliable developer access


Annual Market Size

Device Power Consumption

1W10W100W1,000W

1M

10M

100M

1B

1000x Lower Power Changes the World

The next 10 billion Computers will be deeply integrated into the

fabric of our lives


Modern Mobile Silicon Functionality

• Rich end-user experiences

need significant processing

- Parallel Computing

- GPU Computing

- 3D Graphics

- 2D Graphics

- Camera control

- Image processing

- Video encode

- Video decode

- Audio encode

- Audio decode

• All this silicon needs

to be programmed


The Critical Role of APIs

Khronos APIs are low-level – just above raw silicon - to create the

“foundation” functionality needed on every platform

APIs enable software developers to turn

silicon functionality intorich end user experiences


Khronos - Forum for Industry Cooperation

Open International MembershipAny company is welcome to join to suggest working groups and technology directions

Open StandardsPublicly available onweb-site to be used

by anyone

Royalty FreeNo per-device royalty

to ship APIimplementations

Industry CooperationHundreds of man years invested by industry experts to create state-of-the APIs

Opportunity to compete to create chips, software and systems that deliver amazing end user experiences in international markets

International Market Growth= ¥ $ € !


Board of Promoters

Over 100 members – any company worldwide is welcome to join

Apple

http://www.amd.com/

http://www.aplix.co.jp/en/

http://www.codeplay.com/

http://www.antixlabs.com/

http://www.altsoftware.com/

http://www.dell.com/content/default.aspx?c=us&l=en&s=gen

http://www.google.com/

http://www.hu1.com/

http://www.fujitsu.com/

http://global.mitsubishielectric.com/

http://www.nds.com/

http://www.s3graphics.com/

http://www.st.com/

http://www.sasken.com/

http://www.toshiba.com/

http://www.vivantecorp.com/

http://www.yumetech.com/

http://www.marvell.com/index.jsp

http://www.umu.se/umu/index_eng.html

http://www.gshark.com/


Khronos Family of Standards

Embedded 3DCross platform

desktop 3D

3D Digital Asset Exchange format

Advanced Audio Vector 2D

Context, Sync and Surface Management

Mobile OSAbstraction

Au

tho

rin

g a

nd

a

cce

ssib

ilit

yA

pp

lica

tio

n

Acce

lera

tio

n

Parallel Computing

Plugin-free 3D Web Content

Streaming MediaSafety Critical 3D

Syste

m

Inte

gra

tio

n

Window SystemAcceleration

A coordinated ecosystem of compute, graphics and media

standards and APIs

Video, Audio andImage Acceleration

Codec Creation


Khronos Process

Working GroupsOne working group per APIOne company – one vote

Promoter Members- Participation and vote in working groups- Board seat for strategy, budget and spec ratification

Contributor Members- Participation and vote in working groups

Academic Members- Participation in working groups

Any member can propose a new working group and can contribute technology they want to see adopted by the Industry

Ratified Specifications

Conformance Tests

SDKs, Sample Implementations, Documentation and Man Pages

Adopters Build conformant implementation and products

DevelopersDevelop applications using the APIs

Khronos Adopters Agreement enables API trademark to be used on conformant products


Khronos IP Framework

IP “Safety Zone”Khronos members agree to not sue

other members over Khronos standards

Industry ProtectionKhronos membership and hence mutual

IP non-assert from other members is open to any international company

Member ProtectionNo IP disclosure is necessaryVery restricted license grant:

- No implementation IP- Only essential IP (i.e. no alternative)- Only for conformant implementations

Health WarningKhronos makes no warranties

about non-members IP status - though Khronos working groups attempt to

not infringe third party IP

Implementation IPIs not licensed – enabling market

competition over cost, performance, power etc.


Cooperation versus Competition

100%COMPETITION

100% COOPERATION

Competing technologies confuse and fragment market: e.g. video codecs ,

VHS vs. Betamax, BluRay vs. HD DVD

Lack of competition stifles innovation and slows creation of new user-valued functionality

Productive cooperation to enable market competition - maximizing

end-user value and market growth

$ CommercialOpportunity

Every industry needs foundational standards to enable sufficient value to be delivered

economically by competitive vendors to enough end users to build a market


Ecosystem Growth

Tools and SDKs

Conformance Tests

Open Authoring Standards

High-quality platforms

Open Acceleration API Standards

Market demand for rich applications

Great Applications

IndustryCooperation

Building a upward spiral of commercial opportunity


Major Releases This Year at Khronos

Nov09

WebGL 1.0 Provisional

Specification Released

OpenCL 1.1Specification

Released

Jun10OpenGL 4.0

Released

Mar10

OpenGL 4.1Released

Jul10


OpenGL for Each Hardware Generation

Geometry Shaders

1.X

Fixed Function

2.X

Vertex and Fragment Shaders

3.X

Tessellation and Compute

4.X

„Surfacerealism‟

„Shape realism‟


Processor Parallelism

CPUsMultiple cores driving performance increases

GPUsIncreasingly general purpose data-parallel

computing

Graphics APIs and Shading

Languages

Multi-processor

programming – e.g. OpenMP

EmergingIntersection

HeterogeneousComputing

OpenCL is a programming framework for heterogeneous compute resources


OpenCL Timeline

• Six months from proposal to released OpenCL 1.0 specification

- Due to a strong initial proposal and a shared commercial incentive

• Multiple conformant implementations shipping

- For CPUs and GPUs on multiple OS

• 18 month cadence between OpenCL 1.0 and OpenCL 1.1

- Backwards compatibility protect software investment

Apple proposes OpenCL working group and contributes draft specification to Khronos

Khronos publicly releases OpenCL 1.0 as royalty-free specification

Khronos releases OpenCL 1.0 conformance tests to ensure high-quality implementations

Jun08

Dec08

May09

2H09

Multiple conformant implementations ship across diverse OS and platforms

Jun10

OpenCL 1.1 Specification released and first implementations ship


Mobile OS Fragmentation

• Every handset is unique from the programmers perspective

- Differences in OS functions, Java implementations and media functionality

Symbian^3, PocketPC / Windows Mobile / WinCE

Linux variants – Android, Limo, WebOSJava MIDP-1, MIDP-2, JSR fragmentation

RTOS – Nucleus, SynergyBrew, WIPI, iPhone OS

Severe platform fragmentationISVs traditionally need to port to and support 100's (even 1000's) of

source variants of each application/title

We need cross platform standard

APIs to de-fragment access to graphics, media and

compute acceleration if the

mobile market is to be fully realized

http://www.apple.com/

http://images.google.com/imgres?imgurl=http://www.cio.com/images/content/articles/body/2007/11/android_robot.gif&imgrefurl=http://www.cio.com/article/156250&h=135&w=121&sz=4&hl=en&start=4&sig2=VCDouhSNWn_-wOlH6y2rhQ&tbnid=sUOsxQroWspMpM:&tbnh=92&tbnw=82&ei=JQmVSISCAajmigHa5Zy8Cg&prev=/images?q=google+android+logo&gbv=2&hl=en&sa=X


Other Needs for Mobile Acceleration APIs

0.36 0.14

1.85

17.8

0

2

4

6

8

10

12

14

16

18

ARM9 32-Bit RISC at

400MHz

DSP at

175MHz

APA 512

MiMagic 6

POWER EFFICIENCYLess Power!

Hardware accelerators exploit media pipeline parallelism and caching for a x10 increase in power efficiency over software

Better User ExperienceSmall screens need advanced graphics processing per pixel

Faster Performance at Higher QualityHardware delivers smoother interaction with much

better looking visuals

Software 3D Accelerated 3D


The Khronos Mobile API Family

Mobile 3D

Advanced Audio

Vector 2D

Parallel Computing

Video, Audio and ImageSilicon Acceleration

Steaming Media

POSIX-like Mobile OS

Abstraction:User Input, files,

threads, math libraries

Context, Sync and Surface Management

Complete set of APIs for rich, portable mobile application development


EGL Becoming an Interoperability Hub

Buffers, textures and video streams flow efficiently

between any combination of client APIs

Inter-API Synchronization events enable efficient

resource sharing

EGL Interoperability Conformance Tests being

created to ensure that client APIs can cleanly communicate

Enabling the Khronos individual APIs to interoperate as a

coherent ecosystem


Mobile Visual ComputingCompute, graphics and AV APIs

interoperate through EGL

Visual Computing Ecosystem

Desktop Visual ComputingOpenGL and OpenCL have direct

interoperability. OpenCL objects can be created from OpenGL Textures, Buffer

Objects and Renderbuffers


Mobile Application Portability

CompositingUser Interfaces

Games and Game EnginesGames and

Game EnginesGames and Game Engines

Flash PlayersFlash PlayersFlash and SVG

Players

TV and Video StacksTV and Video

StacksTV and Video Stacks

Applications are portable toKhronos-enabled platforms

OS resource abstraction

Browsers

Silicon and platform

independent graphics, media

and compute acceleration

http://images.google.com/imgres?imgurl=http://www.cio.com/images/content/articles/body/2007/11/android_robot.gif&imgrefurl=http://www.cio.com/article/156250&h=135&w=121&sz=4&hl=en&start=4&sig2=VCDouhSNWn_-wOlH6y2rhQ&tbnid=sUOsxQroWspMpM:&tbnh=92&tbnw=82&ei=JQmVSISCAajmigHa5Zy8Cg&prev=/images?q=google+android+logo&gbv=2&hl=en&sa=X


Raising 2D and 3D Visual Quality

Older generation APIs Provide rudimentary graphics functionality and quality

State-of-the-art APIs enable compelling consumer displays

Advanced functionality, fast interactivity and extremely high quality

High-quality 2D graphics and test using OpenVG

High-quality 3D displays using OpenGL ES

Video and image processing with

OpenMAX IL


OpenGL ES Penetration

0

100

200

300

400

500

600

700

800

900

2006 2007 2008 2009 2010 2011 2012 2013

3D phones

Phones with OGL ES

Source: Jon Peddie Research, March 2009

OpenGL ES has become the most widely deployed 3D APISmartphones, games consoles, GPS, media players, automotive, tablets…


Driving API Adoption into Device Platforms

Plug-in

Native support

SurfaceFlinger

EGL and OpenSL ES now supported in Android 2.3 NDK

http://www.apple.com/


WebGL


WebGL – 3D on the Web – No Plug-in!

• Historic opportunity to bring accelerated 3D graphics to web

- WebGL defines JavaScript binding to OpenGL ES 2.0

• Leveraging HTML 5 and uses <canvas> element

- Enables a 3D context for the canvas

• Working group launched March 2009

- Mozilla, Apple, Google and Opera working closely with OpenGL GPU vendors

OpenGL ES 2.0 on every web-capable device

JavaScriptbinding to

OpenGL ES 2.0Increasing JavaScript performance and HTML5 Canvas


ContentJavaScript, HTML, CSS, ...

WebGL Implementation

Middlewaree.g. X3D

WebGL HTML5

JavaScript CSS

Browser provides WebGL functionality alongside other HTML5 specs

- no plug-in required

OS Provided Drivers. WebGL on Windows can use Google Angle to create

conformant OpenGL ES 2.0 over DX9

OpenGL ES 2.0OpenGL

DX9/Angle

Content downloaded from the WebMiddleware can make WebGL accessible to

non-expert 3D programmers

http://code.google.com/p/angleproject/


WebGL Timeline Update

• Provisional spec – November 2009

- Implementations now in public daily builds of Chrome and Mozilla

• Extensive WebGL middleware already shipping

- GLGE, SpiderGL, X3DOM, EnergizeGL, SceneJS, O3D, CopperLicht, CubicVR

• WebGL Extension Registry Online

- http://www.khronos.org/registry/webgl/doc/spec/extensions/

• WebGL 1.0 final specification expected spring 2011

- With full conformance tests

Plug-in

Production support expected in 1Q11

http://www.khronos.org/registry/webgl/doc/spec/extensions/




WebGL Compositing

• WebGL is HTML - fully participates in canvas/browser compositing

- 3D is not trapped in a rectangular window

- 3D can overlay and underlay HTML content - including SVG

- Easy to make HUDs or user interfaces

• WebGL TexImage2D accepts

- Raw buffer

- HTML image element

- HTML video element (takes current frame)

- HTML canvas element

E.g. draw text to 2D canvas, use as bump-map texture in WebGL

then composite 3D model over HTML text


Newsflash - WebGL Now in Chrome Beta

http://www.chromeexperiments.com/webgl/

http://www.chromeexperiments.com/webgl/


OpenGL Ecosystem

Leading-edge functionality developed first on desktop

OpenGL ES 2.0 on desktop as subset of OpenGL 4.1 for mobile

content flexibility – including native support for WebGL

WebGL will drive new-generation security into

OpenGL family

Mobile functionality subset that is deployed on billions of devices

Pervasive OpenGL ES 2.0 availability enables Browser vendors to build 3D

directly into HTML5


Thank you!

• Any questions?

Seoul DevU December 2010 - Khronos Group...Developers creating user experiences need efficient...

Documents

Transcript of Seoul DevU December 2010 - Khronos Group...Developers creating user experiences need efficient...