Beijing, March 2012 - khronos.org · © Copyright Khronos Group, 2012 - Page 11 Khronos Conformance...

© Copyright Khronos Group, 2012 -

Khronos Overview Beijing, March 2012

Neil Trevett President, The Khronos Group

Vice President Mobile Content, NVIDIA


Khronos Connects Software to Silicon

APIs enable software developers to turn silicon

functionality into rich end user experiences

Embedded and Mobile 3D

Advanced Audio

Parallel Computing

Camera, Images and Streaming Media

Khronos creates OPEN STANDARD APIs for advanced hardware acceleration

Graphics, video, audio, compute, visual and sensor processing

15 ACTIVE APIs


Why are Khronos Standards Key? • Khronos APIs define core mobile device functionality

- Low-level “Foundation” functionality needed on every platform

• Khronos standards have strong industry momentum

- Shipping on billions of devices

- Shipping on multiple OS

• They are FREE

- No royalties

Khronos standards drive mobile market growth by expanding device capabilities

Tianyu

HTC ZTE


Khronos History

2000 2003 2004 2005 2006 2008 2009 2011

Embedded and Mobile 3D

Advanced Audio

Parallel Computing

Camera, Images and Streaming Media

3D Digital Asset Exchange format

Vector 2D

Cross platform desktop 3D

Plugin-free 3D Web Content

Web Compute

Hardware accelerated vision functions

OpenVL

Unified Sensor and Input Processing

Over 100 Members 15 Active Standards


Board of Promoters

Over 100 members – any company worldwide is welcome to join

Apple

http://www.amd.com/

http://www.antixlabs.com/

http://www.altsoftware.com/

http://www.google.com/

http://www.hu1.com/

http://www.fujitsu.com/

http://global.mitsubishielectric.com/

http://www.nds.com/

http://www.st.com/

http://www.sasken.com/

http://www.toshiba.com/

http://www.vivantecorp.com/

http://www.marvell.com/index.jsp

http://www.umu.se/umu/index_eng.html

http://www.gshark.com/


Board of Promoters


Khronos has 30% Asian

Membership

http://www.hu1.com/







Board of Promoters


Apple

The Khronos Board of Directors has made Chinese industry participation a priority and committed

significant financial and staffing resources!

http://www.amd.com/

http://www.antixlabs.com/

http://www.altsoftware.com/

http://www.google.com/

http://www.hu1.com/



http://www.nds.com/

http://www.st.com/

http://www.sasken.com/



http://www.marvell.com/index.jsp

http://www.umu.se/umu/index_eng.html



The Value of Khronos Participation

Gather Industry Requirements for future

silicon acceleration

Draft Specifications Confidential to

Khronos members

Publicly Release Specifications and Conformance Tests

Companies Release Products

Have a voice in how key standards

evolve to suit your business

needs

Develop products in

parallel with spec drafting

for faster time to market

See an early window into the

future of the industry technology

roadmap before products are developed

Products are aligned with global market needs and trends

The Khronos standardization process is proven to RAPIDLY generate industry consensus on future hardware acceleration functionality to EFFICIENTLY create new market opportunities

Members can ship products faster than

non-members


Khronos Working Group Process

Working Groups (WGs) One working group per API

Promoter Members - Participation and vote in WGs - Board seat for strategy, budget and spec ratification

Contributor Members - Participation and vote in WGs

Academic Members - Participation in WGs

Ratified Specifications

Conformance Tests and Adopters Program

SDKs, Sample, Ref Cards and Man Pages

Adopters Build conformant implementation and products

Developers Develop applications using the APIs

$$$ $

$

Members

Wider Industry


Khronos Cooperative Model

Khronos Specification Conformance Tests

Khronos standards have significant value - 100s of

man years invested by industry leading experts

Khronos creates open, royalty-free API standards

Khronos Members agree not to assert IP claims against other Members or Adopters

for CONFORMANT IMPLEMENTATIONS

• Solid legal framework for industry cooperation • ‘By the industry for the industry’ • Open to any company to join

• How does Khronos make money? • It doesn't! • Khronos fees cover costs of

creating, deploying and promoting specs and tests


Khronos Conformance Process

• Implementers of Khronos specifications are not covered under the Khronos

IP framework unless their products are submitted and pass conformance!

• Khronos administers an Adopters Program for each API

- Adopters program provides full test access and trademark license for small fee

Company executes Adopters

Agreement and pays fee (usually

US$10K for unlimited products)

Port and execute tests on products to generate test

results

Upload test results to Khronos private

web-site. Peer Review by

members/Adopters

Successful Review of results enables products to use

Khronos trademarks and to be listed on Khronos website

Company implementing Khronos spec

wishes to use the trademark

Example: “We implemented OpenGL ES”

Full use of logo and

trademark

Full use of logo and

trademark with small disclaimer

Restricted use trademark (not

logo) with disclaimer language

Adopter Benefit


Khronos API Ecosystem Evolution

New API technology first

evolves on high-end platforms

Mobile is the new platform for

apps innovation. Mobile

APIs unlock hardware and

conserve battery life

Apps embrace mobility’s

unique strengths and need

complex, interoperating APIs

with rich sensory inputs

e.g. Augmented Reality

Diverse platforms – mobile, TV,

embedded – means HTML5 will

become increasingly important

as a universal app platform

DESKTOP

MOBILE

INTEROP, VISION AND SENSORS

WEB


Augmented Reality – the Next Big Thing

Camera video stream sent to the compositor

3D Augmentation Rendering

3D augmentations composited with video stream

Camera Tracking

Camera images used to track the

camera’s location and

orientation

Camera-to-scene transform locks the 3D rendering to the real world


Sensor Fusion with App Portability

Advanced Sensors Everywhere Standard cameras, depth cameras

motion and position, touch, microphones wireless controllers

Apps Need Sophisticated Access to Sensor Data

Without coding to specific systems or sensor hardware

Apps request semantic sensor information StreamInput defines list of possible semantic requests

“Am I in an elevator?” “Give me gestures and face position”

Processing graph provides sensor data stream Apps gain ‘magical’ situational awareness

Utilizes optimized, sensor smart middleware


Current StreamInput Participants • Aiming for specification release in 2012


OpenVL • Vision Hardware Acceleration Layer

- Enable hardware vendors to implement

accelerated imaging and vision algorithms

• OpenVL can be used by high-level

libraries or applications directly

- Primary focus on enabling mobile and

embedded systems

• OpenCV is widely used open source

library for vision projects

- Future version will leverage OpenVL

• Working group just starting

Get Involved Now!

OpenCV open source library

Application

OpenVL

Open source sample implementation?

Hardware vendor implementations

Other higher-level CV libraries


Augmented Reality Functionality

Camera Processing

3D Rendering and Video Composition

Audio Rendering

Application on CPU

Positional and GPS Sensor Data

Computer Vision and Tracking

Position

and Tracking

Semantics

Control Camera, Preprocess and generate

video streams

Video TAP to CPU

Synchronization and sensor

fusion

Video stream to GPU

Positional Sensors

Camera EGLStream

Much more flexibility than just “overlay augmentations

over background”

OpenVL


Possible Implementation of Vision Stack

Semantics and fusion of camera and positional sensors

Accelerated computer vision algorithms

Parallel computation

OpenVL

High-level computer vision library

Use OpenCL to implement OpenVL with

parallel execution

Accelerate OpenCV

library with OpenVL

Functions

Implement StreamInput vision sensor modules with

OpenCV

Data and event interop with CL /

GL / ES for display and compute processing

Camera input from OpenMAX AL or other camera subsystems


Android Native API Adoption

OpenGL ES OpenGL ES 2.0

Shipping - Android 2.2

OpenSL ES OpenSL ES 1.0

Shipping – Android 2.3

OpenMAX AL OpenMAX AL 1.0

Shipping - Android 4.0

EGL EGL 1.4

Shipping under SDK

OpenCL Not yet adopted

StreamInput Working group will ship spec in 2012

OpenVL Working group will ship spec in 2012


Extended Native APIs on Android • Native APIs can be shipped as NDK extensions before Google Adoption

- Do not break/change existing Google APIs

• Khronos APIs have strong momentum in the silicon community

- Evangelize Google to adopt into standard platform

• Extended APIs can be used by:

- Bundled apps, Market apps with API selection

- Multiple APKs behind single multi-APK SKU

Standard Android SDK

and NDK

Extended NDK for

Key use cases

Evangelize Google and ISVs to

adopt enabling APIs

E.g. video, camera, imaging,

sensor and composition APIs


Khronos Members

Enhancing Educator Efforts

KITE Educators

Students

Industry Best Practices

Khronos-related courseware and exams

Skilled Employees

Khronos does not teach courses … … Khronos assists Educators in teaching Khronos APIs

Courseware For Review


Khronos and China • Participation in Khronos standardization

will enable Chinese companies to

compete in global mobile market

• Avoid costs and market confusion caused

by fragmentation

• Khronos has prioritized Chinese company

participation to help grow market

opportunities for all

• Key Chinese companies:

- Silicon vendors

- Mobile OEMs

- Mobile carriers Khronos hosts first DevU in

China at PC Expo Beijing and meeting with Mayor of Tianjin

October 2004

China Mobile Terminal Industry Development Forum Shanghai

November 2005

Mobile Entertainment World Shanghai November 2007

DevU Beijing – Tsinghua University Dec 2010

Outreach to Chinese Industry Beijing and Shanghai

March 2012

Khronos Group are Gold Sponsors of the China Game

Developer Conference on July 25-27, 2012 in Shanghai

Khronos China

Outreach


In Summary • APIs are key to enable compelling applications on advanced hardware –

APIs developed on high-end hardware are now enabling mobile devices

• APIs no longer exist alone – they are interoperating to form platform

ecosystems for advanced content

• Significant cooperation happening between native and Web APIs to bring

advanced visual computing to HTML5

• Khronos is driving open standards for hardware acceleration

Participate, change the industry AND get the inside edge for your products!

Connecting Software to Silicon

Beijing, March 2012 - khronos.org · © Copyright Khronos Group, 2012 - Page 11 Khronos Conformance...

Documents

Transcript of Beijing, March 2012 - khronos.org · © Copyright Khronos Group, 2012 - Page 11 Khronos Conformance...