2013 April Meetup Asia - Khronos Overview

© Copyright Khronos Group 2013 |

Khronos Overview 概述 Beijing China, April 2013 北京中国， 2013年4月

Erik Noreke Technology Visionary

Vice President Business Development, The Khronos Group


Topics 主题 • Why open standards advance the industry

• How Khronos creates open API standards for silicon acceleration

• State of the art in visual computing APIs

• Mobile Multimedia and Advanced Audio

• Vision and sensor processing for Augmented Reality

• Advanced hardware acceleration in HTML5

• 开放标准是如何推动行业发展的

• Khronos是如何为芯片加速创建API标准的

• 视觉计算API的最新动态

• 移动多媒体和高级音频

• 增强现实的视觉和传感处理

• HTML5的高级硬件加速


Why do we NEED Standards? 我们为什么需要标准？ •  Need standard INTERFACES for

INTEROPERABILITY … … Widely adopted standard interfaces can grow market opportunity

•  Interoperability enables compelling user experiences … … to be built cheaply enough to build a mass market

•  Standards avoid fragmentation that adds no value … … Industries need to COOPERATE to build a market and then COMPETE

•  The mobile industry is defined by standards -  GSM/EDGE, UMTS/HSPA, LTE, IEEE 802.11,

Bluetooth, USB, HDMI …

• 需要为交互操作提供标准界面…

…受到广泛采用的标准界面可以带动市场增长

• 互操作性实现了更强的用户体验…

…以最低的成本构建巨大的市场

• 标准避免了毫无价值的存储残片问题…

…业界需要合作建立市场然后再竞争

• 移动行业是由标准定义的

- GSM/EDGE, UMTS/HSPA, LTE, IEEE 802.11,

Bluetooth, USB, HDMI …


Khronos Connects Software to Silicon 链接软件与芯片

• Khronos APIs define processor acceleration capabilities -  Graphics, video, audio, compute, vision and sensor processing •  Khronos的API定义了处理加速性能 - 图形图像、视频、音频、计算、视觉和传感处理

APIs developed today define the functionality of platforms and devices tomorrow

今天开发的API定义着未来平台和设备的性能


• Khronos defines APIs at the software silicon interface -  Low-level “Foundation” functionality needed

on every platform

• Khronos standards have strong industry momentum -  100s of man years invested by industry

experts -  Shipping on billions of devices across

multiple operating systems -  Rigorous conformance tests for cross

-vendor consistency

Software

Silicon

APIs BY the Industry FOR the Industry 业界为行业开发的API

• Khronos为软件芯片界面定义API -  每个平台都需要的低水平“基础”性能

• Khronos标准拥有强大的行业意义

-  行业领先专家超过100人年的投入

-  在数十亿的设备和各种操作系统上使用

-  严格的符合性测试以保证各方面一致性


• Khronos is OPEN for any company to join and participate -  Standards are cooperative – one

company, one vote -  Proven legal and IP framework for

industry cooperation -  Khronos membership fees to cover

expenses

• They are FREE

-  Members agree to not request royalties

Software

Silicon

APIs BY the Industry FOR the Industry 业界为行业开发的API

• Khronos欢迎所有公司加入和参与

-  真正开放的标准 —— 一家公司一个投票权

-  为业界合作提供巩固的法律和知识产权框架

-  Khronos会员费仅仅用于组织运作费用

• 全部是免费的

-  会员同意无需支付版税


Over 100 members – any company worldwide is welcome to join

Apple

Board of Promoters


Khronos Standards

15 Active Standards 15个现行标准

2000 2003 2004 2005 2006 2008 2009 2011

Embedded and Mobile 3D Advanced Audio

Parallel Computing

Camera, Images and Streaming Media

3D Digital Asset Exchange format

Vector 2D

Cross platform desktop 3D

Plugin-free 3D Web Content

Web Compute

Hardware accelerated vision functions

Unified Sensor and Input Processing

嵌入和移动3D

照相机、图片、流媒体

矢量 2D

3D 数字资产交换格式

高级音频

跨平台桌面3D

并行计算

无插件3D网页内容

统一传感器和输入处理

硬件加速视觉功能

网页计算


Additional Khronos Initiatives 其他活动 • KITE

-  Connecting Industry and Educators

• Chapters -  Grass Roots Community for Khronos

-related activities -  Global network of like minded volunteers

and participants – similar to ACM -  Local networking, education and

activities

(Education) Chapters

(Community) Industry

(Technology)

WebGL Meet-up San Francisco Sep11 WebGL旧金山线下活动，2011年9月

• KITE - Khronos培训教育学院 - 链接业界与学术界

• 分会 -  Khronos草根社群 – 相关活动 - 全球网络汇集志同道合的志愿者和

参与者 – 类似ACM - 当地社交活动，教育和沟通


Khronos Working Groups 工作组

Ratified Specifications 规范批准

Conformance Tests and Adopters Program 符合性测试和采用者项目

SDKs, Sample, Ref Cards and Man Pages 软件开发包、示例、参考卡、手册

Adopters 采用者 Build conformant implementation and products 建立符合性执行和产品

Developers 开发者 Develop applications using the APIs 使用API开发应用程序

Educators / Certifiers 教育者/认证者 Academic Courses 学术课件 Training and Certification 培训和认证

Educator Guidelines and Courseware 教育者指导和课件

API Working Groups (Industry and Academic) 工作组（企业与学术会员）

Members 会员 Wider Community 更广泛的社群

Certification Materials 认证素材

KITE encourages academic participation in working groups to produce materials for educators and certification KITE鼓励学术会员参与到工作组中，为教育者和认证方面创建素材

Promoter Members 发起者会员 - Participation and vote in WGs -  Board seat for strategy, budget and

spec ratification -  工作组参与权和投票权 -  董事会席位，参与战略、预算和规范审批

Contributor Members 贡献者会员 -  Participation and vote in WGs -  工作组参与权和投票权

Academic Members 学术会员 -  Participation in WGs -  工作组参与权


Academia is Responding Positively 学术界的积极参与 • Nine New Academic Members joining in 2012 since announcing Kite!

• 自2012年推出KITE项目，已有9个新的学术会员加入！

Khronos Academic Member 学术会员 Country 国家 Date Joined Khronos 加入日期

Beijing University of Posts and Telecommunications 北京邮电大学 China 中国 8 Apr 2013

Nihon University 日本大学 Japan 日本 18 Mar 2013

University of Bristol 布里斯托大学 Great Britain 英国 7 Aug 2012 ITRI 工业促进协会 Taiwan 台湾 5 July 2012

Oregon State University 俄勒冈州大学 USA 美国 29 May 2012

Politecnico di Milano 米兰理工大学 Italy 意大利 01 Mar 2012

Universita di Bologna 博洛尼亚大学 Italy 意大利 01 Mar 2012

Universite Claude Bernard Lyon 里昂第⼀一大学 France 法国 13 Feb 2012

Imperial College London 帝国理工大学 Great Britain 英国 17 Jan 2012

University of Toronto 多伦多大学 Canada 加拿大 31 May 2011

University Western Australia 西澳大学 Australia 澳大利亚 14 May 2010

Kyungpook National University 庆北大学 Korea 韩国 23 Mar 2010

Seoul National University 首尔国立大学 Korea 韩国 26 May 2009

UMEÅ 于默奥大学 Sweden 瑞典 18 Apr 2008


API Standard Evolution API标准的演进

New API technology first

evolves on high-end platforms

全新API技术首先进入高端平台

Mobile is the new platform for apps innovation. Mobile

APIs unlock hardware and conserve battery life

手机是应用创新的新平台。移动AP

I解脱了硬件和电池消耗的束缚

Apps embrace mobility’s unique strengths and need complex,

interoperating APIs with rich sensory inputs e.g. Augmented Reality

各种应用涵盖了移动设备的特殊性能并需要

复合功能的实现，于是带来了拥有丰富传感输入的内部操作API，如增强现实

Diverse platforms – mobile, TV, embedded – means HTML5 will become increasingly important

as a universal app platform

各种平台– 手机、电视、嵌入式HTML

5将越来越重要

DESKTOP 桌面

MOBILE 移动

INTEROP, VISION AND SENSORS

交互操作，视觉和传感

WEB 网页


Technology Overview 技术概述 Beijing China, April 2013 北京中国， 2013年4月

Erik Noreke Technology Visionary

Vice President Business Development, The Khronos Group


OpenGL 20th Birthday - Then and Now OpenGL 20岁生日 – 过去与未来

1992 Reality Engine

8 Geometry Engines 4 Raster Manager boards

2012 Mobile NVIDIA Tegra 3 Nexus 7 Android

Tablet

2012 PC NVIDIA

GeForce GTX 680 Kepler GK104

Triangles / sec (millions) 三角形/秒（百万） 1 103 (x103) 1800 (x1800)

Pixel Fragments / sec (millions) 像素碎片/秒（百万） 240 1040 (x4.3) 14,400 (x60)

GigaFLOPS 0.64 15.6 (x25) 3090  (x4830)

Rage -id Software Ideas in Motion - SGI

1.5KW <5W

Launched OpenGL 4.3 at SIGGRAPH 2012 发布OpenGL4.3


OpenGL 4.3 Compute Shaders 计算着色器 •  Execute algorithmically general-purpose

GLSL shaders -  Operate on uniforms, images and textures

• Process graphics data in the context of the graphics pipeline -  Easier than interoperating with a compute

API IF processing ‘close to the pixel’

•  Standard part of all OpenGL 4.3 implementations -  Matches DX11 DirectCompute functionality

Physics AI Simulation Ray Tracing Imaging Global Illumination 物理 AI 模拟光线追踪成像全局光照

• 在算法上执行整体目标GLSL着色器 -  在uniforms、图片、纹理上操作

• 在图形图像管线上下文中处理图形图像数据 -  比和一个计算API IF处理“close to the pixel”交互操作更容易

• 全部OpenGL 4.3 执行的标准部分 -  和DX11 DirectCompute性能匹配


OpenGL ES •  Streamlined subset of desktop

OpenGL for embedded and mobile devices

• 用于嵌入式和移动设备的桌

面OpenGL的流线化子集

OpenGL ES 1.0 Spec Released




OpenGL ES 1.1 Platforms Released


2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

OpenGL ES 2.0 Content




OpenGL 1.5 OpenGL 2.0 OpenGL 4.3 OpenGL 2.1 OpenGL 3.0

OpenGL 3.1 OpenGL 3.2

OpenGL 3.3 OpenGL 4.0

OpenGL 4.1

OpenGL 4.2

OpenGL 4.3 is a superset of DX11

GL4/DX11-class Capabilities come

to ES ??

Fixed function 3D Pipeline

修复后功能3D管线

Programmable vertex and fragment shaders

可编程顶点和碎片着色器

ES3 is backward compatible with ES2 so

new features can be added incrementally

ES3是和ES2向后兼容的，所以可以直接添加新功能


OpenGL ES 3.0 Highlights 亮点 •  Better looking, faster performing

games and apps – at lower power -  Incorporates proven features from

OpenGL 3.3 / 4.x -  32-bit integers and floats in shader

programs -  NPOT, 3D textures, depth textures,

texture arrays -  Multiple Render Targets for deferred

rendering, Occlusion Queries -  Instanced Rendering,

Transform Feedback …

•  Make life better for the programmer -  Tighter requirements for supported features to

reduce implementation variability

•  Backward compatible with OpenGL ES 2.0 -  OpenGL ES 2.0 apps continue to run

unmodified

•  Standardized Texture Compression -  #1 developer request!

• 更好的外观、更高的游戏和应用性能 – 更低的能耗 -  融合经证实的OpenGL 3.3 / 4.x 的功能 -  着色器编程中32位的integer和float -  NPOT，3D纹理，深度纹理，纹理排列 -  延迟渲染、Occlusion Queries的多渲染器目标 -  实例渲染，转换反馈…

• 让编程员的生活更好

-  对降低执行可变性的支持功能提出更高的要求

• 与OpenGL ES 2.0向后兼容

-  OpenGL ES 2.0 应用程序无需修改继续运行

• 标准化纹理压缩

-  开发者的首要需求！


OpenGL ES Deployment in Mobile 移动领域部署

OpenGL ES is the 3D API used in Android, iOS and almost every other mobile and embedded OS – other than Windows

OpenGL ES是在Android，iOS和几乎每一个其他移动和嵌入式操作系统上使用的3D API，远远不只Windows

Use of 3D APIs in Mobile Devices Source: Jon Peddie Research 移动设备3D API的使用来源：Jon Peddie Research 研究公司

On PC – DirectX is used for most apps. On mobile the situaVon is reversed 在PC上 – 大多数应用使用DirectX

在移动设备上确实恰恰相反的


Processor Parallelism 处理器并行性

CPUs Multiple cores driving performance increases

多核驱动性能提高

GPUs Increasingly general purpose

data-parallel computing 不断增长的整体目标数据并行计算

Graphics APIs and Shading

Languages 图形图像API和着色语言

Multi-processor

programming – e.g. OpenMP 多处理器编程 –

如OpenMP

Emerging Intersection 新兴交叉点

Heterogeneous Computing 异构计算

OpenCL is a programming framework for heterogeneous compute resources OpenCL是一种异构计算资源的编程框架


OpenCL – Heterogeneous Computing 异构计算 •  Native framework for programming diverse parallel

computing resources -  CPU, GPU, DSP – as well as hardware blocks(!)

•  Powerful, low-level flexibility -  Foundational access to compute resources for higher-level

engines, frameworks and languages

•  Embedded profile -  No need for a separate “ES” spec -  Reduces precision requirements

• 为多种并行计算资源编程的本地框架

-  CPU, GPU, DSP和硬件组块（！）

• 强大的低水平灵活性

-  为高水平引擎、框架和语言基础接入到计算资源

• 嵌入式profile

-  无需独立的”ES”规范 -  降低精确度要求

OpenCL Kernel Code

OpenCL Kernel Code

OpenCL Kernel Code

OpenCL Kernel Code

CPU CPU

OpenCL Kernel Code

OpenCL Kernel Code

OpenCL Kernel Code

OpenCL Kernel Code

CPU

GPU

GPU

One code tree can be executed on CPUs or GPUs

一个代码树可以在CPU和GPU上使用 A cross-platform, cross-vendor standard for harnessing all the

compute resources in an SOC 为巩固SOC内各种计算资源的一个跨平台，跨用户标准


OpenCL 1.2 Update 更新

•  Create an OpenCL image from a OpenGL multi-sampled texture

-  Provides more flexibility in interoperating 3D graphics and compute

•  Create 2D images from an OpenCL buffer -  Process memory structures using the advanced

properties of OpenCL images

•  Security features for WebCL implementations layered over OpenCL

-  Initialize local and private memory before a kernel begins execution

-  Query and API to terminate an OpenCL context to ensure a long running kernel does not affect system stability

•  Load an OpenCL program object from a Standard Portable Intermediate Representation (SPIR) instance

-  Increased tool chain flexibility and avoids the need to ship kernel source in commercial applications

• 从OpenGL多示例纹理创建一个OpenCL图像 -  在交互操作3D图形图像和计算中提供更多的灵活性

• 从OpenCL缓冲创建2D图像

-  使用OpenCL图像的高级性能处理存储结构

•  WebCL执行分层的OpenCL安全特性

-  在核执行之前的本地初始化和独立存储 -  队列和API终止OpenCL文本，以确保长时间核运行不会影响系统稳定性

• 从一个SPIR载入OpenCL项目目标

-  不断提高的工具链灵活性并避免了商业应用中跳过核资源的需求


Mobile Innovation Hot Spots 移动创新热点 • New platform capabilities being driven by SILICON and APIs

• 芯片和API所带动的新平台性能

Console-Class 3D Performance, Quality, Controllers and TV connectivity 遥控类3D 性能，质量，遥控器和电视连接

HTML5 Web Apps that can be

discovered on the Net and run on any platform

网页应用可以在网页上开发并在任何平台上运行

Vision - Camera as sensor Computational Photography

Gesture Processing Augmented Reality 视觉 – 摄像机传感器

计算性相片姿势处理增强现实

Sensor Fusion Devices become ‘magically’ context aware – location, usage, position 传感器融合设备成为“神奇的”文本感应 – 地点，使用，位置

•  Media and Image Streaming •  Heterogeneous Parallel Processing

•  媒体和图像流 •  异构并行处理


OpenMAX - Media Acceleration 媒体加速 •  Family of royalty-free, cross-platform open API standards for video, image stream and

camera processing

• 免版税跨平台开放API标准家族，用于视频、图像流和相机处理

Audio Codecs e.g. MP3

Applications 应用程序

“Application Layer” Native Multimedia Framework

“应用程序层” 本地多媒体框架

“Integration Layer” Media component interfaces

“整合层” 媒体组件界面

Video Codecs e.g. 264

Image Codecs e.g. JPEG

Portable access to native multimedia hardware

acceleration 可移植的接入本地多媒体硬

件加速

Portable and flexible access to media

components (codecs) 可移植的灵活的接入媒体组

件（编解码器）

Sophisticated apps that provide their own media

framework processing can interface to codecs

directly 为其提供媒体框架处理的复杂的应用程序处理可以直接和编解码器对接

OpenMAX does not define CODECs – but provides

codec interfaces OpenMAX不定义CODEC，但是提供编解码器界面


OpenMAX AL Streaming Media Framework 流媒体框架 •  Enables key video, image stream and camera

use cases -  Enables optimal hardware acceleration with app

portability

•  Create Media Objects to play and process images and video with AV sync

-  Connect to variety of input and output objects to PLAY and RECORD media

•  Full range of video effects and controls -  Including playback rate, post processing, and

image manipulation

OpenMAX AL Media Object

Processing URI

Memory

Camera

Audio Input

URI

Memory

Audio Mix

Display Window DSrc DSnk

Analog Radio

Sources Sinks EGLStream to GPU Data Tap to CPU

EGLStreams extension enables

efficient transfer of image stream to GPU

texture memory EGLStreams扩展版实现了高效图片流传输到

GPU纹理内存

OpenMAX AL includes

sophisticated camera controls

OpenMAX AL包括复杂的照相机

控制

Efficient data routing for CPU processing 为CPU处理的高效数据

路由

• 实现关键视频，图片流和相机使用案例 -  通过应用程序可移植性实现可选硬件加速

• 实现媒体目标用于播放和图片处理以及

视频与AV同步 -  连接各种输入输出目标到播放和记录媒体

• 完整的视频效果和控制

-  包括回放速率、后处理、图片管理


OpenSL ES – Advanced Audio 高级音频 •  OpenSL ES does for audio what OpenGL ES

does for graphics -  Advanced audio functionality from simple

playback to full 3D positional audio

•  Object-based native audio API for simplicity and high performance

-  Same object framework as OpenMAX AL -  Reduces development time

•  Attractive alternative to open source frameworks

-  Tightly defined specification with full conformance tests

-  Robust application portability across platforms and OS

•  OpenSL ES 对音频的作用就像OpenGL ES对图形图像的作用

-  从简单回放到完整的3D定位音频的各种高级音频功能

• 目标为基础的本地音频API实现简化和高性能 -  像OpenMAX AL的相同目标框架 -  减少开发时间

• 有吸引力的交互的开源框架 -  严格定义完整的符合性测试 -  高效的应用程序可移植性可以跨平台和跨操作系统


EGLStream – Streaming Images 流图片

•  EGL – originally embedded version of WGL -  Abstraction layer to window systems and memory buffers

•  Role has expanded to provide API interop – data and events -  EGLImages – single buffers that can be passed between APIs -  EGLStreams – provides stream of images – with AV sync -  Cross process EGLStreams – Producer and Consumer can be in different processes for performance or

security – e.g. browser compositing process

•  Android SurfaceTexture is a Java wrapper around EGLStreams -  Captures video decode or camera preview to OpenGL ES texture

•  EGL – 来源于嵌入式版本的WGL

-  窗口系统和内存缓冲的缩写层

•  功能扩展为提供API交互操作-数据和事件 -  EGLImages – 单独的缓冲可以在API间传递 -  EGLStreams – 提供图片流 – 通过AV同步

-  跨流程EGLStreams – 制造商和消费者可以在不同的功能或安全流程中 – 如浏览器编写流程

•  安卓SurfaceTexture是一个在EGLStreams周边的Java包装 -  捕捉视频解码或相机预览到OpenGL ES纹理


EGLStream – Streaming Images 流图片

Camera File URL Etc. 相机文件

MEDIA PLAYER Object

媒体播放器目标 GL_TEXTURE_EXTERNAL

OpenMAX AL Media Player is the EGLStream “Producer” and controls

production of frames. OpenMAX AL媒体播放器是

EGLStream“生产者”和产品框架控制

OpenGL ES GL_TEXTURE_EXTERNAL is the EGLStream “Consumer” and

converts video format into RGB OpenGL ES texture

OpenGL ES GL_TEXTURE_EXTERNAL 是EGLStream“消费者”，并将视频格式转换成RGB OpenGL ES纹理

? EGLStream

EGLStreams enables and hides details of video frame transport. Enables multiple buffering modes for different uses cases

e.g.: FIFO and explicit latch/release EGLStreams实现并隐藏了视频框架传输细节。实现了不用使用情况的多缓冲模式，如FIFO和

显式锁定/释放


Portable Access to Sensor Fusion 传感融合的便携式获取

Advanced Sensors Everywhere RGB and depth cameras, multi-axis

motion/position, touch and gestures, microphones, wireless controllers, haptics

keyboards, mice, track pads 高级传感无处不在

RGB和深度相机，多轴动作/位置，触摸和姿势，麦克风，无线遥控，触觉键盘，

mice，track pad

Apps Need Sophisticated Access to Sensor Data

Without coding to specific sensor hardware

应用程序需要复杂的传感数据接入无需编码到具体的传感硬件

Apps request semantic sensor information StreamInput defines possible requests, e.g.

“Provide Skeleton Position” “Am I in an elevator?” 应用程序需要语义的传感信息

StreamInput定义了可能的需求，如“提供骨骼位置”，“我是⼀一个电梯吗？”

Processing graph provides sensor data stream Utilizes optimized, smart, sensor middleware Apps can gain ‘magical’ situational awareness

处理图形提供传感数据流使用优化的，只能传感中间件

应用程序可以拥有“神奇的”感应能力

Universal Timestamps

Standardized Node Intercommunication

Input Device

Input Device

Input Device

Filter Node

Filter Node

App Filter Node

Spec expected in 2013 预计2013年推出规范


OpenVX •  Vision Hardware Acceleration Layer

-  Enables hardware vendors to implement accelerated imaging and vision algorithms

-  For use by high-level libraries or apps

•  Focus on enabling real-time vision -  On mobile and embedded systems

•  Diversity of efficient implementations -  From programmable processors, through GPUS to dedicated

hardware pipelines

• 视觉硬件加速层

-  使硬件供应商可以执行加速成像和视觉算法 -  用于高水平库和应用程序

• 关注实现实时视觉

-  在移动和嵌入式系统上

• 高效执行的多样化

-  从可编程处理器，通过GPUS到专门的硬件管线

Open source sample implementation? 开源实例执行？

Hardware vendor implementations 硬件供应商执行

OpenCV open source library 开源库

Other higher-level CV libraries

其他更高水平CV库

Application 应用

Dedicated hardware can help make vision processing performant and low-power enough for pervasive ‘always-on’ use

专門的硬件可以帮助是视觉处理实现性能，同时低能耗足够满足长期的持续使用

OpenVX does not duplicate OpenCV functionality – JUST

provides essential acceleration OpenVX不是复制OpenCV的功能 –

只是提供必要的加速


HTML5 – Cross OS App Platform 跨操作系统应用平台

•  Increasing diversity of devices creates a demand for a true cross OS programming platform

• BUT need more than “more HTML”

• 不断提高的设备多样性创造了对真正跨操作系

统编程平台的需求

• 但是我们所需要的“不仅仅是HTML”


HTML5 – Cross OS App Platform 跨操作系统应用平台

Rich Experiential Processing Multi-core CPUs Rich 2D and 3D GPU GPU Computing Multiple HD cameras Image and vision processing Video encode/decode Audio encode/decode Inertial and positional sensors

How can the Browser rapidly assimilate such diverse functionality? 浏览器如何可以迅速地吸收这些多样化的功能？

Traditional Web-content 传统网页内容丰富的体验处理多核CPU 丰富的2D和3D GPU GPU 计算多个高清相机图片和视觉处理视频编码/解码音频编码/解码内部和位置传感


Leveraging Proven Native APIs into HTML5 本地API到HTML5上的过渡 •  Leverage native API investments into the Web

-  Faster API development and deployment -  Familiar foundation reduces developer learning curve

•  Will this enable web based AR apps? -  Access to necessary hardware acceleration is there!

• 本地API到网页上的投资

-  比API开发和执行的更快 -  对基础的熟悉降低了开发者的学习曲线

• 这将实现网页基础的增强现实应用程序吗？

-  必要的硬件加速已经推出！

Native APIs shipping or working group underway

JavaScript API shipping or working group underway

WebVX? Vision

Processing 视觉处理

WebMAX? Camera control

and video processing

相机控制视频处理

Possible future JavaScript APIs

Device and Sensor APIs

Device Orientation

Working Groups

设备和传感API设备主导工作组

Native

JavaScript Canvas WebSL?

Easy to use JavaScript

Audio 使用简便

本地API已推出或工作组正在进行中

JavaScript API已推出或工作组正在进行中

未来可能的 JavaScript API


WebGL – 3D on the Web – No Plug-in! 网页上的3D – 无插件！ •  Historic opportunity to bring accelerated

3D graphics to web -  WebGL defines JavaScript binding to

OpenGL ES 2.0

•  Leveraging HTML 5 and uses <canvas> element

-  Enables a 3D context for the canvas

•  WebGL 1.0 Released at GDC March 2011 -  Mozilla, Apple, Google and Opera working

closely with GPU vendors

Availability of OpenGL and OpenGL ES on almost every web-

capable device

JavaScript binding to

OpenGL ES 2.0 Increasing JavaScript performance. HTML 5 Canvas Tag

• 历史性的机遇：将3D图形图像带入网页 -  WebGL定义了JavaScript绑定到OpenGL ES

2.0

• 衡量HTML 5 并使用 <canvas> 元素 -  为canvas实现3D上下文

•  2011年3月在GDC上发布了WebGL 1.0 -  Mozilla, Apple, Google 和 Opera与GPU供应

商紧密合作

几乎在每个网页设备上都可以获得的OpenGL和OpenGL ES

不断提高的JavaScript性能


WebCL – Parallel Computing for the Web 网页并行计算 •  JavaScript bindings to OpenCL APIs

-  Enables initiation of Kernels written in OpenCL C within the browser

• Bindings stay close to the OpenCL standard

-  Maximum flexibility to provide a foundation for higher-level middleware

-  Minimal language modifications for 100% security and app portability

-  E.g. Mapping of CL memory objects into host memory space is not supported

• API definition underway – public draft released

-  https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html

• Compelling use cases -  Physics engines for WebGL games -  Image and video editing in browser

River Trail

JavaScript binding to OpenCL APIs to enable initiation of OpenCL C kernels

Data-parallel language extensions to JavaScript

for offload at run-time

•  JavaScript 绑定OpenCL API -  实现在浏览器上用OpenCL C编写核

• 绑定与OpenCL标准保持紧密联系 -  最大化灵活度为高水平中间件提供基础 -  最小化语言修改以实现100%安全和应用可移植性

-  如CL内存目标到主机内存空间mapping不支持

• API正在定义中 – 公开草本发布 -  https://cvs.khronos.org/svn/repos/registry/trunk

/public/webcl/spec/latest/index.html

• 使用案例正在完成 -  WebGL游戏屋里引擎 -  浏览器上图片和视频编辑


Mobile Operating Systems 手机操作系统

API Adoption 采用

OpenGL ES 2.0 Shipping - Android 2.2

OpenSL ES 1.0 Shipping – Android 2.3

OpenMAX AL 1.0 Shipping - Android 4.0

EGL 1.4 Shipping under SDK -> NDK

Chrome on ICS will have WebGL. Opera and Firefox WebGL now

OpenGL 3.2 on MacOS

OpenCL 1.1 on MacOS

OpenGL ES 2.0 on iOS

Can enable on MacOS Safari iOS5 enables WebGL for iAds

Microsoft WinRT/Metro: -  Only Microsoft native APIs -  HTML5 but not yet WebGL -  只有微软本地API -  HTML5,但还没有WebGL


Extended Native APIs on Android 安卓上扩展的本地API

• Native APIs can be shipped as NDK extensions before Google Adoption -  Do not break/change existing

Google APIs

• Khronos APIs have strong momentum in the silicon community -  Evangelize Google to adopt into

standard platform

•  Extended APIs can be used by: -  Bundled apps, Market apps with API

selection -  Multiple APKs behind single multi

-APK SKU Standard

Android SDK and NDK

Extended NDK for Key use cases

Evangelize Google and ISVs to adopt enabling APIs

E.g. video, camera, imaging, sensor and composition APIs

• 在Google采用前，本地API可以作为NDK扩展进行使用 -  不要破坏/修改已有Google的API

• Khronos API拥有强大的芯片行业动力 -  传道Google采用到其标准平台

• 扩展API可以使用在： -  捆绑附带应用程序，带有API选项的市场应用

程序 -  在单一多个APK SKU背后的多个APK


In Summary 总结 •  APIs are key to enable compelling applications on advanced hardware – APIs developed

on high-end hardware are now enabling mobile devices

•  APIs no longer exist alone – they interoperate and provide input AND output processing to form a complete platform for advanced content

•  Significant cooperation underway between native and Web APIs to bring advanced visual computing to HTML5

•  Khronos is driving open standards for hardware acceleration Participate, change the industry AND get the inside edge for your products!

•  API是在高级硬件上实现具有竞争力的应用的关键 – 在高端硬件上开发的API现在已经在移动设备上实现

•  API不再单独存在了 – 其交互操作并提供输入和输出处理，以为高级内容搭建一个完整平台

• 本地和网页API直接的重大合作正在进行中，这将为HTML5带来高级视觉计算

•  Khronos带动着硬件加速开放标准，参与并改变整个行业，让你的产品真正进入到尖端市场吧！

Connecting Software to Silicon

2013 April Meetup Asia - Khronos Overview

Technology

Transcript of 2013 April Meetup Asia - Khronos Overview