© Copyright Khronos Group, 2004 - Page 1 NAB 2004 Briefing Neil Trevett Senior VP Market...

© Copyright Khronos Group, 2004 -

NAB 2004 BriefingNAB 2004 Briefing

Neil TrevettNeil TrevettSenior VP Market Development, 3DlabsSenior VP Market Development, 3Dlabs

President, Khronos President, Khronos

Jean-Luc DeryJean-Luc DeryProduct Development Manager, Discreet Product Development Manager, Discreet

Chairman, OpenML Working GroupChairman, OpenML Working Group


OpenML – Market Catalyst OpenML – Market Catalyst For Media AuthoringFor Media Authoring

OpportunityFalling costs of media

authoring hardware and software by moving from

custom designs to off-the-shelf components

ProblemNo open, cross platform

standards to enable interoperability

dmSDK

OpenGL

EvolutionOpen Source Programs,

Adopters Programs,Specification evolution

SolutionThe OpenML standard provides software interoperability between

Platforms, Operating systems, Hardware devices

OpenML 1.0 SDK announced here at

NAB 2004!

OpenML 1.0 brings key technologies into open

standardsDeployment

OpenML 1.0 SDK enables and encourages

widespread industry implementation


Open standards bodyCreating royalty-free API standards

Leveraging OpenGL-related technologiesBuilding markets for dynamic media

Small footprint 3D graphics API for

embedded systems

Media API for Dynamic Media

Authoring

MEDIAAUTHORING

MEDIADELIVERY


Khronos Member MomentumKhronos Member Momentum50 member companies and growing50 member companies and growing

Promoting Members

Contributing Members


Dynamic Media ProgrammingDynamic Media ProgrammingSynergistic integration of OpenML and OpenGL Synergistic integration of OpenML and OpenGL • What OpenGL is to GRAPHICS - 2D, 3D and image processing…

…OpenML is to MEDIA - video and audio processing• OpenML and OpenGL together create a unified programming environment

- With seamless integration of graphics and media

• Tightly integrated graphics, video and audio = DYNAMIC MEDIA

Workstation3D graphics

Professional video and audio authoring

OpenML enables interoperability of media authoring applications across: Platforms, Operating Systems, Hardware devices


?

The OpenML Digital Media SolutionThe OpenML Digital Media SolutionProgramming EnvironmentProgramming Environment

ApplicationsApplications

WindowWindow SystemSystem OpenGLOpenGL

®® MLdc ML

Synchronization

OpenGL Extensions

2D Graphics 3D Graphics Displays Video Audio


Technical Strategy for OpenML 1.0Technical Strategy for OpenML 1.0Move PROVEN technologies into Open StandardsMove PROVEN technologies into Open Standards

UST/MSCSynchronization

#4: Synchronization of diverse media streams (graphics, audio, video)

SGI’s dmSDKDigital Media Handling

#1: Configure and control the Input/Output and processing of video and digital audio ML

OpenGL ExtensionsIntegrating graphics and video

#3: Enable OpenGL hardware to add more value to the media authoring workflow

SGI’s Xdc extension to X11Display Control

#2: Configure and control display devices MLdc

Problem Solution


AdopterPackagesAdopter

Packages

Consortium Participation ModelConsortium Participation Model

PromotersPromoters

ContributorsContributors

AdoptersAdopters

RatifiedSpecifications

RatifiedSpecifications

Members

ImplementersImplementers

Participation and vote in Working Groups

Conformance Tests etc. Fee for access – fees waived for members

Anyone can download specifications and SDKs and implement royalty-free products

Conforming products can use trademark

Openly and publicly distributed – free of charge

Any organization can join. All work is done in WORKING GROUPS: WGs

Produce Final Specifications and other products. WGs open to ANY member. All work-in-progress is Confidential to the

membership

Contributor rights PLUS guaranteed seat on the Board. The Board decides on strategy: what Working Groups, how spend the budget, ratification of Final

Specifications

SDKsSDKs

Libraries, utilities and examples. Licensed for commercial use – free of charge


OpenML 1.0 SDK ReleaseOpenML 1.0 SDK ReleaseEnabling and encouraging OpenML AdoptionEnabling and encouraging OpenML Adoption

Executable ML and MLdc Executable ML and MLdc libraries and utilities to run libraries and utilities to run OpenML applications for OpenML applications for IRIX, Linux and WindowsIRIX, Linux and Windows

ML and MLdc library ML and MLdc library source and header files source and header files to enable new OpenML to enable new OpenML applications to be builtapplications to be built

Source code for example Source code for example audio and video device audio and video device

modules with full modules with full documentation documentation

Source code for example Source code for example OpenML applications OpenML applications

with full documentation with full documentation

SDK

Available today at Available today at www.khronos.orgwww.khronos.org

Free of charge for Free of charge for commercial use commercial use under Khronos under Khronos

licensing agreementlicensing agreement

Can be used as a Can be used as a starting point for starting point for

building commercial building commercial applicationsapplications


OpenML Future Development OpenML Future Development Encouraging widespread industry adoptionEncouraging widespread industry adoption

OpenML 1.0 SDK

OpenML Open Source Program

OpenML 1.0 Adopters Package

OpenML 2.0Specification

NAB 2004 SIGGRAPH 2004 IBC 2004 NAB 2005

Utilities, libraries and source code to enable rapid implementation

To enable the industry to co-develop support for many devices across all

platforms

Conformance Tests to enable conformant products to use the OpenML trademark

Next generationMedia LibrarySpecifications


Possible OpenGL/OpenML RoadmapPossible OpenGL/OpenML Roadmap

OpenGL 1.2Pbuffers for off-screen storage. Imaging Library for Filtering, Color correction, Blending, Image

enhancement, Color space conversion

OpenML 1.0OpenGL extensions for:

Synchronization, Interlace processing, Chroma resampling,

Flexible alpha processing.ML MLdc UST/MSC

OpenGL 2.0High-level shaders for

flexible and powerful image and video processing

OpenML 2.0Enhanced media synchronization, extended video format and color

space processing

OpenGL 2.1Advanced memory management and synchronization


OpenML Outreach and EngagementOpenML Outreach and EngagementIndustry Involvement will drive momentumIndustry Involvement will drive momentum• SDK Released - OpenML Forums at www.khronos.org

- Adopters can request, discuss programming and the evolution of OpenML and the SDK

• OpenML - for multimedia application vendors- Complete software environment for advanced media on wide selection of systems and devices

• OpenML – for hardware vendors- Gives more ISVs better access to hardware device capabilities

• OpenML - for end users- Better functionality across a wider range of platforms at cheaper prices

The Khronos Group encourages your participation to help grow the dynamic media market!


Any Questions?Any Questions?

OpenGL is a registered trademarks of Silicon Graphics, Inc., and OpenML is a trademark of Silicon Graphics, Inc., used with permission by the Khronos Group. All other product names, trademarks,

and/or company names are used solely for identification and belong to their respective owners.

www.khronos.org


OpenML 1.0 OpenML 1.0 Technical OverviewTechnical Overview


?

The OpenML Digital Media SolutionThe OpenML Digital Media SolutionProgramming EnvironmentProgramming Environment

ApplicationsApplications

WindowWindow SystemSystem OpenGLOpenGL

®® MLdc ML

Synchronization

OpenGL Extensions

2D Graphics 3D Graphics Displays Video Audio


OpenML’s “ML” API OpenML’s “ML” API Media device control interfaceMedia device control interface• API to configure/control video/audio device input, output and processing

- Low level - provides device independence without imposing policy

• Perfect environment for implementing higher-level software - Such as QuickTime

OpenInventorOpenInventor QuickTimeQuickTime

OpenGLOpenGL MLML

Low levelLow level(ioctl interface)(ioctl interface)

3D Media


OpenML’s “ML” API OpenML’s “ML” API Market-proven audio/video processingMarket-proven audio/video processing• ML enables abstracted query and control of media processing devices

- E.g. IEEE1394 cameras, video transcoders, OSS audio processors

• ML is used to setup processing pipelines between I/O and devices- With optimized buffering and synchronization between devices and the application- Asynchronous communication between application and media devices

ApplicationApplicationMedia Input Jacks Media Output Jacks

TranscoderTranscoder

Control over input/output parameters such as data format and quality levels

Control over transcoder operation

Buffered, synchronized message-based data communication

Synchronization and control


MLModulesMLModulesDevice Driver FrameworkDevice Driver Framework

MLML

ApplicationApplication

ML

MLmodules

user

kernel

h/w AudioAudio

ossoss

Ossaudio

Ossaudio

VideoVideo

V4lV4l

V4lvideo

V4lvideo

1394 camcorder1394 camcorder

13941394

S/WXcoders

S/WXcoders

Higher-level libraryHigher-level library

Creates device independent abstractions


OpenML’s “MLdc” APIOpenML’s “MLdc” APIAbstracted display control Abstracted display control • Allows an application to control what is displayed on monitors

- Sophisticated level of control beyond most windowing systems

• MLdc sets up channels to drive each display screen- Channels that are driven from sections of physical frame buffers - Number and type of supported channels depends on the display device

• Properties that can be controlled include:- The number of channels and where their display data is derived- Interrogating monitor information- Precise video positioning- Video format query and settings- Vertical retrace rate - Interlaced vs. progressive- Gamma ramp control- Video sync sources- Genlock state notification

Frame Buffers

Progressive ScanComputer MonitorProgressive ScanComputer Monitor

Progressive Scan HDTV Monitor

Progressive Scan HDTV Monitor

Interlaced Video Monitor

Interlaced Video Monitor

Channels


Media SynchronizationMedia SynchronizationA critical part of any authoring environmentA critical part of any authoring environment• Need to ensure that all media streams start at the same time

- And stay in synch

• Non-trivial problem to solve- Devices from various manufacturers - Device latencies are unpredictable- OS scheduling latencies

Audio Device AAudio Device A

Video Device AVideo Device A

Audio Device BAudio Device B

Gfx Device BGfx Device B

Gfx Device AGfx Device A

ApplicationApplication Do all the streams come out in sync?


The UST/MSC/SBC SolutionThe UST/MSC/SBC SolutionAccurate media synchronizationAccurate media synchronization• UST - Unadjusted System Time – time reference for software and devices

- 64-bit monotonic counter with nanosecond resolution, microsecond system-wide accuracy

• MSC - Media Stream Counter - incremented for each channel sample- i.e. Video - per frame, Graphics - per retrace, audio - per audio sample, e.g. 44.1KHz

• SBC – Swap Buffer Count - incremented on graphics buffer swap- Not necessarily in synch with retrace

UST

Graphics devices

Digital audio/video devicesSBC

OpenGLOpenGL MLML

MSC

Output graphicFrames/ samples

Video/AudioIO samples


Using UST/MSC/SBCUsing UST/MSC/SBCRobust Synchronization – even on non real-time OSRobust Synchronization – even on non real-time OS• Every media sample is time-stamped

- Timestamp can be accessed through OpenGL and ML APIs for temporal analysis and control

• Application can synchronize output events to UST- Enabling precise timing and synchronization in non-real-time operating systems

• Applications can detect dropped samples- And take corrective action


OpenML’s OpenML’s OpenGL ExtensionsOpenGL ExtensionsVideo imaging and 3D graphics integrationVideo imaging and 3D graphics integration• OpenML is built around OpenGL• OpenML and OpenGL: A tight and synergistic integration


OpenML and OpenGLOpenML and OpenGLTight and synergistic integrationTight and synergistic integration• Compliant OpenML assumes a robust OpenGL implementation

- OpenGL 1.2.1 with GLX 1.3 (Linux/UNIX) or WGL (Windows)

• OpenML extends OpenGL with video primitives and DCC rendering features- Defines a set of OpenGL extensions that OpenML compliant hardware must implement

• Pbuffers are the fundamental method of integrating graphics and video- Enables buffering and off-screen processing of video data

• OpenGL imaging extensions are used to process video streams- Filtering, color correction, blending, image enhancement, color space conversion

• Significant opportunity for OpenGL hardware vendors to add more value- Use OpenGL graphics hardware for 3D and video acceleration


OpenML’s OpenGL ExtensionsOpenML’s OpenGL ExtensionsBringing together the worlds of 3D and videoBringing together the worlds of 3D and video• Synchronization control

- Primitives to control UST/MSC synchronization from OpenGL- Functions added: WaitForMSC, SwapBuffersMSC, GetSyncValues

• Interlace support- Based on two previous extensions: SGIX_interlace, INGR_interlace_read

Enables reading and writing of the frame buffer while skipping every other line- Enables processing (e.g., convolution) on interlaced video streams

• Texture color mask management to enable advanced compositing- Enables independent loading of RGB images and Alpha masks in texture memory- Recently promoted from an SGI extension to an ARB extension- Works for textures like the OpenGL color mask works for the color buffer

• Chroma resampling to support pixel formats such as YCrCb- Upsampling and downsampling when converting between YCrCb and RGB- Sampling options are: Replicate, Zero fill, Average

© Copyright Khronos Group, 2004 - Page 1 NAB 2004 Briefing Neil Trevett Senior VP Market...

Documents

Transcript of © Copyright Khronos Group, 2004 - Page 1 NAB 2004 Briefing Neil Trevett Senior VP Market...