Overview of Selected Current MPEG Activities

Overview of SelectedCurrent MPEG Activities

Christian Timmerer

Klagenfurt University (UNIKLU) Faculty of Technical Sciences (TEWI)Department of Information Technology (ITEC) Multimedia Communication (MMC)

http://research.timmerer.com http://blog.timmerer.com mailto:[email protected]

January 2010

A report from the 91st MPEG meeting, Kyoto, Japan

Christian Timmerer, Klagenfurt University, Austria 2

Outline

• Background / Introduction– MPEG, who the f*** is MPEG?– Benefits for using MPEG technologies and contributing to MPEG

• Selected Current MPEG Activities– Req: MVS, MMT, AIT– Sys: MPEG-M/-U/-V– Vid: 3DVC, HVC– Aud: SAOC

• ConclusionsJanuary 2010

Christian Timmerer, Klagenfurt University, Austria

Moving Picture Experts Group (MPEG)

January 2010 4-3

MPEG Family of Standards: Applications and Technologies

MPEG-1,2,4: have fueled tremendous growth in digital media content

MPEG-7: makes digital content searchable (value in the metadata!)

MPEG-21: transactions of digital items

MPEG Standards Evolution: – Coding Content Description Transactions

“Media ContentDescription”

(MPEG-7)

EnterpriseContentMgmt

“Transactions of Digital Items”

(MPEG-21)

E-Commerce Of Digital Content

ContentAdaptation

“Digital ContentCoding”

(MPEG-1,2,4)

Streaming

Broadband

VideoStorage

• Compression• Coding• Communications

• Automatic indexing• Multimedia search engines• Content-based retrieval• Personalization & summarization

Object-basedManipulation

MediaLogging

FlexibleBusinessModels

Applications

Technologies

Repurposing

© John R. Smith, Tutorial at WWW’04

Rights management Media mining and decision support


Multimedia Middleware

(MPEG-E/-M)

Moving Picture Experts Group (MPEG)

January 2010

“Umbrella” standards – Multimedia

Application Formats(MPEG-A) Tools – MPEG

System, Video, Audio

(MPEG-B/-C/-D)

Immersive User Experience

(MPEG-U/-V)

MPEG Standards Evolution: – Framework Integration Tool Sets Middleware

Immersive User Experience Platform

Platform(Advanced

IPTV Terminal)

MPEG-A: value combinations of existing standards MPEG-B/-C/-D: have fueled tremendous growth in digital

media content MPEG-E/-M: make standards accessible enabling tiny

applications MPEG-U/-V: increase the Quality of Experience

• Combination of Standards / Profiles / Levels for applications

• Payload / file / delivery formats• Generic coding tools

• Architecture• Application Programming Interfaces• Resource / quality management• Fault / integrity management• Protocol payload formats

• Widgets & Advanced User Interaction• Sensory Information• Virtual World Object Characteristics

• Elementary / Aggregated Services• APIs, Protocols, Payload Formats

Technologies

MPEG Family of Standards: The Success Story continues …

Applications

Music Player

PhotoPlayer

Streaming

Archival

Broadcasting

Surveillance Surround Sound

Streaming

Video Tool Library

Object-based Manipulation

Metadata Coding

Tiny, Fast Application

Deployments

Multimedia Value Chain

Streaming in Heterogeneous Environments

Content Sharing, the smart way

Virtual Worlds

Multi-DeviceRendering + Interaction

Streaming

Internet-TV

Media Value

Networks

Mobile-TV

TV-Everywhere

PaymentCashing

Interactive Personalized TV


Why using MPEG and contributing to MPEG?

Why using MPEG standards?

• Leading edge SDO for multimedia coding and systems

• Open IPR, world, access (public), meeting, interface, use

• On-going support• Reference software and

conformance bitstreams available

Why contributing to MPEG standards?

• Direct, tough feedback from the experts improve➪

• Visibility of your research results impact➪

• Good networking, contact to industry research ➪project(s) (proposals)

• Patents / Licensing $$$➪

January 2010


Mobile Visual Search (MVS)

January 2010

Requirements

Applications:• SnapTell service by Amazon/A9• Point & Find service by Nokia• kooaba application by kooaba ltd• Google Goggles application by Google

Scope: Bitstream enabling efficient transmission of local image features in mobile visual search and mobile augmented reality applications.

Goals:• Interoperability between visual

search applications and visual database servers

• Improve performance of mobile visual search applications

• Facilitate efficient implementation of visual search functionality on mobile phones http://www.chiariglione.org/mpeg/working_documents.htm#Explorations

http://www.snaptell.com/

http://pointandfind.nokia.com/

http://www.kooaba.com/

http://www.google.com/mobile/goggles/

http://www.chiariglione.org/mpeg/working_documents.htm%23Explorations


MPEG Media Transport (MMT)

January 2010

Requirements

Multimedia Applications

MPEG Media Transport (MMT)MPEG Media Transport (MMT)

Objectives• Efficient delivery of media• Enable content and

network adaptive• Enable cross-layer

optimization• Support integrated services• Support DRM tools• Support content forwarding

and retransmission

Multimedia Applications

MPEG Media Transport (MMT)

Data Link Interface

Timeline Call for Proposal in its next meeting in April 2010Evaluation of submissions in its July 2010 meetingPhase I: Application layer protocols to be finalized by the first quarter of 2011

http://www.chiariglione.org/mpeg/working_documents.htm#Explorations



Advanced IPTV Terminal (AIT)

January 2010

Requirements

Service: a set of content and functions to be offered to a User within the AIT value chain.

Elementary Service: a service for which protocols and APIs are defined within AIT.

Aggregated Service: two or more elementary services.

Aim: definition of elementary services with corresponding set of protocols and APIs to enable any user in an AIT value chain to access those services in an interoperable fashion. The service definition and its composition is also within the scope of AIT.

An AIT value chain is a collection of Users, including Creators, End Users and Service Providers .




AIT: Elementary Services

January 2010

Requirements


MPEG-M: MPEG-Extensible Middleware (MXM)• A set of APIs to access possibly all MPEG

technologies• Simple methods to call complex functionalities

inside MXM engines• “Thin” applications because the complexity is in

the MXM engines• Replacement of MXM engines with better

performing ones at no cost• Creation of a global market of MXM Engines,

MXM Applications and MXM Devices

January 2010

Systems

• Part 1 – MXM Architecture and Technologies• Part 2 – MXM Application Programming Interfaces (APIs)• Part 3 – MXM Reference Software• Part 4 – MXM Protocols: specifies a set of protocols

enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection information

http://mxm.wg11.sc29.org/

http://mxm.wg11.sc29.org/


MPEG-U: Rich Media User Interfaces (RMUI)

January 2010

Systems

Objectives• Enable one to consistently deliver

a rich experience across consumer electronics and mobile devices

• Broad range of app domains (home network, broadcast, Web, mobile)

• Update UI continuously• Integration w/ media types (audio,

video, text, 2D and 3D graphics) Use Cases

Remote UI and customized/personalized UIUI aggregation from different applicationsWidget Communication, downloadable widget,secured widgetMultimodal widget http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-U

http://www.chiariglione.org/mpeg/technologies/mpeg-u/mpu-widgets/index.htm

http://www.chiariglione.org/mpeg/working_documents.htm%23MPEG-U

http://www.chiariglione.org/mpeg/technologies/mpeg-u/mpu-widgets/index.htm


MPEG-V: Media Context and Control (MCC)

January 2010

System Architecture

Pt. 1: Architecture

Pt. 3: Sensory Information

Pt. 4: Virtual World Object Characteristics

Pt. 2: Control Information

Systems

Pt. 6: Common Types and Tools

Pt. 7: Conformance and Reference Software

Pt. 5: Data Formats for Interaction Devices

http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-V

http://www.chiariglione.org/mpeg/working_documents.htm%23MPEG-V


3D Video Coding (3DVC)

Timeline• Done: Test material defined• Final depth maps + preliminary anchor coding for new sequences (until April 2010)• Final Anchor coding for new sequences (until July 2010)• Draft CfP (April 2010), internal• Preliminary and Final CfP (July/Oct. 2010)

January 2010

Video

State of the art:• ISO/IEC 23002-3 (MPEG-C Pt. 3): 2D+Depth

o Backward compatible w/ legacy deviceso Agnostic of coding formatso Limited depth range (doesn’t handle

occlusions)• ISO/IEC 14496-10 Amd: Multiview Video

Coding (MVC)o Direct coding of multiple viewso Exploits inter-camera redundancyo Rate of MVC encoded video is proportional

to the number of views




High-performance Video Coding (HVC)• Final CfP issued jointly: MPEG + VCEG• Timeline

– 01/22-02/15: formal registration period (33 pre-registered)– 02/22: coded test material shall be available at the test site – 03/02: Start of subjective tests– 04/12-13: registration/submission of documents– 04/17: subjective test results available– 04/17-23: evaluation of proposals at standardization meeting

• Test classes (* … only PSNR evaluation)– Class A*: Cropped areas of size 2560x1600 (30fps):– Class B: 1920x1080p 24 fps & 1920x1080p 50-60 fps– Class C: 832x480p 30-60 fps (WVGA)– Class D: 416x240p 30-60 fps (WQVGA)– Class E: 1280x720p 60fps

• Testing fee: 8900 CHF plus 7.6% VAT• Test coordinator: Vittorio Baroncini (FUB)

January 2010

Video

Applications ➪ Everything but nothing new!? ➪ Just more performance!?

Requirements ➪ Compression Performance ➪ Picture formats (8Kx4K) ➪ Color spaces + sampling ➪ Frame rates ➪ Scanning modes ➪ Complexity ➪ Low delay ➪ Random access, trick modes ➪ Error resilience ➪ Buffer models ➪ Interface to system layers ➪ Scalable Video Coding




Spatial Audio Object Coding (SAOC)Transmission of high-quality multi-channel audio signals• @bitrates previously used for mono/stereo signals• Backward compatibility with existing receivers

January 2010

Audio

http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-D

Spatial Audio Object Coding (SAOC)

MPEG-4 Object-Based Audio Representation• Bitrate & computational decoding complexity

grows approximately proportionally with the number of transmitted objects

• Structural complexity of the decoder can be an obstacle to implementation in many real-world applications

• High compression efficiency due to using a downmix signal plus accompanying parametric side information

• Backward compatibility with respect to existing transmission structures

http://www.chiariglione.org/mpeg/working_documents.htm%23MPEG-D


Conclusions

• What is MPEG and what are they doing (right now)?– SDO in the area of multimedia coding & systems– MVS, MMT, AIT, MXM, RMUI, MCC, 3DVC, HVC, SAOC,

etc.

• Long time commitment but sustainability

• Next meeting: 19-23 April 2010, Dresden (not that far away)

January 2010


Thank you for your attention

... questions, comments, etc. are welcome …

Ass.-Prof. Dipl.-Ing. Dr. Christian TimmererKlagenfurt University, Department of Information Technology (ITEC)

Universitätsstrasse 65-67, A-9020 Klagenfurt, [email protected]

http://research.timmerer.com/Tel: +43/463/2700 3621 Fax: +43/463/2700 99 3621

© Copyright: Christian Timmerer

January 2010

Overview of Selected Current MPEG Activities

Technology

Transcript of Overview of Selected Current MPEG Activities