MPEG Systems and 3DGC Technologiesdocuments.epfl.ch/.../OlivierAvaroMPEGSystems.pdf · MPEG Systems...

0 MPEG Systems Technologies- 27/10/2007

MPEG Systems and3DGC TechnologiesOlivier Avaro – Systems Chairman


Overview of The Presentation

Key Standards Developed in the Systems Group so FarMPEG-2 SystemsMPEG-4 Systems and BIFSMP4 File Format

Standards Developed in the Last 2 YearsMPEG-B BiM : Binarization of XML DataOpen Font FormatLASeR : Light Application Scene Representation

On-going / Future ActivitiesMPEG Application Formats Developed in Systems

• Musical Slide Show, Portable Video and Digital Broadcasting, Stereoscopic, …Web-IP-Mobile TVMultimedia eXtensible MiddlewareInformation Exchange with Virtual Worlds


MPEG-4 Systems : Architecture and BIFS


MPEG-4 File Format

ISO Base Media File Format

(14496-12)


(14496-12)

Motion JPEG2000 File Format (15444-3)

(.mj2)


(.mj2)

MPEG-4 File Format (14496-14)


AVCFile Format (14496-15)


*.mp4 files

Apple’s QuickTime (QT) File Format (*.mov)

Apple’s QuickTime (QT) File Format (*.mov)ISO File Format is based on

QT architecture.Original Design

3GPP & 3GPP2 File Format

(.3gp & .3g2)


(.3gp & .3g2)




(14496-12)


(14496-12)


(.mj2)


(.mj2)





*.mp4 files

Apple’s QuickTime (QT) File Format (*.mov)

Apple’s QuickTime (QT) File Format (*.mov)ISO File Format is based on

QT architecture.Original Design


(.3gp & .3g2)


(.3gp & .3g2)




MPEG-B BiM : Binarization of XML DataAU1 AU2 AU3

FUU FUU

Description FragmentationFragment based representation of the sceneEnables the dynamic and/or progressive transmission of descriptionsStreams the fragment in sequence (allowing dynamic evolution of the description)

Description EncodingTeM : Text-based representationBiM: Efficient binarized form.

Description IndexingIndexing functionality without the cost of a real database.Allow search the index information to determine which description fragments contain specific searched information.


New MPEG-4 standard based on OpenType font format

Complex script layout

The use of the OpenType layout tables for complex language scripts supports:

•Bi-directional text flow•Horizontal and vertical writing•Ligatures•Contextual substitution•Glyph positioning•Glyph variants

Open Font Format


LASeR : The Standard for Rich Media Services

Objectives: Standard format for Interactive RichMedia services

Standardized as MPEG-4 Part 20LASeR : Lightweight Application Scene RepresentationSAF : Simple Aggregation Format

RoadmapFinal technical V1 spec. October 2005Final technical V2 spec. April 2007Final technical V3 spec. October 2007Now : Promotion in OMA, 3GPP, DVB, FLO, …

LASeR

Binary Encoding

LASeRUpdates

LASeRExtensions

W3C/SVG Scenes


What Does LASeR Specify ?

Reference to SVG Tiny 1.2graphics, text, media, structure, hyperlink, interpolation animation, reuse, scripting, font, plugin, gradients, metadata

LASeR key extensionsclipping, pointer emulation, events, scrolling, fast switching, screen orientation…

Scene updates/commandsBinary format

compatible with MPEG-B, MPEG-7, DVB-CBMS…

Core profileJ2ME LASeR

Binary Encoding

LASeRUpdates

LASeRExtensions

W3C/SVG Scenes


LASeR Engine as Middleware

MMI

MW

NetworkStacks

FileSystem

HTTPProtocol

RTP/RTSP Protocol

MobileTV

RichMediaApps

Broadcast & Interactive

Network

PlayerApp

AV player

Java RichMediaApps

JSR 287/MHP interfaces

JVM

LASeRBrowser

LASeR Engine(C/C++ Version)

WebBrowserEngine

BrowserApp


Musical Slide Show MAF

Personal image slide showPhoto-Music albumEducational materials (foreign language exercise tools, e.g. Toefl)Storytelling, Karaoke applications

ISO Base Media and MPEG-4 File FormatMPEG-1/-2 Layer 3 (“mp3”) audio codingJPEG ISO Standard MPEG-7 Meta-data (aligned to Music Player MAF and Photo Player MAF) MPEG-4 Part-20 “LASeR” Mini Profile3GPP TS 26.245 Timed Text

Synchronizedtext (Lyric 1 )

Synchronized text(Lyric 2)



Image 1 Image 2 Image 3 Image 4 Image 5

ImageSamples

t

0 sec 5 sec 7 sec 10 sec

0 sec 4 sec 8 sec 12 sec

ImagesTime stamps

TextTime stamps

14 sec

LASeRScript

animation animation animation animation animation

TextSamples


Portable Video Player MAF

Playback of content the users generated on their own for PVP (downsize and encode), e.g. user created content (“UCC”)

Playback of content sold on disk media (mini-DVD), comparable to the DVD business

Playback of content sold over the internet, comparable to the online music store business

ISO Base Media / MPEG-4 / AVC File FormatMPEG-4 AVC Baseline Profile, Level 1.2MPEG-4 HE-AAC Profile, Level 2, Stereo 48kHzMPEG-4 Streaming Text Format for subtitlesMPEG-7 Metadata for movie information like film title, etc.JPEG ISO Standard for still images of e.g. movie postersMPEG-7 Multimedia description scheme, UsageHistory DS, HierarchicalSummary DSMPEG-21 File Format and DIDMPEG-4 LASeR

PlayArtist InfoGallery

“Cappuccino Kiss – Only U”

Scrolling text (e.g. a song title)

Background image with changing opacity value

Menu options

Digital Multimedia Broadcasting MAF

1. Storage of Mobile Broadcasting Contents2. IP Media Services3. Interchange of DMB contents between Terminals4. User-Creative DMB contents5. Metadata Applications with DMB MAF

Internet

InternetIP Network

(wired/wireless,fixed/mobile)

<XML >

</XML>

<XML >

</XML>

<XML >

</XML>

DMB-HE

IP-HE

InternetDMB Network

(terrestrial/satellite)

Contents,Metadata,Data,License

DMB-MAF Terminals

Interchange

IP-net.DMB DataServer

InternetInternet

InternetInternetIP Network

(wired/wireless,fixed/mobile)

<XML >

</XML>

<XML >

</XML>

<XML >

</XML>

<XML >

</XML>

<XML >

</XML>

<XML >

</XML>

<XML >

</XML>

DMB-HE

IP-HE

InternetInternetDMB Network

(terrestrial/satellite)

Contents,Metadata,Data,License

DMB-MAF Terminals

Interchange

IP-net.DMB DataServer

1

2

3

5

5

4

12 MPEG Systems Technologies- 27/10/200712 MPEG Systems Technologies- 27/10/2007

Stereoscopic MAF

1. Storage of Stereoscopic Richmedia Contents2. Support of the combination contents of 2D and stereoscopic sequences 3. Support of interaction functionality on stereoscopic contents4. Interchange of stereoscopic contents between mobile devices including phones5. Metadata Applications with stereoscopic MAF


Web-IP-Mobile TV

TV on IP Heavily use MetadataXML Data representing information from the Electronic Service Guide.Only the « What » is specified (semantic). No presentation layer.

ISO/MPEG - LASeRRich Media scene description formatOnly the « How » is specified (presentation). No semantic layer.

Areas of WorkDefine a way to “link” LASeR scene representation to the elements of XML structured dataDefine a way to embed LASeR scene component as a part of XML structured data to define its graphical representation


Multimedia eXtensible Middleware (MxM)

Status : Exploration – Requirement PhaseMission : A project to exploit MPEG standards in tune with industry practices and needs

OSMXM Components

OS standard APIs MXM APIs

MXMApplications

OS

Areas of WorkArchitecture Components APIsReference SoftwareOpen Source Project

APIs for :Media encoders/decodersIPMP functionalityAudio/Video playersPresentation EngineMetadata editing/parsingFile Format editing/parsingStream/consume content


Information Exchange with Virtual Worlds

Status : New Project Started – Requirement PhaseMission : Enable interoperability between virtual worldsand the real world

InteroperabilityInterfaces for displays, control devices, sensors, robotics, …Framework for security, trust, privacy, …Metadata supporting avatar mobility and platform diversificationUser profile characterizationCombining the new media paradigm with the best of traditional media

ApplicationsInterplay of Virtual and Real World Social BehaviorVirtual TravelSerious Gaming for Ambient Assisted LivingSerious Games for Disaster management


Let’s go now to 3DGC …

MPEG Systems and 3DGC Technologiesdocuments.epfl.ch/.../OlivierAvaroMPEGSystems.pdf · MPEG Systems...

Documents

Transcript of MPEG Systems and 3DGC Technologiesdocuments.epfl.ch/.../OlivierAvaroMPEGSystems.pdf · MPEG Systems...