MPEG-21: The 21st Century MPEG-21: The 21st Century Multimedia FrameworkMultimedia Framework

Jan Bormans, Jean Gelissen, and Andrew PerkisJan Bormans, Jean Gelissen, and Andrew Perkis

IEEE Signal Processing Magazine, IEEE Signal Processing Magazine, March 2003March 2003

OutlineOutline

Context and motivation of MPEG-21Context and motivation of MPEG-21 An overview of MPEG-21 specificationsAn overview of MPEG-21 specifications UMA and MPEG-21UMA and MPEG-21

Multimedia TechnologyMultimedia Technology

Multimedia technologyMultimedia technology provides provides content creatorscontent creators and and consumersconsumers with a myriad with a myriad of coding, access, and of coding, access, and distribution possibilities.distribution possibilities.

Individuals are Individuals are producing producing more and more digital mediamore and more digital media for both professional and for both professional and personal use.personal use.

Content creators and consumers

More and more digital media are created

Content Access and DeliveryContent Access and Delivery Communication infrastructureCommunication infrastructure is is

being put into place to enable being put into place to enable access to information and access to information and multimedia services from multimedia services from anywhereanywhere at at anytimeanytime..

Existing business modelsExisting business models for for trading physical goods must be trading physical goods must be extended and extended and new modelsnew models for for distributing and trading digital distributing and trading digital contents electronically are contents electronically are required.required.

Communicating everywhere at anytimeCommunicating everywhere at anytime

Traditional and new business models

Problems of MultimediaProblems of Multimedia

No end-to-end solutions exist that allow different No end-to-end solutions exist that allow different user communities to interact in an user communities to interact in an interoperableinteroperable and and standardizedstandardized way, thus stalling the way, thus stalling the deployment of advanced multimedia packaging deployment of advanced multimedia packaging and distribution applicationsand distribution applications

Users are not given tools to deal efficiently with Users are not given tools to deal efficiently with the intricacies the intricacies of this new multimedia usage of this new multimedia usage contextcontext

MPEG-21MPEG-21

In 2002 June, MPEG (ISO/IEC JTC1 SC29 WG11) started working on the definition of enabling normative technology for the multimedia applications of the 21st century: MPEG-21 “Multimedia FrameworkMultimedia Framework”” To enable To enable transparenttransparent and and augmentedaugmented use of multimedia use of multimedia

resources across resources across a wide range of networks and devicesa wide range of networks and devices To support transparent and highly automated transactions, To support transparent and highly automated transactions,

especially taking in accountespecially taking in account Digital rights managementDigital rights management (DRM) requirements (DRM) requirements Multimedia access and delivery using Multimedia access and delivery using heterogeneous networks and heterogeneous networks and

terminalsterminals

Digital Rights ManagementDigital Rights Management

The desire to achieve The desire to achieve interoperabilityinteroperability may be in violation may be in violation with the requirement to protect the value of the content and with the requirement to protect the value of the content and the rights of the rights holdersthe rights of the rights holders DRM systems can go against the very goal of interoperability if DRM systems can go against the very goal of interoperability if

they they use non-standardized protection mechanismuse non-standardized protection mechanism To realize an open multimedia infrastructure, To realize an open multimedia infrastructure, more interoperability more interoperability

in DRM systemsin DRM systems are crucial are crucial IPMP in MPEG seriesIPMP in MPEG series

MPEG-4MPEG-4 describes a set of standard interfaces to proprietary describes a set of standard interfaces to proprietary intellectual property management and protection (IPMP) systemsintellectual property management and protection (IPMP) systems

IPMP is at the very core of the IPMP is at the very core of the MPEG-21MPEG-21 specifications specifications

Heterogeneous Terminals and NetworksHeterogeneous Terminals and Networks

The heterogeneous terminals and networks The heterogeneous terminals and networks makes it makes it difficultdifficult for content creators and service for content creators and service providers to ensure that their content can be providers to ensure that their content can be usedused and and renderedrendered in a meaningful way in a meaningful way

Terminals with different computation and rendering capability

Networks with differing bandwidth and characteristics

Different Points of ViewsDifferent Points of Views• Content accessing and creating should be offered services with an a priori known subjective quality at a known/agreed price• Network and terminal installation/management/implementation issues should be shielded• “high-level” user parameters, subjective quality and price, need to be mapped transparently to the underlying network and terminal parameters

• Application serving the user should be able to translate the user requirements into a network “QoS” contract• The contract is handled between the user and the network and guarantees the delivery of a given QoS network service• The contract is likely to have a dynamic nature

• The impact on the end-user perception of the variation in resource requirements associated to accessing dynamic, heterogeneous content should be kept hidden or minimum• The terminal should allow for trade-offs between the resource budgets and the end-user’s perception

User

Network

Terminal

Media Scalability in MPEG-4

MPEG-21 VisionMPEG-21 Vision

MPEG-21 aims at defining a MPEG-21 aims at defining a

normative open frameworknormative open framework For multimedia delivery and For multimedia delivery and

consumption for use by all the consumption for use by all the players in the delivery and players in the delivery and consumption chainconsumption chain

To enable transparent and To enable transparent and augmented use of multimedia augmented use of multimedia resource across a wide range resource across a wide range of networks and devices used of networks and devices used by different communitiesby different communities

Elaboration Analysis

Synthesis

Composition

Packaging

Delivery

Interaction

Commission

Consumption

Library

Capture

Production Post-production

Delivery Consumption some metadata

discarded

some metadata

discarded

Current Status of MPEG-21Current Status of MPEG-21Part Current Status

1 Vision, Technology and Strategies TR

2 Digital Item Declaration (DID) FDIS

3 Digital Item Identification and Description (DII) FDIS

4 Intellectual Property Management and Protection (IPMP) CD

5 Rights Expression Language (REL) FCD

6 Rights Data Dictionary (RDD) FCD

7 Digital Item Adaptation (DIA) CD

8 Reference Software SW

9 File Format WD

10 Digital Item Processing (DIP) WD

11 Evaluation Methods for Persistent Association Technologies Request　

12 Test Bed for MPEG-21 Resource Delivery WD　

Two Essential Concepts of MPEG-21Two Essential Concepts of MPEG-21

Digital Item (DI)Digital Item (DI) ““What” of the multimedia frameworkWhat” of the multimedia framework

User (U)User (U) ““Who” of the multimedia framework Who” of the multimedia framework

• Goal of MPEG-21

Defining the technology needed to support users to exchange, access, consume, trade, and otherwise manipulate digital items in an efficient, transparent, and interoperable way.

Digital ItemsDigital Items

Digital ItemsDigital ItemsA structured digital object with a standard A structured digital object with a standard

representation, identification, and associated metadata representation, identification, and associated metadata within the MPEG-21 frameworkwithin the MPEG-21 framework

The fundamental unit of distribution and transaction The fundamental unit of distribution and transaction within the MPEG-21 framework within the MPEG-21 framework

MPEG-21 defines a set of abstract terms and concepts MPEG-21 defines a set of abstract terms and concepts to form a useful model for defining DI in DIDto form a useful model for defining DI in DIDDigital representation of some workDigital representation of some workThe unit that is acted uponThe unit that is acted upon

UsersUsers UsersUsers

Any entity that interacts within the MPEG-21 environment or Any entity that interacts within the MPEG-21 environment or makes use of DImakes use of DI

Including individuals, consumers, communities, organizations, Including individuals, consumers, communities, organizations, corporations, consortia, and governmentscorporations, consortia, and governments

A content A content providerprovider and a and a consumerconsumer are both are both usersusers.. A user may assume specific or unique rights and responsibilities A user may assume specific or unique rights and responsibilities

according to their interaction with other users within the according to their interaction with other users within the framework framework

User A User BTransaction/Use/Relationship

Digital ItemAuthorization/Value Exchange

Users are identified specifically by their relationship to another User for a certain interaction

Interactions between U and DIInteractions between U and DI

User A User B

Digital Item Declaration

Digital Item Identification and Description

Content Handling And Usage

Intellectual PropertyManagement and Protection

Terminals and Networks

Content Representation

Transaction/Use/RelationshipDigital Item

Authorization/Value Exchange

Event Reporting

Examples:•Container•Item

Examples:•Unique identifiers•Content Description

Examples:•Storage Management•Content Personalization

Examples:•Usage Permissions•Authentication

Examples:•Resource Abstraction•Resource Mgt. (QoS)

Examples:•Error Resilience•Scalability

Event Reporting provides metrics and interfaces for other six key elements

What is the structure of the fundamental unit of distribution and transaction?

What content actually has been delivered?

How is the content used and delivered?

How are rights controlled in respect of each User?

Is the content delivered over a cable line or cell phone?

Is it natural or synthetic content? How does it scale?

What reportable event has happened and how is it described?

Part 1: Vision, Technologies, and StrategyPart 1: Vision, Technologies, and Strategy

Fundamental purpose of this technical reportFundamental purpose of this technical report Define a Define a visionvision for multimedia framework to enable transparent for multimedia framework to enable transparent

and augmented use of multimedia resource across a wide range and augmented use of multimedia resource across a wide range of networks and devices to meet the needs of all usersof networks and devices to meet the needs of all users

Achieve the integration of components and standards to facilitate Achieve the integration of components and standards to facilitate harmonization of harmonization of technologiestechnologies for the creation, management, for the creation, management, transport, manipulation, distribution, and consumption of DIs. transport, manipulation, distribution, and consumption of DIs.

Define aDefine a strategy strategy for achieving a multimedia framework by the for achieving a multimedia framework by the development of specifications and standards based on well-development of specifications and standards based on well-defined functional requirements through collaboration with other defined functional requirements through collaboration with other bodies bodies

Part 2: Digital Item DeclarationPart 2: Digital Item Declaration

There are many kinds of content and probably just There are many kinds of content and probably just as many possible ways of describing it to reflect its as many possible ways of describing it to reflect its context of use.context of use.

A powerful and flexible model for DIs must be able A powerful and flexible model for DIs must be able to to Accommodate the myriad forms content can take now Accommodate the myriad forms content can take now

and in the futureand in the futureUsed to represent any DI unambiguouslyUsed to represent any DI unambiguouslyUsed to communicate DIs successfully Used to communicate DIs successfully

Part 2: Digital Item Declaration (cont.)Part 2: Digital Item Declaration (cont.)

Two Examples of DI comprising a series of media resourcesTwo Examples of DI comprising a series of media resources

MPEG-21 Music Album

Audio files (Track)

Text file(Lyrics)

Image(Cover)

Text file(Introduction)

…

Image(Artwork)

Web Page

HTML file

Image(GIF)

Image(JPEG)

Video

Links

Elephant Go!!Go!!

Scripts

?

Part 2: Digital Item Declaration (cont.)Part 2: Digital Item Declaration (cont.)

RelationshipsRelationships between the resources and how they relate between the resources and how they relate to the DI itself is defined in the DIDto the DI itself is defined in the DID

DID is a document that specifies the makeup, structure, DID is a document that specifies the makeup, structure, and organization of DIand organization of DI Three normative clausesThree normative clauses

ModelModel Abstract terms and concepts to form a model for defining DIs Abstract terms and concepts to form a model for defining DIs

RepresentationRepresentation Description of the syntax and semantics of each DI declaration elements Description of the syntax and semantics of each DI declaration elements

( represented in XML)( represented in XML) SchemaSchema

The XML schema comprising the entire grammar of the DID representation The XML schema comprising the entire grammar of the DID representation

Part 3: Digital Item Identification and DescriptionPart 3: Digital Item Identification and Description

Besides references to the resources, the DID can contain Besides references to the resources, the DID can contain information about the item itself and consisting partsinformation about the item itself and consisting parts. .

Current situationCurrent situation Proprietary identifying and description systems co-exist with Proprietary identifying and description systems co-exist with

standardized schemesstandardized schemes Some identifiers have been successfully implemented and Some identifiers have been successfully implemented and

commonly used, but they are commonly used, but they are specific to individual media typesspecific to individual media types The majority of content lacks identification and description.The majority of content lacks identification and description. There is no mechanism to ensure the identity and description There is no mechanism to ensure the identity and description

information is information is persistently associatedpersistently associated with the content with the content Through file headersThrough file headers Through Through digital watermarkingdigital watermarking

Part 3: Digital Item Identification and Description (cont.)Part 3: Digital Item Identification and Description (cont.)

DII provides a normative way to express how the DII provides a normative way to express how the identification can be identification can be expressedexpressed and and associatedassociated with DIs, with DIs, containers, components, and fragments by including containers, components, and fragments by including them in a specific place in the IDthem in a specific place in the IDDI and resources are identified by encapsulating DI and resources are identified by encapsulating Uniform Uniform

Resource Identifier (URI)Resource Identifier (URI) into the identifier element into the identifier elementLikely identifiers include descriptive, control, revision tracking Likely identifiers include descriptive, control, revision tracking

and identifying info. and identifying info.

Interoperability in DRM Interoperability in DRM

IPMP in MPEG-4IPMP in MPEG-4 HooksHooks

A set of A set of standard interfacesstandard interfaces to proprietary IPMP systems, deeply to proprietary IPMP systems, deeply embedded in MPEG-4 systemsembedded in MPEG-4 systems

If you want to play some content, you only need to plug in the right IPMP If you want to play some content, you only need to plug in the right IPMP system, and where to obtain it can be signaled in the bitstreamsystem, and where to obtain it can be signaled in the bitstream

Interoperability lossInteroperability loss E.g. a portable music player cannot download a IPMP systemE.g. a portable music player cannot download a IPMP system

Interoperability in DRMInteroperability in DRM A very A very difficultdifficult problem since standardized trust is needed problem since standardized trust is needed E.g. content owners must trust all the players that consume E.g. content owners must trust all the players that consume

contentscontents

Part 4: Intellectual Property Management and ProtectionPart 4: Intellectual Property Management and Protection

MPEG-21 part 4 defines an interoperable MPEG-21 part 4 defines an interoperable framework for IPMP, built further on IPMP work in framework for IPMP, built further on IPMP work in MPEG-4, includingMPEG-4, includingStandardizedStandardized ways of ways of

Retrieving IPMP tools from remote locations Retrieving IPMP tools from remote locations Exchanging messages between IPMP tools Exchanging messages between IPMP tools Exchanging messages between IPMP tools and terminalsExchanging messages between IPMP tools and terminals

Authentication of IPMP toolsAuthentication of IPMP tools Integrating rights expressions according to REL and Integrating rights expressions according to REL and

RDDRDD

Part 5: Rights Expression LanguagePart 5: Rights Expression Language

MPEG-21 Part 4: Rights Expression Language is a MPEG-21 Part 4: Rights Expression Language is a machine-interpretable language intended to provide machine-interpretable language intended to provide mechanisms to support use of digital content in a mechanisms to support use of digital content in a way that protects digital content and honors the way that protects digital content and honors the rights, conditions, and fees specified for digital rights, conditions, and fees specified for digital contents.contents.

It also supports specification of access and use It also supports specification of access and use controls and exchange of private or sensitive digital controls and exchange of private or sensitive digital contentcontent

Part 6: Rights Data Dictionary Part 6: Rights Data Dictionary

A dictionary of key terms required to describe rights A dictionary of key terms required to describe rights of those who control DIs, including intellectual of those who control DIs, including intellectual property rights and the permission they grants.property rights and the permission they grants.

Part 5 and Part 6 together allow the Part 5 and Part 6 together allow the expression of expression of rightsrights in an interchangeable form using a in an interchangeable form using a standardized standardized syntaxsyntax (REL) and standardized (REL) and standardized termsterms (RDD). (RDD).

Universal Multimedia AccessUniversal Multimedia Access UMA deals with the UMA deals with the

delivery of media delivery of media resources under resources under different different network conditionsnetwork conditions, , User User preferencespreferences, and , and capabilities of terminal capabilities of terminal devicesdevices..

Wired and wireless Wired and wireless systems can access systems can access the the same media resourcesame media resource providerprovider and receive media and receive media resources enabled for their resources enabled for their system capabilitysystem capability

UMA will be a driving force UMA will be a driving force behind the development of behind the development of services in 3G systems services in 3G systems

UMA Network

Broadcast Mobile Phone (3G terminals)

PDALaptop

Desktop

Media Resource Provider

The UMA concept

Part 7: Digital Item AdaptationPart 7: Digital Item Adaptation

Digital Item

Digital Item Adaptation Engine

Resource

Adaptation Engine

Description

Adaptation Engine

DIA Tools

Adapted Digital Item

DIs are subject to a resource adaptation engine, as well as a descriptor adaptation engine, DIs are subject to a resource adaptation engine, as well as a descriptor adaptation engine, which together produce the adapted DIwhich together produce the adapted DI

•Adaptation engines are non-normative

•Descriptions and format-independent mechanisms are normative

Part 7: Digital Item Adaptation (cont.)Part 7: Digital Item Adaptation (cont.)

•User preferences•Demographic info.

•Delay characteristics•Error characteristics•Bandwidth characteristics

Requirements on Usage Environment Description

•Acquisition properties•Device type and profile •Output properties•H/W and S/W properties•System properties•IPMP capabilities

Terminal capabilities

Network capabilities

•Supported transport protocol types•Supported connection types

Delivery capabilities

User info.

•Location•Type of location•Available access network•Velocity•Illumination

Natural Env.

•User roles•Type of service

Service capabilities

Interactions and relations

•Taking usage environment changes into consideration

•Dynamic updating of descriptions

Mechanisms

Vocabularies

Part 7: Digital Item Adaptation (cont.)Part 7: Digital Item Adaptation (cont.)

Requirements on media resource adaptatibilityRequirements on media resource adaptatibilityFormat descriptions independent of actual content repreFormat descriptions independent of actual content repre

sentation formatssentation formatsDescription of Description of scalablescalable content representationcontent representation formats formatsDescriptions that can be automatically extracted from thDescriptions that can be automatically extracted from th

e resource in a format-independent waye resource in a format-independent wayDescription of resources in terms of Description of resources in terms of perceived qualityperceived quality an an

d associated d associated processing complexityprocessing complexityDescription of metadata in terms of Description of metadata in terms of perceived importancperceived importanc

ee and associated and associated processing complexityprocessing complexity

Related WorkRelated Work

Mediacom 2004Mediacom 2004 Established by ITUEstablished by ITU A framework for the harmonized and coordinated development of A framework for the harmonized and coordinated development of

global multimedia communication standardsglobal multimedia communication standards Capacity-exchange related standardsCapacity-exchange related standards

W3CW3C HTTP 1.1’s content negotiating HTTP 1.1’s content negotiating Composite Capability/ Preferences Profile (CC/PP) for terminal to adapt content Composite Capability/ Preferences Profile (CC/PP) for terminal to adapt content

Web ForumWeb Forum IETFIETF ISOISO

DVB-MHPDVB-MHP

ConclusionConclusion

MPEG-21 offers exciting solutions to support MPEG-21 offers exciting solutions to support interoperable exchanging, accessing, consuming, interoperable exchanging, accessing, consuming, trading, and otherwise manipulating DIs between trading, and otherwise manipulating DIs between Users in an efficient, transparent, and interoperable Users in an efficient, transparent, and interoperable wayway

MPEG-21 provides DI Adaptation to enable UMA MPEG-21 provides DI Adaptation to enable UMA scenarios.scenarios.