MPEG-21 bitstream syntax descriptions for scalable video codecs

download MPEG-21 bitstream syntax descriptions for scalable video codecs

of 19

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of MPEG-21 bitstream syntax descriptions for scalable video codecs

  • 7/27/2019 MPEG-21 bitstream syntax descriptions for scalable video codecs


    Multimedia Systems (2006) 11(5): 403421DOI 10.1007/s00530-006-0021-5

    R E G U L AR PA P E R

    Davy De Schrijver Chris Poppe Sam Lerouge Wesley De Neve Rik Van de Walle

    MPEG-21 bitstream syntax descriptions for scalable video codecs

    Published online: 8 February 2006c Springer-Verlag 2006

    Abstract In order to obtain a useful multichannel publica-tion environment, a content producer has to respect the dif-ferent terminal and network characteristics of the multime-

    dia devices of its target audience. Embedded scalable videobitstreams, together with a complementary content adapta-tion framework, give the possibility to respond to heteroge-neous usage environments. In this paper, temporally scalableH.264/MPEG-4 AVC encoded bitstreams and bitstreams en-coded by relying on the fully-embedded MC-EZBC wavelet-based codec are used. The MPEG-21 Bitstream Syntax De-scription Language (BSDL) specification is used to generatehigh-level XML descriptions of the structure of a bitstream.As such, the adaptation of a scalable video stream can berealized in the XML domain, rather than on the bitstreamitself. Different transformation technologies are comparedto each other as well. Finally, a practical setup of a video

    streaming use case is discussed by relying on the MPEG-21BSDL framework.

    Keywords Content Adaptation H.264/MPEG-4 AVC MPEG-21 BSDL Scalable Video Coding UniversalMultimedia Access

    1 Introduction

    Since the 1950s, people have been able to watch news bul-letins on the television. Now, there is the opportunity to fol-low digital news feeds on the Internet; and in the near future,people will be able to see the news on mobile devices such as

    D. De Schrijver (B) S. Lerouge W. De NeveDepartment of Electronics and Information Systems,Multimedia Lab, Ghent University IBBT,Sint-Pietersnieuwstraat 41, B-9000 Ghent, BelgiumE-mail: {davy.deschrijver, sam.lerouge, wesley.deneve}

    C. Poppe R. Van de WalleDepartment of Electronics and Information Systems, Multimedia Lab,Ghent University IBBT IMEC, Sint-Pietersnieuwstraat 41, B-9000Ghent, BelgiumE-mail: {chris.poppe, rik.vandewalle}

    cellphones, Personal Digital Assistants (PDAs), etc. Thesedifferent devices and networks all have dissimilar charac-teristics. In order to provide the end-users with the highest-

    quality content, a multichannel publication environment isrequired.

    In a multichannel publication environment, contentproviders want to give their users the possibility to consumecontent on all possible devices without having to create adifferent version (of the content) for each possible device. Inother words, a content provider wants to create content onceand then publish it on every device. This concept is betterknown as Universal Multimedia Access (UMA) [1].

    In this paper, attention will be paid to one type of con-tent, in particular digital video. The latter takes an importantrole in our modern data communication and will even be-come more important in the future. Think for instance about

    the usage of digital video in mobile data communication.Memory, processing power, and often a considerable amountof bandwidth are needed to consume a video fragment. Toextract multiple reduced bitstreams from one basic videostream (for example, bitstreams with lower spatial resolu-tion or lower quality), a scalable video bitstream is neededas well as a decision taking mechanism to determine the bestpossible stream for the targeted usage environment.

    How to adapt a bitstream in a codec independent man-ner will be discussed in this paper. The Bitstream SyntaxDescription Language (BSDL), as defined in the MPEG-21standard [2], is used to achieve this goal. MPEG-21 BSDLgenerates descriptions in the Extensible Markup Language

    (XML) of the high-level structure of a bitstream allowing torealize the adaptation in the XML domain [3]. Therefore, anXML transformation is needed. In order to achieve the trans-formations, different approaches will be compared to eachother in terms of execution times and memory consumption.

    The outline of the paper is as follows: in Sect. 2, wediscuss the functioning of the BSDL framework and howBSDL is enclosed in MPEG-21. Further in this section, asmall overview is provided about wavelet-based ScalableVideo Coding (SVC) together with an explanation on howto exploit temporal scalability in H.264/MPEG-4 part 10

  • 7/27/2019 MPEG-21 bitstream syntax descriptions for scalable video codecs


    404 D. De Schrijver et al.

    Advanced Video Coding (AVC). The section is closed byan overview of related work about bitstream structure de-scription languages and scalable coding schemes. In Sect. 3,possible Bitstream Syntax (BS) schemata are described forthe scalable video codecs of Sect. 2 and it is explainedhow scalability can be exploited in a theoretical manner. InSect. 4, we discuss how we have tested the performance of

    the BSDL framework by using the MPEG-21 reference soft-ware and the different implementations of the transforma-tions. In Sect. 5, the performance of the different steps inthe BSDL framework is discussed. A practical use case withregard to a format-agnostic video adaptation framework forstreaming applications is discussed in Sect. 6. Finally, theconclusions are provided in Sect. 7.

    2 Technologies used

    In this section, an overview is given with respect to thework flow of an MPEG-21 BSDL-based adaptation frame-work. After that, we explain how an embedded wavelet-based scalable video codec works and how temporal scal-able H.264/AVC bitstreams can be generated in an elegantmanner. We close this section with an overview of relatedwork.

    2.1 Bitstream syntax description language (BSDL)

    The Bitstream Syntax Description Language (BSD Lan-guage) is part of the Bitstream Syntax Description tool,which is in its turn embedded in the Digital Item Adaptationpart of MPEG-21 [4]. MPEG-21, also known as ISO/IEC21000, is the latest standard of the Moving Picture ExpertsGroup (MPEG). In contrast to the other MPEG standards,MPEG-21 is a generic framework for multimedia produc-tion and consumption. MPEG-21 is built around one centralconcept: the Digital Item (DI). All communication withinthe framework is based on the transaction of DIs. A Dig-ital Item is a structured digital object represented in XMLthat contains content and metadata. Because of its broadscope, MPEG-21 is divided into different parts. A completeoverview of these parts is given in [5]. The part to whichBSDL belongs is part 7: Digital Item Adaptation (DIA).

    DIA defines mechanisms for the adaptation of a DI andthe resources that the DI contains. The framework is de-veloped to achieve interoperable transparent multimedia ac-cess, hereby taking into account the network and terminalcharacteristics. DIA comprises several tools [2] and one ofthese tools is Bitstream Syntax Description (BSD). This toolconsists of two related concepts, in particular BSDL and themore generic approach generic Bitstream Syntax Schema.

    The idea behind BSDL is to develop a mechanism thatis able to automatically generate a description of the (high-level) structure of a bitstream [6]. This description is for-matted in XML and hereby, it should be easier to adapt anXML description rather than directly modifying the bits of

    the bitstream. The descriptions act as an abstraction of thecompressed bitstream since their high-level nature only re-quires a limited knowledge about the structure of the mediaresource in question. Moreover, once we have the descrip-tion of the bitstream in XML, it is possible to add extra in-formation (metadata) about the video fragment or bitstream(for example, semantic information about the stream). These

    metadata, which are encapsulated in the description, can beused to adapt a bitstream description in a more intelligentway (for example, to only extract the foreign news from acomplete news bulletin). This kind of adaptation is not di-rectly possible on the bitstream because a bitstream containsno semantic information. The (adapted) bitstream can begenerated automatically from the adapted description oncethe latter is available.

    The complete work flow of BSDL is sketched in Fig. 1and a simple example is given in Fig. 2. We start from agiven bitstream (encoded with a certain codec) and we wantto obtain a high-level description for this bitstream in XML.In the example, the bitstream is described not bit-by-bit but

    frame-by-frame (of course, a frame-based description of avideo is much more high-level than a bit-based description).Therefore, a BS schema has to be developed specifically forthis codec. This BS schema is a representation of all possible

    Fig. 1 MPEG-21 BSDL framework

    Fig. 2 Simple example of the MPEG-21 BSDL framework

  • 7/27/2019 MPEG-21 bitstream syntax descriptions for scalable video codecs


    MPEG-21 bitstream syntax descriptions for scalable video codecs 405

    kinds of bitstreams that can be generated by the encoder.A BS schema defines the structure of a possible bitstream

    just as an XML schema defines the allowable structure of agroup of XML documents [7]. Once we know what all pos-sible bitstreams look like and what the given bitstream is, itis possible to generate the bitstream description. Therefore,once we have the BS schema, we can generate a descrip-

    tion for each possible bitstream that uses the correspondingcodec. This generation can be done automatically with theBintoBSD tool as represented in the figures. How to imple-ment this engine is completely open: only the structure andthe tags that can appear within the BS schema are standard-ized. As output of the engine, we get an XML descriptionthat repres