On MPEG Modern Transportover Networks

Christian Timmerer, Michael Eberhard, Ingo Kofler, Robert Kuschnig, Michael Ransburg, Michael Sablatschan, Hermann Hellwagner

Klagenfurt University (UNIKLU) Faculty of Technical Sciences (TEWI)

Department of Information Technology (ITEC) Multimedia Communication (MMC)

http://research.timmerer.com http://blog.timmerer.com mailto:christian.timmerer@itec.uni-klu.ac.at

m16307

Acknowledgement: This work is supported in part by the European Commission in the context of the ENTHRONE (FP6-IST-038463) and P2P-Next (FP7-ICT-216217) projects, by the Austrian Research Promotion Agency (FFG) in the context of the SCALIPTV (818089) and SCALNET (CP5-022) projects, and by the Austrian

Science Fund (FWF) in the context of the ASSSV (P19159) project. Further information is available at: http://www.ist-enthrone.org/, http://www.p2p-next.org/, http://scaliptv.itec.uni-

klu.ac.at, http://www.scalnet.info, and http://www.wavelab.at/project-asssv.shtml.

Outline

• Background / Introduction

• MPEG Transport and File Formats for P2P Video Streaming

• Towards Interoperability for Cross-Layer Optimizations

• Towards Context- and Content-Aware Networks

• Conclusions

2009/04/23 2Christian Timmerer, Klagenfurt University, Austria

Background / Introduction

• 87th MPEG meeting, the requirements group has approved a document entitled MPEG Modern Transport over Networks– Envisages the development of a media-aware client– In line with MPEG philosophy of concentrating the work on the

media client

• Five areas of interest have been identified:1. Transport- and file format friendly stream format2. Cross layer optimization between video and transport layer3. Error resilience for MPEG streams4. Conversion between transport mechanisms5. Content adaptation to different networks

• This contribution aims to address areas 1, 2, and 5.

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 3

MPEG Transport and File Formats forP2P Video Streaming

• MP4 file format by itself is not a streaming format =>mediation service/device towards the actual delivery format (MPEG-2 TS / RTP)

• Peer-to-Peer (P2P) traffic is increasing =>delivery mechanism for future IPTV services– Proprietary formats: BitTorrent& extensions (e.g., for

streaming and metadata) thereof are under development

– No standardization body feeling responsible for this kind of delivery format, it will probably remain proprietary (maybe IEFT with PPSP)

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 4

MPEG Transport and File Formats forP2P Video Streaming (cont’d)

• MPEG-21 Digital Item Declaration (DID) provides support for P2P systems– May be efficiently mapped onto the MPEG-21 file format

• ISO Base Media File Format amended with timed metadata• XML and Media Resource Streaming Instructions (XSI/MRSI)

from MPEG-21 Digital Item Adaptation (DIA)– Stream and process fragments of XML-based metadata and

media resources in a synchronized way

• Conclusion. MPEG could – as part of the “Transport- and file format friendly stream format” area – start working towards a file format which explicitly supports P2P video streaming and associated metadata.

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 5

Towards Interoperability forCross-Layer Optimizations

• Cross-layer designs– Aim: increase QoS/QoEby performing coordinated actions

across the network layers => violating the protocol hierarchy and isolation model

– Approaches: bottom-up or a top-down or jointly optimizing parameters at the different layers

– Common property: compromising interoperability in favor of performance

• Increasing the interoperability of cross-layer designs by adopting an open standard – MPEG-21 Digital Item Adaptation – for describing the functional dependencies across network layers

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 6

Towards Interoperability forCross-Layer Optimizations (cont’d)

• Three-step approach(developed as part of the ENTHRONE project)

1. ENTHRONE Cross-Layer Model (EXLM): describing the relationship between QoS metrics at different levels

2. Instantiation of the EXLM by utilizing MPEG-21 metadata• AdaptationQoS (AQoS): describe the relationship between constraints, feasible

adaptation operations satisfying these constraints, and associated utilities (qualities)• Usage Environment Description (UED): context information (network conditions, terminal

capabilities, user preferences, etc.)• Universal Constraints Description (UCD): limitation and optimization constraints

3. Cross-Layer Adaptation Decision-Taking Engine (XL-ADTE)• Software module solving an optimization problem adopting any algorithm

• Conclusion– There is a need to close the interoperability gap which has been opened due

to the emergence of cross-layer designs.– Therefore, it is recommended to start towards closing this gap at the level

where it concerns MPEG (and its philosophy of concentrating the work on the media client), i.e., between the coding and the transport layer.

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 7

Towards Context- and Content-Aware Networks

• Context: any information that can be used to characterize the situation of an entity (e.g., user, terminal, network, etc.)– MPEG-21 DIA Usage Environment Description

• Content: can be described with metadata – MPEG-7 offers a comprehensive catalogue but requires out-of-

bound signaling (add’l bandwidth and synchronization efforts)

• Mismatch between content and context => adaptation engine – Transcoding: common approach but requires specific solutions

for a growing number of instances – Scalable coding: general solution but only if widely adopted =>

Scalable Video Coding (SVC)

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 8

Towards Context- andContent-Aware Networks (cont’d)

• SVC is based on AVC it may make use of Supplemental Enhancement Information (SEI)– Information pertaining to the actual media resource– Information can be used by network elements, e.g., Media-Aware Network

Elements (MANEs), making them aware of the actual content being transported

• MPEG-21 DIA UED makes these elements aware of the context and MXM protocols provide means for signaling this kind of information

• MXM defines APIs to be deployed within such elements• Scalable signaling and aggregation of this context information still an open

issue, specifically for the distributed, multi-step adaptation• Conclusion

– SEI messages are tightly coupled with the AVC standard => probably beneficial to make this technology available outside of AVC + enabling extensions thereof by various application domains

– Not only media-aware clients but also media-aware networks are becoming more and more important => interface between the coding layer and delivery layer are becoming indistinct or blurred which conveys the development of cross-layer designs and MANEs

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 9

Conclusions

• Provided an input to three areas of interest identified so far:1. Transport- and file format friendly stream format2. Cross layer optimization between video and transport

layer3. Error resilience for MPEG streams4. Conversion between transport mechanisms5. Content adaptation to different networks

• Overview about existing challenges/issues and MPEG standard support

• MPEG Modern Transport over Networks is an interesting issue but needs further exploration

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 10

Thank you for your attention

... questions, comments, etc. are welcome …

Ass.-Prof. Dipl.-Ing. Dr. Christian TimmererKlagenfurt University, Department of Information Technology (ITEC)

Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIAchristian.timmerer@itec.uni-klu.ac.at

http://research.timmerer.com/Tel: +43/463/2700 3621 Fax: +43/463/2700 3699

© Copyright: Christian Timmerer

112009/04/23 Christian Timmerer, Klagenfurt University, Austria

References

• Requirements Group, “MPEG Modern Transport (MMT) over Networks”, ISO/IEC JTC 1/SC 29/WG 11/N 10496, Lausanne, Switzerland, February 2009.

• BitTorrent, http://www.bittorrent.com/• ISO/IEC 21000-2:2005, Information technology -- Multimedia framework (MPEG-21) -- Part 2:

Digital Item Declaration, October 2005.• ISO/IEC 14496-12:2008, Information technology -- Coding of audio-visual objects -- Part 12: ISO

base media file format, October 2008.• ISO/IEC 21000-7:2007, Information technology -- Multimedia framework (MPEG-21) -- Part 7:

Digital Item Adaptation, November 2007.• M. van derSchaar and N. Sai Shankar, “Cross-Layer Wireless Multimedia Transmission:

Challenges, Principles, and New Paradigms”, IEEE Wireless Communications, vol. 3, no. 4, pp. 50–58, Aug. 2005.

• V. Kawadia and P. Kumar, “A Cautionary Perspective on Cross Layer Design”, IEEE Wireless Communications, vol. 12, no. 1, pp. 3-11.

• L. Choi, W. Kellerer, and E. Steinbach, “On Cross-Layer Design for Streaming Video Delivery in Multiuser Wireless Environments”, EURASIP Journal on Wireless Communications and Networking, vol. 2006, pp. 1-10, 2006.

• ENTHRONE, http://ist-enthrone.org/.• I. Kofler, C. Timmerer, H. Hellwagner, and T. Ahmed, “Towards MPEG-21-based Cross-layer

Multimedia Content Adaptation”, Proceedings of the 2nd International Workshop on Semantic Media Adaptation and Personalization (SMAP 2007), London, United Kingdom, December 17-18, 2007.

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 12

References (cont’d)

• A. Vetro, C. Timmerer, "Digital Item Adaptation: Overview of Standardization and Research Activities", IEEE Trans. on Multimedia, vol. 7, no. 3, pp. 418-426, Jun. 2005.

• D. Mukherjee, E. Delfosse, J.-G. Kim, Y. Wang, "Optimal Adaptation Decision-Taking for Terminal and Network Quality-of-Service", IEEE Trans. on Multimedia, vol. 7, no. 3, pp. 454-462, Jun. 2005.

• I. Kofler, C. Timmerer, H. Hellwagner, A. Hutter, F. Sanahuja, "Efficient MPEG-21-based Adaptation Decision-Taking for Scalable Multimedia Content", Proc. 14th SPIE Annual Electronic Imaging Conf. – Multimedia Computing and Networking (MMCN 2007), San Jose, CA, USA, Jan./Feb. 2007.

• T. Zahariadis, C. Lamy-Bergot, T. Schierl, K. Grüneberg, L. Celetto, C. Timmerer, “Content Adaptation Issues in the Future Internet”, accepted for publication in the Proceedings of the Future of the Internet Conference, Prague, Czech Republic, May 11-13, 2009.

• A. Vetro and C. Timmerer, “Digital Item Adaptation: Overview of Standardization and Research Activities”, IEEE Transactions on Multimedia, vol. 7, no. 3, pp. 418-426, June 2005.

• B.S. Manjunath, P. Salembier, T. Sikora, (eds.) Introduction to MPEG-7: Multimedia Content Description Interface, Wiley & Sons, April 2002.

• R. Kuschnig, I. Kofler, M. Ransburg, H. Hellwagner, “Design options and comparison of in-network H.264/SVC adaptation”, Journal of Visual Communication and Image Representation, Sept. 2008.

• ISO/IEC 29116-1, Information technology -- Supplemental media technologies – Part 1: MPEG extensible middleware (MXM) protocols, Committee Draft, March 2009.

• MPEG Extensible Middleware (MXM), http://mxm.wg11.sc29.org/.• M. Ransburg , C. Timmerer, and H. Hellwagner, “Dynamic and Distributed Multimedia Content Adaptation based

on the MPEG-21 Multimedia Framework”, in: Mathias Lux, et.al. (eds.), Multimedia Semantics: The role of Metadata, Springer, 2008.

• S. McCanne, V. Jacobson, and M. Vetterli, “Receiver-driven layered multicast”, Conference Proceedings on Applications, Technologies, Architectures, and Protocols For Computer Communications, Palo Alto, California, United States, August 28 - 30, 1996.

2009/04/23 Christian Timmerer, Klagenfurt University, Austria 13