H264 final

download H264 final

of 22

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of H264 final

  • 1. 7FIRST QUARTER 2004 1540-7977/04/$20.002004 IEEE IEEE CIRCUITS AND SYSTEMS MAGAZINEVideocodingwithH.264/AVC:Tools, Performance, and ComplexityEYEWIRE;DIGITALSTOCK;COMSTOCK,INC.1998Jrn Ostermann, Jan Bormans, Peter List,Detlev Marpe, Matthias Narroschke,Fernando Pereira, Thomas Stockhammer, and Thomas WediH.264/AVC, the result of the collaboration between the ISO/IECMoving Picture Experts Group and the ITU-T Video CodingExperts Group, is the latest standard for video coding. The goalsof this standardization effort were enhanced compression effi-ciency, network friendly video representation for interactive(video telephony) and non-interactive applications (broadcast,streaming, storage, video on demand). H.264/AVC providesgains in compression efficiency of up to 50% over a wide rangeof bit rates and video resolutions compared to previous stan-dards. Compared to previous standards, the decoder complexityis about four times that of MPEG-2 and two times that ofMPEG-4 Visual Simple Profile. This paper provides an overviewof the new tools, features and complexity of H.264/AVC.Index TermsH.263, H.264, JVT, MPEG-1, MPEG-2,MPEG-4, standards, video coding, motion compensation,transform coding, streamingAbstractFeature

2. 1. IntroductionThe new video coding standard RecommendationH.264 of ITU-T also known as International Stan-dard 14496-10 or MPEG-4 part 10 Advanced VideoCoding (AVC) of ISO/IEC [1] is the latest standard in asequence of the video coding standards H.261 (1990) [2],MPEG-1 Video (1993) [3], MPEG-2 Video (1994) [4], H.263(1995, 1997) [5], MPEG-4 Visual or part 2 (1998) [6]. Theseprevious standards reflect the technological progress invideo compression and the adaptation of video coding todifferent applications and networks. Applications rangefrom video telephony (H.261) to consumer video on CD(MPEG-1) and broadcast of standard definition or highdefinition TV (MPEG-2). Networks used for video commu-nications include switched networks such as PSTN(H.263, MPEG-4) or ISDN (H.261) and packet networks likeATM (MPEG-2, MPEG-4), the Internet (H.263, MPEG-4) ormobile networks (H.263, MPEG-4). The importance of newnetwork access technologies like cable modem, xDSL,and UMTS created demand for the new video coding stan-dard H.264/AVC, providing enhanced video compressionperformance in view of interactive applications like videotelephony requiring a low latency system and non-inter-active applications like storage, broadcast, and streamingof standard definition TV where the focus is on high cod-ing efficiency. Special consideration had to be given to theperformance when using error prone networks like mobilechannels (bit errors) for UMTS and GSM or the Internet(packet loss) over cable modems, or xDSL. Comparing theH.264/AVC video coding tools like multiple referenceframes, 1/4 pel motion compensation, deblocking filter orinteger transform to the tools of previous video codingstandards, H.264/AVC brought inthe most algorithmic discontinu-ities in the evolution of standard-ized video coding. At the same time,H.264/AVC achieved a leap in cod-ing performance that was not fore-seen just five years ago. Thisprogress was made possible bythe video experts in ITU-T andMPEG who established the JointVideo Team (JVT) in December2001 to develop this H.264/AVCvideo coding standard.H.264/AVC was finalized inMarch 2003 and approvedby the ITU-T in May 2003.The corresponding stan-dardization documentsare downloadable fromftp://ftp.imtc-files.org/jvt-experts and the referencesoftware is available ath t t p : / / b s . h h i . d e /~suehring/tml/download.Modern video communi-cation uses digital videothat is captured from acamera or synthesizedusing appropriate tools likeanimation software. In anoptional pre-processing8 IEEE CIRCUITS AND SYSTEMS MAGAZINE FIRST QUARTER 2004H.264 toH.264 toH.324/MH.264 toRTP/IPH.264 toH.320H.264 toFile FormatTCP/IPH.264 toMPEG-2SystemsH.264/AVC Conceptual LayersVideo Coding LayerEncoderVideo Coding LayerEncoderVCL-NAL InterfaceNetwork AbstractionLayer EncoderNetwork AbstractionLayer EncoderNAL Encoder Interface NAL Decoder InterfaceTransport LayerWired Networks Wireless NetworksFigure 2. H.264/AVC in a transport environment: The network abstraction layer interfaceenables a seamless integration with stream and packet-oriented transport layers (from [7]) .Source(Video)Receiver(Video)VideoVideoPre-ProcessingPost-Processing& Error RecoveryEncodingDecodingScope of StandardBitstreamBitstreamChannel/StorageFigure 1. Scope of video coding standardization: Only the syntax and semantics ofthe bitstream and its decoding are defined.Jrn Ostermann is with the Institut fr Theoretische Nachrichtentechnik und Informationsverarbeitung, University of Hannover, Hannover, Ger-many. Jan Bormans is with IMEC, Leuven, Belgium. Peter List is with Deutsche Telecom, T-Systems, Darmstadt, Germany. Detlev Marpe is withthe Fraunhofer-Institute for Telecommunications, Heinrich Hertz Institute, Berlin, Germany. Matthias Narroschke is with the Institut fr Theo-retische Nachrichtentechnik und Informationsverarbeitung, University of Hannover, Appelstr. 9a, 30167 Hannover, Germany, narrosch@tnt.uni-hannover.de. Fernando Peirera is with Instituto Superior Tcnico - Instituto de Telecomunicaes, Lisboa, Portugal. Thomas Stockhammer iswith the Institute for Communications Engineering, Munich University of Technology, Germany. Thomas Wedi is with the Institut fr Theo-retische Nachrichtentechnik und Informationsverarbeitung, University of Hannover, Hannover, Germany. 3. step (Figure 1), thesender might choose topreprocess the videousing format conversionor enhancement tech-niques. Then the en-coder encodes the videoand represents thevideo as a bit stream.After transmission of thebit stream over a com-munications network,the decoder decodes thevideo which gets dis-played after an optional post-processing step which mightinclude format conversion, filtering to suppress codingartifacts, error concealment, or video enhancement.The standard defines the syntax and semantics of thebit stream as well as the processing that the decoderneeds to perform when decoding the bit stream intovideo. Therefore, manufactures of video decoders canonly compete in areas like cost and hardware require-ments. Optional post-processing of the decoded video isanother area where different manufactures will providecompeting tools to create a decoded video stream opti-mized for the targeted application. The standard does notdefine how encoding or other video pre-processing is per-formed thus enabling manufactures to compete with theirencoders in areas like cost, coding efficiency, errorresilience and error recovery, or hardware requirements.At the same time, the standardization of the bit streamand the decoder preserves the fundamental requirementfor any communications standardinteroperability.For efficient transmission in different environmentsnot only coding efficiency is relevant, but also the seam-less and easy integration of the coded video into all cur-rent and future protocol and network architectures. Thisincludes the public Internet with best effort delivery, aswell as wireless networks expected to be a major applica-tion for the new video coding standard. The adaptation ofthe coded video representation or bitstream to differenttransport networks was typically defined in the systemsspecification in previous MPEG standards or separatestandards like H.320 or H.324. However, only the closeintegration of network adaptation and video coding canbring the best possible performance of a video communi-cation system. Therefore H.264/AVC consists of two con-ceptual layers (Figure 2). The video coding layer (VCL)defines the efficient representation of the video, and thenetwork adaptation layer (NAL) converts the VCL repre-9FIRST QUARTER 2004 IEEE CIRCUITS AND SYSTEMS MAGAZINEDecodedMacroblockIntra/InterIntra-FramePredictionMotion Comp.PredictionInverseTransformDeblockingFilterMemoryMotion DataEntropyDecoding+QuantizedCoefficientsFigure 4. Generalized block diagram of a hybrid video decoder with motion compensation.Macroblock ofInput Image Signal+PredictionError SignalTransform Quant.Intra/InterIntra-FramePredictionMotion Comp.PredictionMotionEstimationInverseTransformDeblockingFilterMemoryMotion DataQuantizedCoefficients EntropyCoding+Figure 3. Generalized block diagram of a hybrid video encoder with motion compensation: The adaptive deblocking filter andintra-frame prediction are two new tools of H.264. 4. sentation into a format suitable for specific transport lay-ers or storage media. For circuit-switched transport likeH.320, H.324M or MPEG-2, the NAL delivers the codedvideo as an ordered stream of bytes containing startcodes such that these transport layers and the decodercan robustly and simply identify the structure of the bitstream. For packet switched networks like RTP/IP orTCP/IP, the NAL delivers the coded video in packets with-out these start codes.This paper gives an overview of the working, perform-ance and hardware requirements of H.264/AVC. In Section2, the concept of standardized video coding schemes isintroduced. In Section 3, we describe the major tools ofH.264/AVC that achieve this progress in video coding per-formance. Video coder optimization is not part of thestandard. However, the successful use of the encoderrequires knowledge on encoder control that is presentedin Section 4. H.264/AVC may be used for different applica-tions with very different constraints like computationalresources, error resilience and video resolution. Section 5describes the profiles and levels of H.264/AVC that allowfor the adaptation of the decoder complexity to differentapplications. In Section 6, we give comparisons betweenH.264/AVC and previous video coding standards in termsof coding efficiency as well as hardware complexity.H.264/AVC uses many international patents, and Section 7paraphrases the current licensing model for the commer-cial use of H.264/