H264 video coding

Click here to load reader

download H264 video coding

of 29

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of H264 video coding

  • 1. H.264 Subhrendu SarkarComputer Science,Columbia UniversityCOMS W4995 - VOIP Security9/29/2008 1

2. Introduction Video Formats and Quality Video Coding and H.264 Performance Conclusion References9/29/2008 2 3. Introduction What is H.264 ? H.264 is a Video Coding Standard alsoknown as MPEG-4 Part-10 (AVC). H.261, H.263, MPEG-1, MPEG-2 aresome predecessors of H.264.VideoPurpose of a standard Define a coded representation (or syntax) that describes visual data in a compressedfrom Latin I see form and method of decoding the syntax to reconstruct visual information. compliant encoders and decoders can successfully interoperate with each other. A Video standard specifically do not define anencoder; rather, they define the output that anencoder should produce. A decoding method is defined in each standard.9/29/20083 4. Introduction Video Formats and Quality9/29/2008 4 5. Pixel (Picture Element) Interlaced Video (Frames andFields) Bitrate and Frame rate. Typically 30 fps is good for human visual system. Higher the bitrate, better the quality of9/29/20085 video. 6. Video Formats NTSC , PAL (analogue video) PAL (Europe, Asia, Australia, etc.) 25 frames/sec SECAM (France, Russia, parts of Africa etc.) 25 frames/sec NTSC (USA, Canada, Japan, etc.) 29.97 frames/sec According to resolution VGA 640x480 (Video Graphics Array ), QVGA CIF 352x288 CIF ( Common Intermediate Format ) QCIF, SQCIF SDTV (e.g 720 x 480) HDTV (e.g 19201080 ) Color Spaces RGB (Red, Green, Blue) YUV also known as YC C (luminance,b r chroma) Y = k r R + k gG + k bB9/29/2008 6 7. YUV Sampling Formats YUV 444 YUY2 (4:2:2) YV12 or YUV420 (4:2:0)Courtesy : Images from H.264 and MPEG-4 Compression Ian Richardson9/29/2008 7 8. Video Quality Subjective Video Quality Objective Video Quality PSNR (Peak Signal to Noise Ratio)measured on a logarithmic scale and depends on themeansquared error (MSE) of between an original and animpaired Image or video frame, relative to (2 n 1) 2 (thesquare of the highest possible signal value in the image,where n is the number of bits per image sample).9/29/20088 9. IntroductionVideo Formats and QualityVideo Coding and H.2649/29/2008 9 10. A video CODEC encodes a source image or video sequence into a compressed form and decodes this to produce a copy or approximation of the source sequence. Three main functional units of a Video Encoder Spatial Model Temporal Model Entropy EncoderCourtesy : Images from H.264 and MPEG-4 Compression 10 Richardson9/29/2008 Ian 11. Macroblock, Block and Sub-Block. 16 x16 Macroblocks. 16 x 8, 8 x16, 8x8, 8x4, 4x8, 4x4Blocks. Temporal Model Prediction from the Previous VideoFrame Optical Field Flow Block based Motion Estimation and MotionCompensation RefBlock Ref BlockCurrent Macroblock9/29/200811 12. Motion vectorsSearch region, finds best matching MBDB C mv : mvx and mvy (distance betweencurrent block and the ref macroblock. mvp mvp mvp mvp - motion vector predictorA mvd - motion vector differencemvd mvp mvd = Difference (mvp, mv)Courtesy : Diagram from http://wiki.multimedia.cx/index.php?title=Motion_Prediction9/29/200812 13. 16x16 MotionFrame Fn Frame Fn-1VectorsResidual Fn Fn-1Motion Compensated Motion Compensated Reference Residual9/29/2008 13 Courtesy : Images from H.264 and MPEG-4 Compression Ian 14. I Frames, P Frames, B Frames I Frames Spatial prediction only for all MBs P Frames has I (intra) spatially predictedMB and P (inter) temporally predicted MBs. B Frames Bi-directionally predicted frames. Group of Pictures (GOP) Display OrderFrame No :0 1 2 3 4 5 6 7 8 9Frame Type : I B B P B B P B BI ... Encoding of Decoding OrderFrame No : 0 3 1 2 6 45 7 899/29/200814 15. Spatial Model Intra Macroblocks. Spatial Correlationbetween Macroblocks Temporal Model Inter macroblocks Temporal Correlationbetween Macroblocks Searching for similarmacroblocks fromreference frames. Transform Time to frequencydomain. Discrete CosineTransform is used. Theoretically not lossy.Courtesy : Diagrams from H.264 and MPEG-4 Compression Ian Richardson9/29/2008 15 16. Quantization Basically dividing the transformedcoefficients by quant values in the encoderand multiplying by the quant value in thedecoder. Lossy Helps meet bitrate constraints. Human eye is less sensitive to higherfrequency transform coefficients. Entropy Coder Reorder (Zig-Zag Scan order) Variable length coding Run Length Coding Huffman Coding Arithmetic Coding H.264 Context Adaptive Variable Length Coding.(CAVLC) Context Adaptive Binary Arithmetic Coding.9/29/2008 (CABAC)16 17. Video Encoder In Loop Deblocking Filter Courtesy : Diagrams from H.264 and MPEG-4 Compression Ian9/29/200817Richardson 18. Video Decoder Deblocking FilterCourtesy : Diagrams from H.264 and MPEG-4 Compression Ian Richardson9/29/200818 19. Deblocking FilterNon Deblocked Image Deblocked ImageCourtesy : Images from http://compression.ru/video/deblocking/9/29/200819 20. H.264 ProfilesCourtesy : Diagram from H.264 and MPEG-4 Compression Ian9/29/2008 20 Richardson 21. Introduction Video Formats and Quality Video Coding and H.264 Performance9/29/2008 21 22. Sample Videos H.263 MPEG-4 Basic MPEG-4 Improved MPEG-4 Part 10 (AVC) or H.264QUALITY CompressionComplexity9/29/2008 22 Courtesy : Videos from http://mac.sillydog.org/qt/compare.php 23. ApplicationsCourtesy : Table from H.264 and MPEG-4 Compression Ian9/29/2008 23 Richardson 24. Conclusion H.264 is a Digital Video Coding Standard H.264 gives best quality and compression whencompared to earlier codec (s). H.264 more complex and thus requires moreprocessing. Different Profiles cater to different kinds ofapplications. Tradeoff between Quality of Video, Compressionachieved and complexity of the codec9/29/200824 25. References H.264 And MPEG-4 Video Compression Ian Richardson ISO/IEC 14496-10 and ITU-T Rec. H.264,Advanced Video Coding, 2003. H.264 Reference Software Version http://iphome.hhi.de/suehring/tml/ Current software version: JM 14.29/29/200825 26. Appendix - H.264 Features Weighted Prediction Entropy Coding CAVLC (Context Adaptive VariableLength Coding) CABAC (Main profile) Context AdaptiveBinary Arithmetic Coding9/29/200826 27. Some H.264 Features Reference Pictures Slices Integer and Sub-sample predictionCourtesy : Images from H.264 and MPEG-4 Compression Ian Richardson9/29/2008 27 28. Appendix - Rate Control9/29/200828Courtesy : Images from H.264 and MPEG-4 Compression Ian 29. Courtesy : Images from H.264 and MPEG-4 Compression Ian Richardson9/29/2008 29