Rochelle 5

7/28/2019 Rochelle 5

1/44

Transcoding of an

MPEG-2 bit stream to anH.264 bit stream


2/44

What is Transcoding ?

The operation of converting video in one format toanother format.

Need: Compatibility between MPEG-2 and H.264devices

Applications: To adapt the bit rate of a compressedstream to the channel bandwidth, to change the spatial

or temporal resolution of a compressed stream etc.


3/44

Criteria considered in Heterogeneous

Transcoding Quality of the transcoded stream should be comparable

to that obtained by complete decoding and re-encodingwith full motion search and to that of the initial inputstream.

The information in the input bit stream should be re-used as much as possible to reduce multigenerationaldegradation.

The computational cost and complexity should be keptminimal.


4/44

MPEG-2 Decoder

Variable LengthDecoding

InverseScan

InverseQuantization

InverseDCT

MotionCompensation

Frame storeMemory

Decoded Pels+

+

MPEG-2 bitstream


5/44

H.264 encoder


6/44

Transcoding Algorithm


7/44

Intra frame coding

MPEG-2 H.264

Macroblock modes supported 8x8 16x16 with 4 directional modes,4x4 with 9 directional modes

Type of intra prediction Fixed prediction of D.C.

coefficient

Adaptive directional prediction

of 4x4 or 16x6 pixel blocks

Transform 8x8 DCT 4x4 Integer transform


8/44

Intra Frame Transcoding

MPEG-2bit stream

H.264bitstream

-

+

VLC Decode Inverse Quantize IDCT

ModeDecision

SpatialPrediction

4x4 IntegerTransform

QuantizeEntropy

Coding


9/44

Complexity of applying mode decisions in

transform domain Example: Vertical

prediction

Predicted block=


10/44

Intra modes in H.264/AVC

0

1

43

57

8

6

Directional modes for an intra 4x4 macroblock

Directional modes for an intra 16x16 macroblock

Directional prediction


11/44

Mode decision algorithm


12/44

Why use standard deviation?

Simple metric

Can be easily computed as the transform domain coefficients are alreadyavailable.


13/44

Post mode decision


14/44

Intra Frame transcoding results


15/44

Subjective Quality of Intra frames

MPEG-2 Input Stream H.264 bit stream obtained H.264 bit streamTest clip :Akiyo by the proposed method obtained from completeBit rate: 1 Mbps Bit rate: 768 Kbps decoding and re-encodingSpatial resolution: 352x240 of the input MPEG-2 bit

stream

Bit rate: 670 Kbps


16/44

Subjective Quality of Intra frames

MPEG-2 Input H.264 bit stream obtained H.264 bit stream bitstream by the proposed method obtained from completeTest clip: Foreman Bit rate: 1Mbps decoding and re-encodingBit rate: 1 Mbps of the input MPEG-2 bitSpatial resolution: 352x240 stream.

Bit rate: 1Mbps


17/44

Inter frame coding

MPEG-2 H.264

MC prediction with pel

accuracy

No, only pel accuracy yes

MC modes 16x16 16x16,16x8,8x16,

8x8,8x4,4x8, 4x4

Multiple reference prediction no yes

Direct modes in B frames no yes

Use of B frames as reference

framesno Allowed, can be selected by the

user


18/44

Inter frame transcoding

VLDInverse

quantise IDCTSum

residuals

MVRefinement Hierarchicalmode decisionsPassparametersInter prediction

4x4 Integer

transformQuantiseVLC/CABAC

Inverse

VLC/CABACInverse

Quantise

Motion

compensate

Store as

reference frame

Rate control

MPEG-2 bitstream

H.264 bit

stream


19/44

Inter frame transcoding

Features:

Motion vector extraction

Motion vector refinement

Motion vector reuse

Hierarchical mode decision


20/44

Motion vector extraction

Motion vectors can be extracted from the MPEG-2 bit stream

after variable length decoding.


21/44

Need for motion vector refinement

Need:

Differences in the quantization parameters of the incoming bit stream andthose selected may differ. When these differences are large it results in quality

degradation. MPEG-2 supports certain modes in which no motion information is coded.

However, since H.264 supports more fine motion estimation block sizes, asmall amount of motion may result upon refinement.

Re-evaluation of the decision to intra code macroblocks in a P frame.

Improves accuracy of the motion vectors and helps achieve compatibilitybetween pel MV accuracy in MPEG-2 and pel MV accuracy in H.264


22/44

Need for motion vector refinement

Compensates for field coding to frame coding changes and vice versa


23/44

Motion vector refinement

MPEG-2 motion vectors are refined over a one pixel window i.e. dx = dy = 3 pixels, in the most recent reference frame in List 0.

Half pixel and quarter pixel refinement is performed with the defined

window.


24/44

Search window size (dx,dy) selection

Before window size selection, differentincreasing window sizes were tested toverify the effect of varying the searchwindow size on the PSNR.

The graph for one such test clip Akiyois as follows

It was observed that the PSNRobtained for a one pixel window

closely approximated the steady statevalue and using a one pixel windowprovided a good tradeoff betweencomplexity and the PSNR.

Frame window size test (Clip Akiyo)

42.57542.58

42.585

42.59

42.595

42.6

42.605

42.61

42.615

0 2 4 6 8 10 12

--> Search Window Size s (dx,dy)

--->

PSNR

(dB)

Frame window size test


25/44

Motion vector reuse


26/44

Hierarchical mode decisions

Coding modes are compared and selected based on the sum of absolutedifference (SAD) value. In the full mode decision method, every codingmode is evaluated ,the SAD value is computed and the mode with theminimum SAD value is selected as the best mode. However , although thismethod would give the best results ,it is very computationally intensive. Forinstance, each macroblock in the P frame would have to be evaluated for16x16, 2 16x8, 2 8x16, 4 8x8, 8 8x4,8 4x8, 16 4x4 intra and skip modes

Hierarchical mode decision process makes use of the fact that after evaluatinga mode and the next level of sub partitioned modes , if sub partitioning doesnot reduce the SAD value then further sub partitioning need not be evaluated.


27/44

Hierarchical mode decisions

The top down splitting approach is shownbelow


28/44

P frame transcoding Results


29/44

P frame results

(see previous slide)


30/44


31/44

Motion vectors in P framesMPEG-2 Input Stream H.264 bit stream obtained H.264 bit streamTest clip :Akiyo by the proposed method obtained from completeBit rate: 1 Mbps Bit rate: 768 Kbps decoding and re-encodingSpatial Resolution:352x240 of the input MPEG-2 bit

streamBit rate: 670 Kbps

MPEG-2 motion vectors H.264 motion vectors after transcoding H.264 motion vectors after full motion search


32/44


33/44

B frame transcoding results


34/44


Comparison of PSNR Values for transcoding with and without the hierarchical

mode decision for the test clip Akiyo (spatial resolution:352x240)

0

10

20

30

40

50

60

0 500 1000 1500 2000 2500 3000

---> Bitrate(kbps)

--->PSNR(dB

Proposed method PSNR

Proposed method w/o hierarchical

mode decision PSNR"

Comparison of Execution time for the proposed method with and without

hierarchical mode decision for the te st clip Akiyo (spatial resolution: 352x240)

0

1000

2000

3000

4000

5000

6000

7000

0 500 1000 1500 2000 2500 3000

---> Bitrate(kbps)

--->Executiont

ime(ms)

Proposed method executiontime

Proposed method w/o

heirarchical mode decision


35/44



36/44

Motion vectors in B framesMPEG-2 Input Stream H.264 bit stream obtained H.264 bit stream.

Test clip :Akiyo by the proposed method obtained from completeBit rate: 1 Mbps Bit rate: 768 Kbps decoding and re-encodingSpatial resolution:352x240 of the input MPEG-2 bit

streamBit rate: 670 Kbps

Forward motion vectors in MPEG-2

Backward motion vectors in MPEG-2

Backward motion vectors in the H.264transcoded bit stream

Forward motion vectors in theH.264 transcoded bit stream

Forward motion vectors in the

H.264 transcoded bit streamBackward motion vectors in theH.264 transcoded bit stream


37/44

Mode decisions in B framesMPEG-2 Input Stream H.264 bit stream obtained H.264 bit stream bitstream. by the proposed method obtained from completeTest clip :Akiyo Bit rate: 768 Kbps decoding and re-encodingBit rate: 1 Mbps of the input MPEG-2 bitSpatial resolution:352x240 stream

Bit rate: 670 Kbps

16x16 modes in the MPEG-2 bit stream16x16 and sub 16x16 modes in theH.264 transcoded bit stream 16x16 and sub 16x16 modes in the

H.264 re-encoded bit stream


38/44


39/44

Comparison of the Input MPEG-2 bit stream

vs. the transcoded H.264 bit stream

The table below illustrates the comparison between the PSNR of theinput MPEG-2 bit streamand the PSNR of the transcoded H.264 bit stream obtained by transcoding 35 frames at 1Mbps

with the IBBPBBP GOP structure


40/44

Comparison of the proposed method with the

DCT domain transcoder proposed by Chang

and Messerschmitt [23]

Comparison of the Proposed method(PM), Complete decoding and

re-encoding (CDRE) and DCT Domain Transcoding(DDT)

30

32

34

36

38

40

42

44

0 5 10 15 20 25 30

--> frame number

-->PSNR(

dB)

PM

DDT

CDRE

The graph shown compares the proposed method with DCT domain transcoding [23]and complete decoding and re-encoding of a 1 Mbps MPEG-2 bit stream (test clipForeman) to an H.264 bit stream with an IBBPBBP.GOP structure at a constant bitrate.


41/44

Proposed method transcoded stream

Proposed method

Full re-encoding


42/44

References

[1] J. Youn and M-T. Sun , Motion Vector Refinement for high-performance transcoding, in IEEE Int. Conf.Consumer Electronics, Los Angeles, CA, Vol. 1, Issue 1, pp. 30-40, March 1999.[2] J. Xin, C-W. Lin and M-T. Sun, Digital Video Transcoding , Proceedings of the IEEE, Vol. 93, pp. 84-97, Jan.

2005.[3] T. Wiegand et. al., Overview of the H.264/AVC Video Coding Standard, IEEE Trans. CSVT, Vol. 13, pp.

560-576, July 2003.[4] A. Vetros, C. Christopoulos and H. Sun, Video transcoding architectures and techniques: an overview, IEEE

Signal Processing magazine, Vol. 20, pp. 18-29,March 2003.[5] H. Kalva, Issues in H.264/MPEG-2 Video Transcoding, IEEE Consumer Communications and Networking

Conf., CCNC 2004, pp 657-659, Jan 2004.[6] Information Technology-Generic coding of moving pictures and associated audio information: Video, ITU-T

Rec. H.262 (2000 E).[7] B. Haskell, A. Puri and A. Netravali, Digital Video: an introduction to MPEG -2, N.Y. Chapman and Hall,

International Thomson Pub., 1997.[8] G. Chen et. al., Efficient block size selection for MPEG-2 to H.264 transcoding, Proceedings of the 12th

annual ACM International Conference on Multimedia, pp. 300-303, Oct. 2004.[9] MPEG-2 software (version 12) from MPEG software simulation group,

http://www.mpeg.org/MPEG/MSSG/#source[10] H.264 Software (JM9.5) from http://iphome.hhi.de/suehring/tml/download/jm94.zip[11] A. Puri, X. Chen and A. Luthra, Video coding using the H.264/MPEG -4 AVC compression standard, Signal

processing: Image communication, Vol. 19, pp. 793-849, Oct. 2004.[12] B. Shen and I. Sethi, Direct feature extraction from compressed images, SPIE: Vol. 2670 Storage and

Retrieval for Image Databases IV, pp. 404-414, 1996.[13] Commercially available transcoders, PSP Video 9, http://www.pspvideo9.com[14] K.R. Rao and J. J. Hwang, Techniques and Standards for Image, Video and Audio coding, Upper Saddle

River, N.J.: Prentice Hall, 1996.
http://www.mpeg.org/MPEG/MSSG/http://www.pspvideo9.com/http://www.pspvideo9.com/http://www.mpeg.org/MPEG/MSSG/


43/44

References continued

[15] M. Ghanbari, Video Coding: an introduction to standard codecs, London, U.K.: Institution of ElectricalEngineers, 1999.

[16] I. E. G. Richardson, H.264 and MPEG-4 video compression: video coding for next generation multimedia,Chichester: Wiley, 2003.

[17]Test streams obtained from ftp://ftp.tek.com/tv/test/streams/Element/MPEG-Video/525/andhttp://www.cipr.rpi.edu/resource/sequences/sif.html

[18] Y-J. Chuang, Y-C. Huang and J-L Wu, An efficient block algorithm for splitting an 8x8 DCT into four 4x4

modified DCT used in AVC/H.264, EURASIP 2005, pp. 311-316.[19] P. Assunco and M. Ghanbari, Post Processing of MPEG-2 coded video for transmission at lower bit rates,Proc. IEEE ICASSP, pp. 1998-2001, Atlanta, GA, 1996.

[20] T. Shanableh and M. Ghanbari, Transcoding Architectures for DCT domain heterogeneous videotranscoding, Proc. IEEE ICIP, Vol. 1, pp. 433-436, Thessaloniki, Greece, Sept. 2001,.

[21] J. Xin, M.T. Sun and K. Chun, Motion re -estimation for MPEG-2 to MPEG-4 simple profile transcoding,Proc. Int. Workshop Packet Video, Pittsburgh, PA, Apr. 2002.

[22] D-Y. Chan, S-J. Lin and C-Y. Chang, A rate control scheme using Kalman filtering for H.263, Journal ofVisual Communication and Image Representation, Vol. 16, pp. 734-748, Dec. 2005.

[23] S. Liu and A. Bovik, Foveated embedded DCT domain video transcoding, Journal of Visual Communicationand Image Representation, Vol. 16, pp. 643-667, Dec. 2005.

[24] I. E. G. Richardson, Video codec design: developing image and video compression systems, Chichester:Wiley, 2002.

[25] G. Sullivan, T. Wiegand and A. Luthra, Draft of Version 4 of H.264/AVC (ITU -T Recommendation H.264and ISO/IEC 14496-10 (MPEG-4 part 10) Advanced Video Coding), JVT Doc., 14th Meeting: Hong Kong,China 18-21 Jan. 2005.

[26] G. F-Escribano et.al., Computational complexity reduction of intra frame prediction in MPEG2/H.264 video

transcoders, ICME, pp. 707-710, July 2005.
ftp://ftp.tek.com/tv/test/streams/Element/MPEG-Video/525/http://www.cipr.rpi.edu/resource/sequences/sif.htmlhttp://www.cipr.rpi.edu/resource/sequences/sif.htmlftp://ftp.tek.com/tv/test/streams/Element/MPEG-Video/525/ftp://ftp.tek.com/tv/test/streams/Element/MPEG-Video/525/ftp://ftp.tek.com/tv/test/streams/Element/MPEG-Video/525/


44/44

Rochelle 5

Documents

Transcript of Rochelle 5