Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding

of 33 /33
Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 11, NO. 1, JANUARY 2009 Xiangyang Ji, Debin Zhao, and Wen Gao 1

Embed Size (px)

description

Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding. IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 11, NO. 1, JANUARY 2009 Xiangyang Ji , Debin Zhao, and Wen Gao. Outline. Introduction Proposed Error Concealment Strategy Experimental Results Conclusion. - PowerPoint PPT Presentation

Transcript of Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding

Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding

Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video CodingIEEE TRANSACTIONS ON MULTIMEDIA, VOL. 11, NO. 1, JANUARY 2009

Xiangyang Ji, Debin Zhao, and Wen Gao1OutlineIntroductionProposed Error Concealment StrategyExperimental ResultsConclusion

2B (Co-located Macroblock)(Temporal Distance Scaling)Direct Mode2IntroductionError concealment(EC) algorithma post-processing tool at decoderconceal the erroneous region due to transmission errors according to the correctly received information.

Temporal error concealment is one of the most important approaches to combat transmission errors.temporal replacement (TR) [3]: each damaged macroblock is directly replaced by the co-located one in the temporally previous picture with zero motion.3[3] J. W. Suh and Y. S. Hu, Error concealment based on directional interpolation, IEEE Trans. Consumer Electron., vol. 43, no. 3, pp. 295302, Aug. 1997.

Temporal error concealment is one of the most important approaches to combat transmission errors.temporal replacement (TR) [3]: each damaged macroblock is directly replaced by the co-located one in the temporally previous picture with zero motion.

3IntroductionTR can be improved by a boundary matching algorithm (BMA) [4]. a suitable motion vector for a damage MB can be selected from the candidates based on side match distortion measure.

This paper not apply BMA due to the high computational estimation.

4[4] W. M. Lam, A. R. Reibman, and B. Liu, Recovery of lost or erroneously received motion vectors, in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Process., Apr. 1993, vol. 5, pp. 417420.

BMA and pixel MB4IntroductionTo conceal the damaged MB, its neighboring MBs need to be correctly received.split the current MB and its neighboring MBs into different slices.one slice data typically fits one packet.Flexible Macroblock Ordering (FMO)

But, FMO is only supported for Baseline and Extended Profiles NOT for Main Profile and High Profile in H.264/AVC and SVC.5

PurposeTackle the whole-picture loss problem.

The advantage of the temporal relationship can be taken when hierarchical B-picture coding is used to support temporal scalability.6

Proposed StrategyTake the temporal relationship among the adjacent video pictures into account.

The motion information of the lost picture is derived simply and efficiently based on the principle of temporal direct mode (TDM).7Proposed StrategyThe MV of the damaged MB can be derived from the MVs of the co-located MBs.temporally neighboring left and/or right B-pictures at next higher temporal level.

This method could also be used to conceal lost key picture as a P-picture.8OutlineIntroductionProposed Error Concealment StrategyHierarchical B-Picture CodingMotion Parameters Recovery Based on the Enhanced TDM (ETDM_EC)Further Improvement on Motion Parameters Recovery (WTDM_EC)Experimental ResultsConclusion

9Hierarchical B-Picture Coding

10B-Picture Prediction (1/2)A B-picture allows two prediction blocks from List 0 and List 1 reference buffers which contain an arbitrary set of reference pictures in forward and/or backward directions [16].At encoder, its motion-compensated prediction signal can be obtained by11

[16] A. M. Tourapis, F.Wu, and S. Li, Direct mode coding for bipredictive slices in the H.264 standard, IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 1, pp. 119126, Jan. 2005.B-Picture Prediction (2/2)At decoder, for a block S in the lost picture , which is concealed as a B-picture, its prediction signal can be achieved by12

Motion Parameters Recovery Based on the Enhanced TDM (ETDM_EC)In the case of the whole-picture loss, the motion vectors of each block have to be recovered based on the temporal motion correlation.

Assume motion among the adjacent pictures is translational, the motion vector of the block in the lost B-picture can be estimated based on Temporal Direct Mode (TDM), as proposed in [17].13[17] Y. Chen, J. Boyce, and K. Xie, Frame loss error concealment for SVC, Joint Video Team(JVT), Doc. JVT-Q046, Oct. 2005.Temporal Direct Mode (TDM) (1/2) of the co-located block in point to the temporally most recent List 0 picture.

14

(4)Temporal Direct Mode (TDM) (2/2) of the co-located block in does not point to the temporally most recent List 0 picture.15

(5)TDM- CasesIf the picture pointed by or of the co-located block in the corresponding or picture, can not be accessed.16

(5)(4)Those pictures may not reside in the current reference buffer.16Enhanced TDM Error Concealment17

Further Improvement on Motion Parameters Recovery (WTDM_EC)For a lost B-picture which is not at the highest temporal level.the temporal motion relationship tends to considerably weaken as the temporal distances become longer.

If the key picture is lost and recovered as a P-picture, the motion vectors of its block usually can not be derived based on TDM.18TMD works well to recover the lost B-picture at the highest temporal level since it usually has short temporal distances relative to their reference pictures.18Further Improvement on Motion Parameters Recovery (WTDM_EC)19

Further Improvement on Motion Parameters Recovery (WTDM_EC)20

(6)(7)

Further Improvement on Motion Parameters Recovery (WTDM_EC)21

OutlineIntroductionProposed Error Concealment StrategyExperimental ResultsConclusion

22Experimental ParametersH.264/AVC SVC reference software: JSVM_8_6.Test sequences: Mobile, Stefan, Foreman, Bus, Tempete and Coastguard.Group of Pictures: 16.Frame rate: 30 Hz.Frame size: 352 X 288 CIF.I-picture is inserted for every 32 pictures. Quantization parameters for highest temporal level are composed of 28, 32, 36 and 40.Four packet loss patterns with average packet loss rates of 3%, 5%, 10%, and 20%23Rate Distortion Curves (1/2)24

Rate Distortion Curves (2/2)25

Quality Gain of Proposed Strategy26

Mobile (1/5)195th frame, temporal level 427

Anchor_ECProposed StrategyCorrectMobile (2/5)230th frame , temporal level 328

Anchor_ECWTDM_ECCorrectETDM_ECMobile (3/5)156th frame , temporal level 2

29

Anchor_ECWTDM_ECCorrectETDM_ECCompared with ETDM EC,

the further improvement by WTDM EC is very important since in hierarchical B-picture coding, if the lost picture is not at highest temporal level, its concealment also affects the neighboring pictures with reference to it.

The same phenomenon also can be observed in Fig. 10 for the 156th picture at temporal level 2.29Mobile (4/5)104th frame , temporal level 130

WTDM_ECCorrectAnchor_EC&ETDM_ECMobile (5/5)256th key frame31

WTDM_ECCorrectAnchor_EC&ETDM_ECTo conceal the lost key picture, Anchor EC and ETDM EC only directly copy the whole picture from the previous key picture with zero motion.

In WTDM EC, the improved motion vector recovery is used and thus, is able to significantly improve the quality of the concealed picture although its visual quality is still obviously worse than the correctly decoded one due to weak temporal motion correlation.31OutlineIntroductionProposed Error Concealment StrategyExperimental ResultsConclusion

32ConclusionThe proposed algorithm derive the motion vector of the damaged block in the lost picture by utilizing the motion information in the temporally neighboring.

The proposed method also can be easily extended to the slice loss case with the error resilient tool like FMO.

The further improvement on motion parameters recovery perhaps introduce an extra delay for video decoding within one GOP.333. since the motion vectors recovery of the lost picture at low temporal level demands the motion information decoding of the B-picture at higher temporal level.

33