Lecture 9 HEVC Standard

8/11/2019 Lecture 9 HEVC Standard

1/23

2014-04-08

1

U N I V E R S I T Y O F B R I T I S H C O L U M B I A

High Efficiency Video Coding(HEVC)

Dr. Panos NasiopoulosDr. Mahsa Pourazad

Introduction! Motivations for developing of new video coding standard:

" Availability of advanced technologies for capturing displaying of ultra-high definition (UHD) resolution video material. Insufficient capacity to transmit large amounts of high definition (HD)

content - let alone UHD" The need for coding efficiency superior to H.264/AVC

Increased use of parallel processing

! ISO/IEC Moving Picture Experts Group (MPEG) andITU-T Video Coding Experts Group (VCEG) establisheda Joint Collaborative Team on Video Coding (JCT-VC)for working on development of a new high-performancevideo coding standard ( 50% bit rate reduction comparedto H.264/AVC with the same visual quality )

2


2/23

2014-04-08

2

Introduction (2)

! A formal Call for Proposals (CfP) on video compressiontechnology was issued in January 2010 , and in response27 proposals were received:

" Key features of seven top high-performing proposals into asingle Test Model under Consideration (TMuC), which becamethe basis for the first HEVC reference encoder software (HM)[1].

! JCT-VC received hundreds of contributions from industryand academia for further improvement of the HEVC testmodel:

" Evaluation of submissions and including the best ones in the

HEVC standard.! After 12 JCT-VC meetings, the first version of the

standard was completed and published in January 2013 .

3

Overview

! Similar to H.264/AVC and most previous video codingstandards utilizes hybrid prediction and is Block-based.

! Several novel features:" larger block sizes" new quad-tree based block structure" new tools for supporting parallel encoding and decoding" improved motion information encoding," extended intra prediction modes," flexible transforms (different block sizes)

" extensive in-loop processing

4


3/23

2014-04-08

3

HEVC Encoder Block Diagram

5

Loop Filtering

Picture partitioning

6

! The basic block of HEVC is the Largest Coding Unit (LCU)which is 64x64 (the same concept of macroblock).

! LCU can recursively split into 4 equally-sized Coding Units(CUs)

" Content-adaptive coding tree structure comprised of CUs" Luma coding blocks : 8x8 to 64x64" Chroma coding blocks: 4x4 to 32x32

HEVC supports only 4:2:0

Partitioning of a LCU to differentsizes of CU (example)


4/23

2014-04-08

4

Picture partitioning(2)

7

Depth 0

Depth 1

Depth 2

Depth 3

! Sample on Quad-tree partitioning structure:

Prediction Units (PU)

8

! In the process of Inter and Intra prediction CUs can further split to smallerunits called Prediction Units (PUs):

! PUs can be as large as their CU root or as small as 4x4 block! Splitting CUs to PUs is not recursive and it can only be done once

N/4 x N (R)

N x N N/2 x N N x N/2 N/2 x N/2

N/4 x N (L) N x N/4 (U) N x N/4 (D)

N x N N/2 x N/2

! For inter prediction

N X N: the size of CU

! For intra prediction a CUwith the size of N X N cansplit PUs:


5/23

2014-04-08

5

Transform Units

9

! Similar to H.264/AVC HEVC applies a DCT-like transform tothe residuals to de-correlate and compact the data.

! Transform Unit (TU) is the basicunit for the transform andquantization processes.

! Each coding block can recursivelysplit to quadrant

! Each CU may contain one or moreTUs.

!

Except for 4x4 TUs the ChromaTU size is always half of Luma TUsize (for 4x4 they have equal size)

CU PartitionsTU Partitions

Demonstration of HEVC Partitioning:

10


6/23

2014-04-08

6

Slices in H.264/AVC

11

! Silces in H.264/AVC support parallel encoding/decoding and also error resiliency (independentlydecoding without using information from any otherslices).

! Flexible Macroblock Ordering (FMO) tool: dividingmacroblocks into slices in a highly flexible manner(not necessarily adjacent).

! Without FMO, slices consist of continuousmacroblocks in raster scan order.

! FMO improves error resiliency in high-losssituations, at the cost of high complexity and lowercoding efficiency (prediction across slice boundariesis disabled).

! Majority of H.264/AVC encoders do not supportFMO.

Without FMO

With FMO

Slices and Tile Structure in HEVC

12

! HEVC also supports slices, similar to slices in H.264/AVC without FMO.! HEVC introduced tiles to support parallel processing, with more flexibility

than normal slices in H.264/AVC (lower complexity than FMO)! Tiles are always rectangular.

Slice 3Slice 1

Slice 2

Slice 4

! Each tile may contain more thanone slice.

! Each slice may contain more thanone tile (each tile must be entirelywithin the slice not partially).

! In each tile, LCUs are processed ina raster scan order.

! Tiles are processed in a raster scanorder within a picture.

Tile Boundaries


7/23

2014-04-08

7

Entropy Slices

13

! HEVC supports parallelization of data processing using multi-core CPU/GPU architecture (for faster performance).

! Each slice can be divided into additional slices called entropy slices eachof which can be entropy decoded independently without reference to otherentropy slices.

! Each core of CPU handles the entropy decoding process (less decodingtime).

! Note that prior to reconstruction of entropy slices, decoded data needs tobe buffered, to support raster scan.

! Each picture contains both regular and entropy slices,! Each regular slice may contain one or more entropy slices while an entrop

slice is contained in only one slice.

Wavefront Parallel Processing

14

! Slices and tiles provide support parallel encoding and decoding at the costof breaking prediction dependencies across boundaries

! Statistics used in entropy coding have to be initialized for every slice/tile.Several LCUs have to be encoded for the entropy coder to learn thestatistics of the image content for efficient coding (causes inefficientencoding in the first several LCUs)

! Solution: Wavefront parallel processing in HEVC (higher efficiency, lesserror resilient)


8/23

2014-04-08

8

Wavefront Parallel Processing

15

! Wavefront processing is a way to achieve parallel encoding anddecoding without breaking prediction dependencies, and using as muchcontext as possible in entropy encoding.

! Start processing a new row of LCUs with a new parallel process (usuallya new thread) as soon as two LCUs have been processed in the rowabove.

! For example, when thread 4 starts encoding its initial CABACparameters will depend upon the content of six previous LCUs, sinceCABAC dependencies propagate down the first two columns.

1

22

3

4

1

Already encoded LCU

LCU currently being encoded by thread 1

CABAC dependencyand

Intra Prediction

! Intra-coded pictures are those without reference to otherpictures in the sequence.

! Intra prediction mode takes advantage of spatialsimilarities between PU s.

! Compared to H.264/AVC, more intra prediction modesare supported in HEVC.

16


9/23

2014-04-08

9

Intra Prediction(Contd)

HEVC Intra PredictionModes (35 Luma & 6Chroma)

H.264/AVC Intra PredictionModes (9 Luma & 4Chroma)

17

Intra Prediction: Intra Planar Mode

18

! It is used to predict and reconstruct smooth regions .! Average of two linear interpolations (horizontal and

vertical).! The right-column and the bottom-row are generated by

replication process from the right-top pixel and the left-bottom pixel inside the current unit


10/23


11/23

2014-04-08

11

Interpolation filter for Quarter-sample-accurate motioncompensation in H.264/AVC

21

! Compensation for motion with fractional pel-accuracy requiresinterpolation of the reference signal.

! In H.264/AVC first a 6-tap filter is applied for half-pel generation and thena bi-linear interpolation of the integer and half pel values is used toprovide 1/4 pel accuracy interpolation.

Quarter-Samples Interpolatione=(A+a+1)/2

a3 = round ( (5 E - F + 20 A + 20 B - G + 5 H )/32 ) b 3 = round ( (5 M - N + 20 A + 20 C - O + 5 P )/32 )c 1 = round ( (5 b 1 - b 2 + 20 b 3 + 20 b 4 - b 5 + 5 b 6)/32 )

Interpolation filter for Quarter-sample-accurate motioncompensation in HEVC (Luma samples)

22

! Similar to H.264/AVC, the accuracy of motion compensation in HECV is1/4 pel for luma samples.

! To obtain the non-integer luma samples, separable o ne-dimensional 8-tap and 7-tap interpolation filters are applied horizontally and vertically togenerate luma half pel and quarter pel samples respectively .


12/23

2014-04-08

12

Interpolation filter for Quarter-sample-accurate motioncompensation in HEVC (Chroma samples)

23

! The prediction values for chroma components are similarly generated byapplying a onedimensional 4-tap DCT-based interpolation filter.

! Accuracy for chroma sample prediction is 1/8 of chroma samples.

Position Filter coefficients

1/8 {-2, 58, 10, -2}

2/8 {-4, 54, 16, -2}

3/8 {-6, 46, 28, -4}

4/8 {-4, 36, 36, -4}

5/8 {-4, 28, 46, -6}

6/8 {-2, 16, 54, -4}

7/8 {-2, 10, 58, -2}

Motion Parameter Encoding and Improved Skip Mode(H.264/AVC)

24

! In H.264/AVC, motion vector (MV) coding is performed Spatially" encoding the difference between the estimated MV and the predicted one,

where the predicted MV is the median of three surrounding MVs (to the left,above and above-right).

! H.264/AVC has a SKIP mode, where no motion parameters or quantizedresiduals are encoded in the bitstream, instead the motion parametersare inferred from a co-located MB in the previous frame.

! A motion parameter set consists of a motion vector, reference pictureindex, and reference picture usage flag.


13/23

2014-04-08

13

Motion Parameter Encoding and Improved Skip Mode(HEVC)

25

! In HEVC, MVs can be predicted either spatially or temporally (using MVsfrom previously coded pictures).

! Motion merge: A number of candidate motion-parameter sets are derivedfrom spatially and temporally close PUs and one is chosen by theencoder and signalled in the bit-stream.

Motion Parameter Encoding and Improved Skip Mode(HEVC)

26

! In HEVC, MVs can be predicted either spatially or temporally (using MVsfrom previously coded pictures).

! Motion vector coding options:" Explicit encoding of motion parameters (i.e. using motion vector prediction

and encoding the MV difference and reference picture index), same conceptas H.264/AVC , but with spatial & temporal candidates

" Motion merge mode (new feature)" SKIP mode (which is closely related to motion merge).


14/23

2014-04-08

14

Motion Merge:

27

! Motion merge: A number of candidate motion-parameter sets are derivedfrom spatially and temporally close PUs . The chosen one by the encoderis signalled in the bit-stream.

! Candidates for motion merge mode:" Four spatial candidates (A1, B1, A0 and B0). If one or more of those PUs is

unavailable or intra coded B2 is also considered)." One temporal candidate ( co-located PU in the

picture closest temporally to the current onethat has already been encoded).

" zero-motion candidate! Encoder signals which candidate is used, and the

motion parameter set for the current PU is copiedfrom the selected candidate.

! Note that motion merge avoids the need to encode a motion vector forthe PU, instead only the index of a candidate in the motion merge list isencoded.

Skip Mode:

28

! HEVC SKIP mode: similar to motion merge mode in HEVC, theencoder signals the index of a motion merge candidate and themotion parameters for the current CU are copied from the selectedcandidate

! Difference with Motion Merge: in SKIP mode no residual data isencoded

" This allows areas of the picture that change very little between frames orhave constant motion to be encoded using very few bits.


15/23

2014-04-08

15

Transform (DCT-like):

29

! Similar to H.264/AVC, HEVC applies a DCT-like transform on theprediction residual.

! HEVC includes transforms that can be applied to blocks of size 4x4 to32x32 pixels .

basis vectors of the 4x4: {64, 64, 64, 64}{83, 36,-36,-83}{64,-64,-64, 64}{36,-83, 83,-36}

8x8 transform: {64, 64, 64, 64, 64, 64, 64, 64}{89, 75, 50, 18,-18,-50,-75,-89}

{83, 36,-36,-83,-83,-36, 36, 83}{75,-18,-89,-50, 50, 89, 18,-75}{64,-64,-64, 64, 64,-64,-64, 64}{50,-89, 18, 75,-75,-18, 89,-50}{36,-83, 83,-36,-36, 83,-83, 36}{18,-50, 75,-89, 89,-75, 50,-18}

Transform (DST):

30

! HEVC also incorporates a discrete sine transform (DST) which is usedfor blocks coded with some directional intra prediction modes (4x4CUs).

! When using intra prediction, the pixels close to the ones used forprediction (i.e. near the top or left boundaries) will usually be predictedmore accurately than the pixels further away.

! Residuals tend to be larger for pixels away from the boundaries.! The DST will usually be better at encoding these kinds of residuals,

because the DST basis functions start low and increase, compared tothe DCT basis functions that start high and decrease .


16/23

2014-04-08

16

Transform:

31

! The transform design of HEVC has the following properties:! 16 bit data representation before and after each transform stage

(independent of the internal bit depth)! 16 bit multipliers for all internal multiplications, no need for correction

of different norms of basis vectors during quantization/dequantization(unlike the integer transform in H.264/AVC)

! Coefficients and basis vectors of the smaller transforms are a subsetof coefficients and basis vectors of larger transforms (significantsavings in memory for storing transform matrix entries).

Loop Filtering

! Loop Fileting in HEVC consists of the following tools:# Deblocking Filter# Sample Adaptive Offset

! These techniques are intended to undo the distortion introduced in the main steps of the encoding process(prediction, transform and quantization ).

! As it is part of the prediction loop , the pictures will serveas better references for motion-compensated prediction,since they have less encoding distortion .

32


17/23

2014-04-08

17

Deblocking Filter

! An in-loop deblocking filter is used similar to the oneused in H.264/AVC to reduce blocking artifact.

! block boundaries in HEVC# Coding Units (CU) boundaries# Prediction Units (PU) boundaries# Transform Units (TU) boundaries

33

Deblocking Filter( Contd)

34

! based on the pixel gradients across the boundary andthresholds derived based on the quantization parameterin the blocks:

# whether the filter should be applied or not ,# strength of the filter

is decided.

! Improvement in both the subjective andobjective quality of the video


18/23

2014-04-08

18

Sample Adaptive Offset (SAO)

! New coding tool introduced in HEVC! It classifies reconstructed pixels into different categories

based on:# intensity or# edge properties

! It will then add an offset either of the following:# Band Offset (BO)# Edge Offset (EO)

to the pixels in each category to reduce distortion .

35

Band Offset (BO)

! Band offset (BO) classifies all pixels of a region intomultiple bands with pixels that are in the same intensityinterval.

! The intensity range is divided into 32 equal intervalsfrom zero to the maximum intensity. Therefore for 8-bitdata, bands will be 8 pixels wide.

36


19/23

2014-04-08

19

Band Offset (BO) (Cont

d)

! The 32 bands are divided into two groups .# A group consisting the central 16 bands .# A group consisting the rest 16 bands .

! The encoder decides which group of bands to applySAO to, so 16 offsets will be encoded in the bit stream.

! Example:

37

Edge Offset (EO)

! Edge Offset (EO) uses one of four one-dimensional 3-pixel following patterns to classify pixels based on theiredge direction

! The encoder selects the pattern to be used in each

region by comparing each pixel with its two neighboringpixels.

38


20/23

2014-04-08

20

Edge Offset (EO) (Cont

d)

! Each pixel can be classified as on of the following:

# peak (greater than two neighbors, category 4)# valley (less than the two neighbors, category 1)# edge (equal to one neighbor, categories 2 and 3)# none of above.

! Four offset values will be calculated for these fourcategories.

39

Category Condition 1 (valley) P < two neighbors 2 (edge) P < one neighbor && P == one neighbor 3 (edge) P > one neighbor && P == one neighbor 4 (peak) P > two neighbors

0 None of the above

Entropy coding (H.264/AVC):

40

! H.264/AVC Entropy Coding:" Context-Adaptive Variable-Length Coding (CAVLC) is the base" Context-Adaptive Binary Arithmetic Coding (CABAC) is optional (Main

and High profiles) ! CABAC can provide better coding efficiency than CAVLC, due to its

arithmetic coding engine and more sophisticated context modeling(complicated probability model for bins of binarized symbol)

! CABAC increases the coding complexity specially at higher bitrateswhere transform coefficient data has a dominant role in encodedbitstreams).


21/23


22/23

2014-04-08

22

RD Curves:

43

Conclusion:

44

! Performance of the current design of HEVC is promising (29.14% to46.73% bit rate saving)

! Subjective results confirmed the quality of HEVC encoded videos atthe same bit-rate is more than 50% better than the H.264-codedvideos (tests was perfomed by MPEG)

! MISSION ACCOMPLISHED $


23/23

2014-04-08

Conclusion:

45

! M. T. Pourazad, C. Doutre, M. Azimi, and P. Nasiopoulos, HEVC:The New Gold Standard for Video Compression, IEEE ConsumerElectronic Magazine, vol.1, issue 3, pp. 36-46, July 2012.

! G.J. Sullivan, J. Ohm, J; H. Woo-Jin, and T. Wiegand, Overview ofthe High Efficiency Video Coding (HEVC) Standard, IEEE Trans. onCircuits and Systems for Video Technology, Vol. 22(12), pp. 1649 1668, Dec. 2012.

Lecture 9 HEVC Standard

Documents

Transcript of Lecture 9 HEVC Standard