Compression Synopsis H264-H265

19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 1

Compression Synopsish.264 vs. h.265

By

Paul HightowerCEO, Instrumentation Technology Systems

Presented to

Optical Systems Group

Fall 2016At

NASA Armstrong Flight Research Center


30 Years of Compression

Each major system has added new compression tools

JPEG

MPEG 1

MPEG 2/H.262

MPEG 4H.261

H.264

H.265

1986 1994 2000 2013

H.263

• Targeted @ video conferencing• Introduced I &P frames

• Targeted @ HD distribution• variable block size• Deblocking filters• 8 modes motion vectors• Context-adaptive binary

arithmetic coding

• Targeted @ SD/ED distribution• Introduced motion vectors• Bidirectional estimation• ½ pixel motion vectors• Intraframe prediction• More frame formats

• Targeted @ UHD distribution• 4K and 8K at up to 120 FPS• Larger more flexible block sizes

using Coding Tree Units• Larger transform size• 33 modes motion vectors• Merge/Skip vector modes

• Targeted at single images• Wavelet transform


Why Compression?

Content delivery

Television & Cable have narrow bands available to deliver content

• Compression is an enabler

• Allocations are in the 10s of Mb/sec

Internet Streaming

• Premium speed limit is 500 Mb/s

Wi-Fi

• 802.11ac max is 1,300 Mb/sec

Content Storage

After Theater Movie Sales

• DVD holds 4.7 GB

• Blu-ray holds 50 GB

• Hard Drives 1000-4000 GB now

• Cinema industries stores 10s of PB/day!

Format Raw Data Rate

SD/30 270 Mb/s

720/60 1,485 Mb/s

1080/60 2,970 Mb/s

2160/60 (4K) 11,880 Mb/s

Format Single Frame Size 2 Hours

SD/30 1.1 MB 250 GB

720/60 3.1 MB 1,400 GB

1080/60 6.2 MB 2,700 GB

2160/60 (4K) 24.8 MB 10,700 GB


Challenges of Compression

A 2 hour movie on a Blu-ray disk

Transport over GigE Ethernet; ONLY 1 Channel, not shared with other traffic

Transport via 100Mb/s Wi-Fi link Not shared with other traffic

Format Minimum Compression Ratio

SD >6:1

720/60 >30:1

1080/60 >50:1

2160/60 >200:1


SD none

720/60 2:1

1080/60 4:1

2160/60 >14:1


SD 3:1

720/60 >15:1

1080/60 >30:1

2160/60 >120:1



Objectives

Reduce Frame Size (bytes) as much as possible

• Take advantage of redundancies in the image data of a frame (spatial)

• Take advantage of image areas that are same frame to frame (temporal)

• Take advantage of information that can be derived across a group of frames (prediction)

• Create standard algorithms that analyze and transform image data into compact nuggets

Enable decoder to render visually good imagery

• Largely a subjective measure

• Take advantage of eye sensitivities to changes in brightness, color and bandwidth

• Different criteria than still imagery

Manage Latency

• The greater the compression:

The more time it takes to crunch it

The more time it takes to re-render the images

The more buffering may be needed at the decoder end


Latency

Introduction of I & P frames (MPEG) introduces interframe dependencies

Latency is dynamic and is continuously variable

• Latency varies with the number of dependent frames, scene complexity, buffer sizes,transport bandwidth, hardware and software from source to destination

Image Quality

Unlike data compression, image compression takes advantage of human perception

• Temporal and spatial image characteristics replace image areas (macroblocks)

• Encoded Pixels are replaced with transform coefficients.

• Decode Filters, motion prediction, pixel interpolation reduce the perceived errors

Bit-depth may be compromised

• Many encoders change sampling from 4:2:2 to 4:2:0 effectively reducing “shades”resolution of pixels and shift color

Can vary with the encoder, transport activity, rendering horsepower



Image Quality

Image quality is largely based on human perception

Noise perceived

Sharpness perceived

Blur

Color Space

Bandwidth sensitivities and limits

No one metric stands alone

“beauty is in the eye of the beholder”

Encode/Decode results may differ from encoder vendor to encoder vendor

There are many settings that vary with the camera, transport, view needs, lightingand content

MPEG & H.xxx specifications define the bit streams and data structures (Decoding)

Encoding is left to the developer

End-to-End quality is not controlled, only output given the same input


Image Quality

Common Metrics of Image Quality

Peak Signal to Noise Ratio (PSNR)

• Basic, but flawed

• PSNR values do not correlate well with perceived picture quality due to the complex,highly non-linear behavior of the human visual system.( source: https://sonnati.wordpress.com/2014/06/20/h265-part-i-technical-overview/)

Source: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/VELDHUIZEN/node18.html

All images have the same PSNR


Image Quality

Structural Similarity (SSIM) : Math

• Compares groups of original pixels with thepost-decoded pixels in the same image area.

• This objective analysis compares favorably withMOS

Mean Opinion Score (MOS): Observation

• Subjective measure

• Uses standard clips

• Uses standard viewing environments

• Human subjects score what they see comparedto raw clips.


H.265

Standard Managed by

ITU-T Video Coding Experts Group (VCEG)

ISO/IEC Moving Picture Experts Group (MPEG)

First joint HEVC standard released Jan 2013

Design Goals

50% reduction of bit rate compared to H.264 at the same perceptual image quality

Support transport of 4K video and beyond (more pixels)

Support high dynamic range color (more shades representable)

Support higher frame rates (120 and up)

Increase data loss resilience

Enabling Technologies

Parallel processing

• Needed for encoding (image analysis) and decoding (re-rendering)

New modulation techniques increasing the bits/Hz ratio

• QAM, quadrature modulation increases the bits/Hz in the transport channel


What is common H.264-H.265

Split frames into Coding Tree Units (macro blocks)

This is essentially the starting point for an I frame

Transform (DCT), scaling & quantization

Spatial motion prediction (Intrapicture) & differences compared

• Prediction of block data derived from within the I (reference) frame alone

The two elements are transformed (DCT), quantized and scaled

Entropy Encoding; Context adaptive binary arithmetic coding (CABAC)

• Lossless data compression (the final step)

The result is put in to the bit stream


What is common H.264-H.265

Temporal Encoding

Motion prediction of elements across a group of frames (GOP) from the I frame

• Results in differential frames

• B and P frames rely on data from the I frame

P frames (predictive) and can be used for prediction blocks in B frames

B frames process predictions based on data before and after the current frame.

• The resultant data for B and P frames is very small

• Most of the average compression comes from P and B frames

• P & B frames may be lost if the reference I frame is corrupted


What is different H.264-H.265

Larger macroblocks

More macroblock types

Spatial prediction; 9 modes > 35 modes

Temporal prediction; H.265 adds rectangular blocks

Transform sizes; H.264 max 16x16; H.265 adds 32x32, 64x64 plus non-square forms

Larger block sizes are more coding efficient


Is H.265 Better?

H.265 is not MORE compression; it is smarter

Increased flexibility

More fine control of prediction vectors

Better pixel interpolation

Finer deblocking


Is H.265 Better?


Is H.265 Better?

A range of scenes with similar PSNR and MOS compared to H.264 scenarios

Bit rates are better, but vary with scene complexity

Note a ±25% bit rate variance with the same encoder/decoder and hardware


Is H.265 Better?

A 2 hour movie on a Blu-ray disk

Transport over GigE Ethernet UDP with exclusive use of the network

Transport via 100Mb/s Wi-Fi link Exclusive use at full bandwidth


H.264 H.265

SD >6:1 3:1

720/60 >30:1 14:1

1080/60 >50:1 28:1

2160/60 >200:1 111:1


H.264 H.265

SD none None

720/60 2:1 None

1080/60 4:1 2:1

2160/60 >14:1 8:1


H.264 H.265

SD 3:1 2:1

720/60 >15:1 8:1

1080/60 >30:1 16:1

2160/60 >120:1 62:1


Conclusions

H.265 expands on legacy of tools

Macro blocks

Spatial and Temporal prediction

Motion vectors

DCT and Entropy

H.265 is not MORE compression; Smarter Processing

More tools, more analytical choices

More Computing Power Required

Parallel processing a must

Real-time (30 to 60 fps) does not change with more work to do encoding/decoding

Encoding can be longer even with more capable hardware

Compression goal met at 48-50% for same image quality

Latency?

Positive Side Effect:

At current transport bandwidths available 720p/60 and 1080p/60 should survivewith better image fidelity than with H.264


Concerns

Greater compression may mean more sensitivity to bit errors in transport

Encoders will vary from source to source

Image quality needs to be scored

H.265 Encoder/Decoder sets must come on line with metadata support

Two H.264 Encoder/Decoders have been tested for Metadata by ITS

Delta Digital 4480E and 9600 (decoder)

Haivision Makito X


Concerns

Decode side is specified

Encode side is up to the supplier

Encoders must be evaluated for image quality

Image quality for engineering test is different than moviegoers

• More details are important

• Critical frames are often most different frames

Latency

For the same channel width is latency greater or lesser?

Does latency vary more?


Bibliography of Resources

Overview of the High Efficiency Video Coding (HEVC) Standard Gary J. Sullivan, Fellow, IEEE, Jens-Rainer Ohm, Member, IEEE, Woo-Jin Han, Member, IEEE, and Thomas Wiegand, Fellow, IEEE;

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 22, NO. 12, DECEMBER 2012

Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial) Vivienne Sze ([email protected]); Madhukar Budagavi ([email protected])

ISCAS Tutorial 2014

Video Coding Basics Yao Wang, Polytechnic University, Brooklyn, NY11201, [email protected], 2003

“Intra Prediction in HEVC,”High Efficiency Video Coding (HEVC): Algorithms and Architectures Springer, 2014. J. Lainema, W.---J. Han,

Compression Synopsis H264-H265

Documents

Transcript of Compression Synopsis H264-H265