Overview of Fine Granularity Scalability in MPEG-4 Video

Standard

Weiping Li

Presented by : Brian Eriksson

Introduction - Problem

• Tradition System– Video encoded close to channel capacity– Decoded using all of the bits

• Internet Video System– Channel capacity changes– Partially decodable within a bit range

Introduction – Video Coding Performance

Low HighChannel Bitrate

Received Quality

Bad

Good

= Distortion-Rate Curve

= Dialup Rate

= Broadband Rate

= Layered Coding

= Desired Coding

Low HighChannel Bitrate

Received Quality

Bad

Good

Previous Layered Coding Methods

= Layered Coding

MPEG-2 Overview

• I-Frame – Intracoded Picture– Contains all frame information

• P-Frame – Predictive-Coded Picture– Uses previous frame and motion

information to reconstruct frame

• B-Frame – Bidirectional-Coded Picture– Uses previous frame, next frame, and

motion information to reconstruct the frame

Temporal Scalability

•Video is encoded into two layers with equal resolution but using different prediction.

•Base layer has lower frame rate

Base Enh. Enh. Base Enh. Enh. Base Enh. Enh.

Base X X Base X X Base X X

With Enhancement Layer : Frame Rate = 30 fps

Without Enhancement Layer : Frame Rate = 10 fps

MPEG-2 SNR Scalability

Enhancement Layer Stream

Base Layer Stream

+ IDCT

Video Output

•Motion prediction includes enhancement information

•Efficiency dependent on two factors

•Encoder uses enhancement layer (drift can occur)

•Decoder receives enhancement layer

Motion Compensation

+

Spatial Scalability

IDCTEnhancement Bitstream

Base Bitstream (Motion Compensated)

Upsampler

+Enhancement Layer Video

Base Layer Video

•Same frame rate, different resolutions

•Base frame = 128x128, enhancement = 256x256

•Enhancement Layer not in prediction loop

Low HighChannel Bitrate

Received Quality

Bad

Good

MPEG-4 Fine Granularity Scalability Technique

= Desired Coding

= Distortion-Rate Curve

Bitplane Coding Technique

Given vector X:

X = [-12, -53, 62, -7, 31,…,180,-43,…,5]

•Convert to signed magnitude

•Separate into sign bit and absolute values

Bitplane Coding Technique

Sign(X) = [0, 0, 1, 0, 1,…,1,0,…,1]

Abs(X) = [12, 53, 62, 7,31,…,180,43,…,5]

•To find the number of bitplanes needed, find the maximum value of abs(X)

•Max(abs(X)) = 180 = 10110100b

•8 bitplanes

•Sign bit plane

•1 bitplane

Bitplane Coding Technique

c

Sign(X) = [0, 0, 1, 0, 1,…,1,0,…,1]

X = [-12, -53, 62, -7, 31,…,180,-43,…,5]

0,1,0,1,1,…,0,1,…,10,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1

Run Level Encoding

• Symbol : (RUN, EOP)– RUN = Number of Consecutive Zeros before a 1– EOP = 0 if there are more ones– EOP = 1 if the rest of the line are zeros.

• Example:– {1,0,1,0,0,0,1,0,0,0,…}

• = (0,0),(1,0),(3,1)– {0,0,0,1,0,0,0,…}

• = (3,1)

MPEG-4 Bitplane FGS Technique

• Base layer reaches lower bound of bit-range• Divide image into 8x8 DCT blocks• Divide blocks into Y,U,V color components• Use bitplane run-level coding to

encode/decode

Fine Granularity Scalability Encoder

+-

DCT Bitplane Encoding

Original Signal

Base Layer Signal (Motion Compensated)

Enhancement Layer Signal

Fine Granularity Scalability Decoder

Bitplane Decoding

IDCT +

Base Layer Video (Motion Compensated)

Enhancement Layer Video

Truncated Bitplane

c

Number of Truncated Layers = 0

Received X = [-12, -53, 62, -7,31,…,180,-43,…,-5]

Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]

0,1,0,1,1,…,0,1,…,10,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1

c

Number of Truncated Layers = 1

Received X = [-12, -52, 62, -6,30,…,180,-42,…,-4]

Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]

0,0,0,0,0,…,0,0,…,00,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1

c

Number of Truncated Layers = 2

Received X = [-12, -52, 60, -4,28,…,180,-40,…,-4]

Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]

0,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1

c

Number of Truncated Layers = 3

Received X = [-8, -48, 56, -0,24,…,176,-40,…,-0]

Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]

0,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,01,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1

Advanced Bitplane Techniques

Frequency Weighting in Bitplane

Normal Bitplane

DCT Index

Frequency Weighted Bitplane

DCT Index

Selectively Enhanced Bitplane

•Detect visually significant area

•Shift upward in bitplane

•More likely to be included in truncated bitstream

Other Methods

•Error Resilience in Bitplane•Random burst errors in the bitstream

•Resynchronization markers are used to resynch.

•Temporal Scalability in Bitplane•Uses FGS to encode/decode the entire temporal enhancement frame

Simulation - DCT

Simulation - DWT

Selectively Enhanced / Weighted

Bitplane Setup

Simulation – Enhanced Weighted Bitplane

1 Layer Removed2 Layers Removed3 Layers Removed4 Layers Removed5 Layers Removed6 Layers Removed

Simulation – Li

Conclusions

• FGS Bitplane method allows for quality parallel to distortion-rate curve

• A wavelet-based approach may yield better results

• Fairly simple implementation

Questions?