JPEG 2000 Image Coding System

Coding of Still Pictures

JPEG 2000 Part I Final Committee Draft Version 1.0

April, 2000

Why another JPEG?• JPEG2000 can provide

• higher compression efficiency for low bit-rate application

• larger images (up to 64k by 64k)• superior performance for

• natural images (e.g. photo)• computer-generated images (e.g. graphics)• compound documents (e.g. bi-level text)

• five progression orders• progressive by quality (SNR) or by resolution

lossless and lossy compression in a single stream• ROI (Region of Interest) coding

JPEG v.s. JPEG2000(Main Differences)• DCT v.s. DWT (Discrete Wavelet

Transform)• Way of coding coefficients

• JPEG• Zigzag scan• Huffman coding, run-length coding

• JPEG2000• Coefficient bit modeling

• Scan algorithm• Probability estimation

• Arithmetic entropy coding

Main Ideas of JPEG2000• DWT• Way of coding coefficients

• Coefficient bit modeling• Scan algorithm:

bitplane and sub-bitplane (coding pass)• Probability estimation

• Arithmetic entropy coding• Tag tree to save tree information in header

• For progressive transmission• ROI coding• Five progression orders

• Image -> Tiles• Apply DWT for each tile

Tile

Basic Coding Scheme of JPEG2000

DWT

ScanAlgorithm

ProbabilityEstimation

EntropyCoding

Tile of a Image

Bit Stream

DC levelshift

(Component Transform)

Coefficient bit modeling

Quantization

DC Level Shift

e.g. • The ith component is unsigned.• The bit depth of the ith component is 8.

( Each sample value is in (0, 255))

I(x, y) = I(x, y) – 2

( Each sample value is in (-128, 127))

(8 – 1)

Component Transform• To decorrelate redundancy between comp

onents to improve compression efficiency

• Component transformation is applied to the three first components.• ICT (Irreversible Component Transform)

• For lossy compression• RCT (Reversible Component Transform)

• For lossy and lossless compression

ICT (Irreversible Component Transform)• The Forward ICT:

Given I0, I1, I2 in the first three components:Y0 = 0.299 I0 + 0.587 I1 + 0.144 I2Y1 = -0.16875 I0 + 0.33126 I1 + 0.5 I2Y2 = 0.5 I0 + 0.41869 I1 + 0.08131 I2

• If the first three components are R, G, and B, then the Forward ICT can be seen as a approximation of a YCbCr transformation.

• The Inverse ICT:I0 = Y0 + 1.402 Y2I1 = Y0 – 0.34413 Y1 – 0.71414 Y2I2 = Y0 + 1.772 Y1

RCT (Reversible Component Transform)• The Forward RCT:

Given I0, I1, I2 in the first three components:

Y0 = floor( (I0 + 2 I1 + I2) / 4 )Y1 = I2 – I1Y2 = I0 – I1

• The Inverse RCTI1 = Y0 – floor( (Y2 + Y1) / 4 )I0 = Y2 + I1I2 = Y1 + I1

High Pass: Y(2n+1) = X(2n+1) - floor( (X(2n) + X(2n+2)) / 2 )

Example of DWT:Lifting-based (5, 3) filter

171172 164 165 142 99 63 55 69171164 55 630 1 2 3 4 5 6 7 8

3

174

12 -3 -113 -11

168 144 60 64

Low Pass: Y(2n) = X(2n) + floor( (X(2n-1) + X(2n+1) + 2) / 4 )

Extension

Run…

-12

-12

14

14

1-12

-12

-18

14

34

14

-18

• Image “Lena”(256x256)• Resolution levels (each

consists of 1 or 3 sub-bands)• LL0• HL1, LH1, HH1• HL2, LH2, HH2

• Precincts ( positions)(packet partitions)e.g. 8 precincts(16x32)

• Code-blockse.g. 2 code-blocks(16x16)

• HL3, LH3, HH3

Example of DWT (con’t)

HL3

HH3LH3

Example of DWT (con’t)(100,100,100,100,100,100, 0, 0, 100,100,100,100,100,100)

(100, 100, 113, 0, 88, 100, 100) (0, 0, 50, -50, 0, 0, 0)

100 100100 100100 100100 100100 100

00000

113113113113113

8888888888

100 100100 100100 100100 100100 100

0 00 00 00 00 0

0 0 00 0 00 0 00 0 00 0 0

5050505050

-50-50-50-50-50

100 100 100 100 100 100100 100 100 100 100 100100 100 100 100 100 100100 100 100 100 100 100100 100 100 100 100 100

100 100 100 100 100 100100 100 100 100 100 100100 100 100 100 100 100100 100 100 100 100 100100 100 100 100 100 100

0 00 00 00 00 0

1D DWT(for row)

HighLow

1D DWT

Example of DWT (con’t)

DWT

Quantization• Each of the transform coefficients f(u, v) is quantize

d to the value q(u, v).

q = sign(f)

• Quantization step-size = 2 (1+ )

• Pair (Exponent , mantissa ) for sub-band b is obtained either explicitly or implicitly.

• Nominal dynamic range is the bit depth for sub-band b to represent the original image.

|f|

b

bR - b b

211

b

b b

Rb

Quantization (con’t)• Explicit quantization

• ( , ) are signaled in the bit stream for every sub-band.

• Implicit quantization• ( , ) is signaled in the bit stream for the

LL sub-band.• ( , ) are derived implicitly from ( , ).

b b

0 0

b b 0 0

Quantization (con’t)

R R+1

R+1 R+2

R+1

R+1

R+2

R is the bit depth of the original image

1 2

2 4

2

2

4

Two level analysis gain

Rbfor each sub-band bR

0 0 0 0 00 1 0 0 00 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

LSB

Bitplane0 10 0 0 00 7 0 0 00 3 0 0 00 –2 0 0 00 0 0 0 00 0 0 0 0

10 -> + 1 0 1 07 -> + 0 1 1 13 -> + 0 0 1 1

-2 -> - 0 0 1 0

0 1 0 0 00 1 0 0 00 1 0 0 00 1 0 0 00 0 0 0 00 0 0 0 0

0 0 0 0 00 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

MSB LSB

0 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

MSB

Code-block

Code-block Scan Pattern

0 0 0 0 00 1 0 0 00 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

LSB

0 1 0 0 00 1 0 0 00 1 0 0 00 1 0 0 00 0 0 0 00 0 0 0 0

0 0 0 0 00 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

0 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

MSB

Bitplane0

Bitplane1

Bitplane2

Bitplane3

Sub-bitplane (Coding Pass)• Each bitplane is coded in 3 sub-bitplanes (c

oding passes)• Significance propagation• (Magnitude) Refinement• Clean-up

• Each coefficient bit in a bitplane is coded only one of the 3 sub-bitplane• Depend on the “significance state” of that coe

fficient and its 8 neighboring coefficients

Significance State

• State of a coefficient at a particular bitplane

• If a coefficient has the first “1” bit at or before the given bitplane is considered “significant”,

significance state = 1

otherwise it is considered “insignificant”.

significance state = 0

Example of Sub-bitplane Coding Order010 1+ 0 1 07 0 1+ 1 13 0 0 1+ 1-2 0 0 1- 00…

Bitplane3Bitplane2Bitplane1Bitplane0(Significance state = 1) The coefficient is significant

S R CC S R C S R C

(Neighbor) Context:Function of Neighbors States

0 0 0 0 00 1 0 0 00 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

LSB

0 1 0 0 00 1 0 0 00 1 0 0 00 1 0 0 00 0 0 0 00 0 0 0 0 0 1 0 0 0

0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

MSBBitplane3

0 0 0 0 00 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

Bitplane2

Significance state (after coding bitplane3)

0 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

V0D0 D1H0 H1D2V1D3

8 neighbors states = 0 context = 0

Bits in a Sub-bitplane

significance state = 1

context ≠ 0

Significance propagationRefinement Clean-up

Significance propagationRefinement Clean-up

true

true

false

false

more important

(significant) (insignificant)

(most likely to be significant)

Bits in a Sub-bitplane (con’t)• Bits coded in each sub-bitplane (coding pass)

• Significance propagation• Bits from “most likely to become significant” coefficie

nts (significance state = 0) and (context ≠ 0)

• Refinement• Bits from already significant coefficients (significance state = 1)

• Clean-up• Bits from all remaining coefficients (significance state = 0) and (context = 0)

Coding Operations• 4 coding operations

• Significance coding (9 context bins)• Sign coding (5 context bins)• Magnitude refinement coding (3 context bins)• Clean-up coding (run-length coding and 9 context bins)

• Coding operations in each sub-bitplane• Significance propagation

• Significance coding and sign coding• Refinement

• Magnitude refinement coding• Clean-up

• Clean-up coding and sign coding

9 Context Bins for Significance and Clean-up Codings

V0D0 D1H0 H1D2V1D3

LL and LH HL HH Context (label)ΣH ΣV ΣD ΣH ΣV ΣD Σ(H+V

)ΣD

2 2 ≧3 8

1 ≧1 ≧1 1 ≧1 2 7

1 0 ≧1 0 1 ≧1 0 2 6

1 0 0 0 1 0 ≧2 1 5

0 2 2 0 1 1 4

0 1 1 0 0 1 3

0 0 ≧2 0 0 ≧2 ≧2 0 2

0 0 1 0 0 1 1 0 1

0 0 0 0 0 0 0 0 0

10 00 00 0 0

e.g.

ΣH = 0ΣV = 1ΣD = 0For HL 5

5 Context Bins for Sign Coding

H V Context (label)

XOR bit

+ + 13

0 expected value = 0

(expected sign = +)

+ 0 12+ - 110 + 100 0 9

0 - 101

expected value = 1 (expected sign = -)

- + 11- 0 12- - 13

V0H0 H1

V1

1+0 0

0

e.g.

H = 0V = +

10

1-1- 1+

0

H = 0V = -

10

HV

Code (sign bit) XOR (XOR bit) bit

3 Context Bins for Magnitude Refinement Coding

First refinement for this coefficient

Context label 16

Σ(H+V+D) = 0

Context label 15

Context label 14

false

false

true

trueV0D0 D1

H0 H1D2V1D3

If the four contiguous coefficient bits

currently satisfy

Run-length Coding for Cleanup Coding

Code 0 (using context label 17)

Code the four bits (using context label 18)

Four bits are all 0true

falseCode 1 (using context label 17)

significance state = 0context = 0

UNIFORMcontext

Run-length context

0 1 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

Huffman Coding

• 設法讓 “較常出現” 的 symbol ，使用較短的 code 。

Symbol a b c d

出現機率 3/4 1/8 1/16 1/16

Fixed-Length Code 00 01 10 11

Huffman Code 0 10 110 111

資訊量 (= log(1/p) ) 0.415 3 4 4

平均資訊量 (Entropy)

(= Σp*log(1/p) )

0.415(3/4) + 3(1/8) + 4(1/16) + 4(1/16) = 1.18625

平均長度2(3/4) + 2(1/8) + 2(1/16) + 2(1/16)= 2

1(3/4) + 2(1/8) + 3(1/16) + 3(1/16)= 1.375

AC:Arithmetic (Entropy) Coding

3/4

1/81/161/16d

cb

a

0

1

0.25

0.1250.0625

0.25

0.16796875

0.1796875

0.16815856

0.168701172

a

0.125

0.25

b b

d

a

Encodebabda0.16815856

0.15625

Example of AC for Two Symbols

0.9

0.1

0

10

A = 1

0

A = 0.9

Encode 0 A = A – 0.1 C = C + 0.1

C = 0.1C = 0

0.09

A: the intervalC: the code

0

A = 0.81

C = 0.19

0.081

0.81 0.729

Encode 0 A = A – 0.09 C = C + 0.09

0

A = 0.729

C = 0.271

Encode 0 A = A – 0.081 C = C + 0.081

Example of AC for Two Symbols (con’t)

0.9

0.1

0

10

A = 1

0

A = 0.1

Encode 1 A = 0.1 C = C

C = 0C = 0

0.01

A: the intervalC: the code

0

A = 0.01

C = 0

0.001

0.09 0.009

Encode 1 A = 0.01 C = C

…

AC for Two Symbols

• Two symbols (0 and 1)• LPS (Less Probable Symbol)

• Probability Qe• MPS (More Probable Symbol)

• Probability (1-Qe)• E.g.

Pr(0) = 0.9Pr(1) = 0.1

MPS = 0LPS = 1Qe = 0.1

The MQ-Coder

1-Qe

Qe

MPS

LPS0

A = 1

0

A – Qe

Encode MPS A = A – Qe C = C + Qe

C = Qe

0.75 ≦ A ＜ 1.5

C = 0

1-Qe

Qe

MPS

LPS0

A = 1

0

Qe

Encode LPS A = Qe

C = 0C = 0

A ～ 1

The MQ-Coder (con’t)• Keep the interval A in the range 0.75≦

A ＜ 1.5 (by using renormalization, if needed), and save code in C• Code MPS

• A = A – Qe (i.e. sub-interval for the LPS)• C = C + Qe

• Code LPS• A = Qe (i.e. sub-interval for the MPS)

Renormalization of The MQ-Coder• Out one bit (most significant bit) of

C• Double A and C

A<<=1, C<<=1

• Obtain a new probability estimate (state) for the context

Finite State Machine of The MQ-Coder

Pr(0) = 0.7Pr(1) = 0.7

Pr(0) = 0.86

Pr(0) = 0.94

Pr(0) = 0.5

Pr(0) = 0.67 Pr(0) = 0.51

Pr(0) = 0.50

0

0

1

1 1

01

Slowly change Pr(0)

Fast change Pr(0)

MPS = 0

… …

…

…

Initialization and Termination of The MQ-Coder• Initialization

• Start at the initial state Pr(LPS) = Qe

• Termination• Terminate coding operations• Generate the required terminate marker

Usage of The MQ-Coder• Each context bin invokes a MQ-Coder.• The initial state of each MQ-Coder:

• UNIFORM context (label 18)• the state with Pr(0) = 0.5, and no state-transaction

• Run-length context (label 17)• the state with Pr(0) = 0.936988

• All-zero-neighbors context (label 0)• the state with Pr(0) = 0.969947

• Other contexts (labels 1-16)• the state with Pr(0) = 0.5

Layer

• A layer consists of some number of consecutive sub-bitplanes (coding passes)• e.g.

layer0 = {the first 2 sub-bitplanes}layer1 = {the following 3 sub-bit-planes}…

layer0 layer1

higher sub-bitplanes lower sub-bitplanes

…layer2

Packet

Packet Header

2 sub-bitplanesfrom code-block0

1 sub-bitplanefrom code-block1

5 sub-bitplanesfrom code-block2 …

e.g.

Tag Tree

• A way of representing a 2D array in a hierarchical way

• To save information in a packet header

• Only the information needed for the current code-block is stored in the packet header• Ability to progressive transmit

Example of Tag Tree

6 43 3 2 23 4 3 4 2 2 3 32 2 4 2

6

4

3

3

2

2

3

4

3 4 2 2

3

3

2

2

4 2

0011 (A>0, A=B=2) 011 (C=n1=3)

0001 (n2>C=3, n2=C+3)

1 (D=B) 01 (n3>D, n3=D+1)

A

B

C

n1 n2

D

n3

A, B, C,n1

n2

D, n3

Minimum of the sub-tree

22

2

2

3

2

3 2 2

Information in Packet Header• Bit for zero (“0”) or non-zero (“1”) length packe

t• For each precinct of each sub-band

• For all code-blocks in the precinct in raster order• Inclusion information ( the layer ID where the

code-block first appears in the precinct of the sub-band)• If included at the 1st time tag tree• else one bit (the code-block is included(1)/not included(0))

• If the code-block is included• If included at the 1st time

• Zero bit-planes information tag tree• # of sub-bitplanes• Length for the code-block• …

ROI (Region of Interest)• ROI is a part of an image coded earlier in the

codestream than the rest part (background)• A tile can be either

• Outside the ROI• Inside the ROI• Partly inside the ROI

• ROI coding• Arbitrary region shapes• Any number of regions• Maxshift method

ROI Coding• 4 steps:

• Generate ROI mask M(x, y)

• Select the scaling value s• Maxshift method

s ≧ (bit depth of max(BG))• Scale down all BG by s• Write s into codestream

1 (x, y) is needed for ROI0 otherwise ( (x, y) BG (background))

M(x,y) =

ROI Mask Generation

e.g.

171172 164 165 142 99 63 55 690 1 2 3 4 5 6 7 8

3

174

12 -3 -11

168 144 60 64

-12

-12

14

14

… …

Sample in ROI

High

Low

ROI Maxshift Method

ROI coefficients

Coefficient values

ROI coefficients

max(BG)

Maxshift

e.g. max(BG) = 10 s = 4

Shift-left 4 bits for each coefficient in ROI Mask

Advantages of ROI Maxshift Method• Support for arbitrary region shapes and any

number of regions with minimal complexity• No need for

• sending shape information• shape codec• ROI mask at decoder side

• Decoder as simple as non-ROI capable decoder

• Support progression by accuracy and by quality

Error Resilience

• Error resilience tools• Packet level• Entropy coding level

Error Resilience Tools at Packet Level

• Packets• Re-synch marker SOP (Start of Packet) at

packet boundaries• Locate errors in a packet

• Short packet format (packed packet)• Move the packet headers to either

• Tile-part header (PPT)• Main header (PPM) of the codestream

Error Resilience Toolsat Entropy Coding Level• Code-blocks

• Re-synch marker at code-block boundaries• Locate errors in a code-block

• Bit-planes• Put segmentation symbol at the end of each bit-plane f

or error detection• Sub-bitplanes (coding passes)

• The AC terminate on• each sub-bitplane• some sub-bitplanes ( selective AC bypass)• last sub-bitplane

• Reset context probabilities on each sub-bitplane

Selective AC bypass

SP AC , terminate AC ACMR AC , terminate AC ACC AC , terminate AC AC, terminate

SP (significance propagation)

AC , terminate AC AC

MR (magnitude refinement)

AC , terminate AC AC

C (cleanup) AC , terminate AC AC

C (cleanup) AC , terminate AC AC1

2

4

…

SP AC , terminate AC RawMR AC , terminate AC Raw ,

terminateC AC , terminate AC AC, terminate

5

…SP AC , terminate AC RawMR AC , terminate AC Raw ,

terminateC AC , terminate AC, terminate AC, terminate

Final

AC terminate on each sub-bitplane

AC terminate on last sub-bitplane

… … ……

… … … …

Progression Order

• The interleaving of the packets can progress along 4 axes:• Component

• e.g. Y, Cb, Cr• Resolution

• e.g. LL0, HL1,LH1,HH1, HL2,LH2,HH2, …• Position• Layer

Layer-resolution-component-position Progressive (1)

layer0 layer1

higher sub-bitplanes lower sub-bitplanes

LL0HL1 LH1 HH1HL2 LH2 HH2 …

Y Cb Cr

position0posotion1posotion2

…

…

Resolution-layer-component-position Progressive (2)

layer0 layer1

LL0HL1 LH1 HH1HL2 LH2 HH2 …

Y Cb Cr

position0position1position2

…

…

Resolution-position-component-layer Progressive (3)

layer0 layer1

LL0HL1 LH1 HH1HL2 LH2 HH2 …

Y Cb Cr

position0position1position2

…

…

Position-component-resolution-layer Progressive (4)

layer0 layer1

LL0HL1 LH1 HH1HL2 LH2 HH2 …

Y Cb Cr

position0position1position2

…

…

Component-position-resolution-layer Progressive (5)

layer0 layer1

LL0HL1 LH1 HH1HL2 LH2 HH2 …

Y Cb Cr

position0position1position2

…

…

What we have learned?• Operations of DWT• Intelligent scan order

• More important coefficient bits are coded earlier.

• MQ coder (Arithmetic coder) for two symbols

• Tag tree• ROI coding

Other Topics

• Rate control (Annex 10)• Visual frequency weighting (Annex J.

8)• TCQ (Trellis coded quantization)• Postprocessing