1 Prof. Dr. Paul Müller, University of Kaiserslautern

5.8. Hybrid Coding Steps for Images, Video, and Audio

Basic encoding steps:

2 Prof. Dr. Paul Müller, University of Kaiserslautern

5.8.2. Image Compression - JPEG (1)

Image preparation and coding: (baseline system)

select color space, color-subsampling, DCT, quantization, coding

3 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (2)

• color space

– greyscale

– RGB

– YCbCr

• The following example

shows an 8x8 pixel image

section

– 8-bit numbers represent

greyscales

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

4 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (3)

Assuming a given 16x16

block with Cb or Cr

The eye can sense local

differences only poorly

Averaged values of 2x2

sections

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

5 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (4)

• DCT

1) normalize values

2) enforce NxN blocks

3) apply 2-D DCT

8x8 block with 8-bit values

thus P=8; left shift of all

items corresponds to -128

this is the source matrix X

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

6 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (5)

• DCT 1) normalize values

2) enforce NxN blocks

3) apply 2-D DCT

• 8x8 block contains only

values between -128 and

127

• Application of the 2-D DCT

leads to the transformed

matrix D = A·X·AT

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

7 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (6)

Quantization select table Q

quantify DCT-coefficients

store results in L

• select a quantization table

Q with items qij according

to JPEG recommendation

• quantify according to

stated formula and obtain

table L with quantified

DCT-coefficients

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

8 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (7)

Coding

DC-coefficient

AC-coefficients

• DC-coefficient

• coding table: 16 categories with increasing number of entries

• code difference to preceding DC-coefficient: – here: 3

– 3 is in group 2 and is on position 4 there (thus bitword 11)

– assuming Huffman-Code for group 2 is 2-bit long (e.g. 01)

– resulting sequence for DC: 0111

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

9 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (8)

Coding

DC-coefficient

AC-coefficients

• AC-coefficients are traversed in zick-zack and coded absolutely

• The (Z,C) table contains Huffman codes for AC-sequences – Z: number of preceding zeros

– C: category of the AC coefficient

– For the example the results are: • 1 -> (0,1) -> 00 1

(2nd position in 1st category)

• -9 -> (0,4) -> 1011 0110

• 3 -> (0,2) -> 01 11

• 0…0 -> (0,0) -> 1010

1) select color space 2) color-subsampling 3) DCT 4) quantization 5) coding

10 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (9)

Result

8x8 section coded with 23 bit

0111 0011 0110 1100 1111 010

i.e. in average 23/64 instead of 8 bit per pixel

the reconstructed block has only small errors [values in %]

11 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (10)

Interactive examples:

JPEG Lines – Applet

JPEG Parrots - Applet

12 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (11)

Interactive examples:

Do it yourself with ImageTrans (*)

(*) This is a Java Webstart application; you'll probably need to install this

software from http://www.java.com/en/download/faq/java_webstart.xml

13 Prof. Dr. Paul Müller, University of Kaiserslautern

Image Compression - JPEG (12)

JPEG PNG

Size: 3,164 bytes Size: 810 bytes

Size: 3,564 bytes Size: 19,047 bytes

14 Prof. Dr. Paul Müller, University of Kaiserslautern

5.8.3. Overview

Wavelets in practice

JPEG 2000

• Preprocessing

• DWT

• Quantizer

• Tier 1 / Tier 2 coding

Comparison of JPEG 2000 with JPEG

15 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG 2000 block diagram

16 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon original

17 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon rgb colorspace

18 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon YCrCb colorspace

19 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon luma subsampled 2x

20 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon luma subsampled 4x

21 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon luma subsampled 8x

22 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon original

23 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon chroma subsampled 2x

24 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon chroma subsampled 4x

25 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon chroma subsampled 8x

26 Prof. Dr. Paul Müller, University of Kaiserslautern

Baboon original

27 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG 2000 block diagram

28 Prof. Dr. Paul Müller, University of Kaiserslautern

Transformation in JPEG 2000

DWT provides frequency band decomposition of the

image

Multi-resolution image representation

Full-frame transformation -> no blocking artifacts

Uses daubechies (9,7) floating point or (5,3) integer filter

Use of integer filters -> both lossless and lossy

compression

Two implementation approaches: filter based / lifting

scheme

29 Prof. Dr. Paul Müller, University of Kaiserslautern

1-D two-band DWT

30 Prof. Dr. Paul Müller, University of Kaiserslautern

1-D two-band DWT

31 Prof. Dr. Paul Müller, University of Kaiserslautern

2-D wavelet decomposition

32 Prof. Dr. Paul Müller, University of Kaiserslautern

Lena original

33 Prof. Dr. Paul Müller, University of Kaiserslautern

Lena horizontal high-pass

34 Prof. Dr. Paul Müller, University of Kaiserslautern

Lena horizontal low-pass

35 Prof. Dr. Paul Müller, University of Kaiserslautern

Lena low + high-pass subsampled

36 Prof. Dr. Paul Müller, University of Kaiserslautern

1-level 2-D wavelet decomposition

37 Prof. Dr. Paul Müller, University of Kaiserslautern

2-level 2-D wavelet decomposition

38 Prof. Dr. Paul Müller, University of Kaiserslautern

3-level 2-D wavelet decomposition

39 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG 2000 block diagram

40 Prof. Dr. Paul Müller, University of Kaiserslautern

Quantization and Dequantization

41 Prof. Dr. Paul Müller, University of Kaiserslautern

Quantization in JPEG 2000

42 Prof. Dr. Paul Müller, University of Kaiserslautern

Quantization by bit-plane coding

43 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane example

Wavelet coefficient = 83; Quantizer step size = 3

Quantizer index = 83/3 = 27 = 00011011

Dequantized value based on fully decoded index

(27 + 0.5)*3 = 82,

Dequantized value after decoding 6 BP‘s:

Decoded index = 000110 = 6; Step size = 12

Dequantized value = (6 + 0,5)*12 = 78

Dequantized value after decoding 4 BP‘s:

Decoded index = 0001 = 1; Step size = 48

Dequantized value = (1 + 0.5)*48 = 72

44 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG 2000 block diagram

45 Prof. Dr. Paul Müller, University of Kaiserslautern

Entropy coding in JPEG 2000

46 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG 2000 Entropy coder blocks

47 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 1

48 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 1 enlarged by 8x

49 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 2

50 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 2 enlarged by 8x

51 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 3

52 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 4

53 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 5

54 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 6

55 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 7

56 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 8

57 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 9

58 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 10

59 Prof. Dr. Paul Müller, University of Kaiserslautern

Bit-plane 11

60 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG 2000 block diagram

61 Prof. Dr. Paul Müller, University of Kaiserslautern

Tier 2 role

Tier 1 generates a collection of bitstreams

One independent bitstream for each code-block

Tier 2 multiplexes the bitstreams for inclusion in the

codestream and signals the ordering of the resulting

coded bitplane passes in an efficient manner

Tier 2 enables SNR, resolution, spatial, ROI and arbitrary

progression and scalability

62 Prof. Dr. Paul Müller, University of Kaiserslautern

Example of coded data

63 Prof. Dr. Paul Müller, University of Kaiserslautern

Example of coded data

64 Prof. Dr. Paul Müller, University of Kaiserslautern

Lowest resolution, highest quality

65 Prof. Dr. Paul Müller, University of Kaiserslautern

Medium resolution, highest quality

66 Prof. Dr. Paul Müller, University of Kaiserslautern

Highest resolution, highest quality

67 Prof. Dr. Paul Müller, University of Kaiserslautern

Highest resolution, target SNR quality

68 Prof. Dr. Paul Müller, University of Kaiserslautern

Highest resolution, target visual quality

69 Prof. Dr. Paul Müller, University of Kaiserslautern

SNR progressive example

70 Prof. Dr. Paul Müller, University of Kaiserslautern

SNR progressive example

71 Prof. Dr. Paul Müller, University of Kaiserslautern

SNR progressive example

72 Prof. Dr. Paul Müller, University of Kaiserslautern

Resolution progressive example

73 Prof. Dr. Paul Müller, University of Kaiserslautern

Resolution progressive example

74 Prof. Dr. Paul Müller, University of Kaiserslautern

Resolution progressive example

75 Prof. Dr. Paul Müller, University of Kaiserslautern

Resolution progressive example

76 Prof. Dr. Paul Müller, University of Kaiserslautern

JPEG vs. JPEG 2000

original

JPEG 2000 (43:1) JPEG (43:1)