Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline...

39
Jpeg Decoder Baseline Sequential DCT-based

Transcript of Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline...

Page 1: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Jpeg Decoder

Baseline Sequential DCT-based

Page 2: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Baseline Sequential DCT-based

Page 3: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Baseline Sequential DCT-based

Encoding Process

• Color Space Conversion

• Subsampling

• Partition

• Encoding Flow Control• Discrete Cosine Transform (DCT)

• Quantization

• Entropy Encoding (Huffman) Decoding

Page 4: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

YCbCrRGB

Page 5: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 6: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Baseline Sequential DCT-based

Encoding Process

• Color Space Conversion

• Subsampling

• Partition

• Encoding Flow Control• Discrete Cosine Transform (DCT)

• Quantization

• Entropy Encoding (Huffman) Decoding

Page 7: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 8: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Baseline Sequential DCT-based

Encoding Process

• Color Space Conversion

• Subsampling

• Partition

• Encoding Flow Control• Discrete Cosine Transform (DCT)

• Quantization

• Entropy Encoding (Huffman) Decoding

Page 9: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 10: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Baseline Sequential DCT-based

Encoding Process

• Color Space Conversion

• Subsampling

• Partition

• Encoding Flow Control• Discrete Cosine Transform (DCT)

• Quantization

• Entropy Encoding (Huffman) Decoding

Page 11: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 12: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 13: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 14: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Huffman Encoding

• DC Diff

• AC Run-Length pair

Page 15: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 16: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

SSSS DIFF values

0 0

1 –1,1

2 –3,–2,2,3

3 –7..–4,4..7

4 –15..–8,8..15

5 –31..–16,16..31

6 –63..–32,32..63

7 –127..–64,64..127

8 –255..–128,128..255

9 –511..–256,256..511

10 –1 023..–512,512..1 023

11 –2 047..–1 024,1 024..2 047

DC Diff

1. Decoding codeword with Huffman decoding for getting SSSS

2. Using following SSSS bits as index for looking up DIFF value

3. Plus the DC value of last block

(ITU-T81 Annex F)

Page 17: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Huffman decoding

Page 18: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 19: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

RRRRSSSS

Page 20: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Huffman decoding

Page 21: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Header Processing

Important header

Page 22: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

TISO0840-93/d020

Compressed image data

SOI Frame EOI

Tables/misc. [ [Frame header DNL

segment Scan2

[[[ [ Scan 1

[[Scan last

Tables/misc.[ [ Scan header [ECS0

Scan

Frame

ECS last-1 ECS lastRST last-1 ]

Entropy-coded segment 0 Entropy-coded segment last

<MCU >, <MCU >, · · · <MCU >1 2 Ri <MCU >, <MCU >, · · · <MCU >n n + 1 last

Figure B.2 – Syntax for sequential DCT-based, progressive DCT-based,and lossless modes of operation

RST0

Page 23: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Code Assignment Symbol Description

Start Of Frame markers, non-differential, Huffman coding

X’FFC0’ SOF0 Baseline DCT

Huffman table specification

X’FFC4’ DHT Define Huffman table(s)

Restart interval termination

X’FFD0’ through X’FFD7’ RSTm* Restart with modulo 8 count “m”

Other markers

X’FFD8’

X’FFD9’

X’FFDA’

X’FFDB’

X’FFDD’

X’FFE0’ through X’FFEF’

SOI*

EOI*

SOS

DQT

DRI

APPn

Start of image

End of image

Start of scan

Define quantization table(s)

Define restart interval

Reserved for application segments

Page 24: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 25: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

MarkerNumber of

componentsInformation

Header Structure (ITU-T81 Annex B)

Page 26: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

SOF

C1 1 1 1 C C

SOF PLf Y X Nf

H V Tq 2 2 V 2H Tq 2 Nf Nf Nf NfH V Tq

n

TISO0850-93/d021

Frame header

Frame component-specification parameters

Component-specificationparameters

Figure B.3 – Frame header syntax

Y: Number of linesX: Number of samples per line

Ci: Component identifierHi: Horizontal sampling factorVi: Vertical sampling factor

Page 27: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Parameter Size (bits) Sequential DCT Progressive DCT Lossless

Baseline Extended

Lf 16 8 + 3* Nf

P 8 8 8, 12 8, 12 2-16

Y 16 0-65535

X 16 1-65535

Nf 8 1-255 1-255 1-4 1-255

Ci 8 0-255

Hi 4 1-4

Vi 4 1-4

Tqi 8 0-3 0-3 0-3 0-1

Page 28: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

In 4:2:0 mode, there are 6 blocks

(2*2+1*1+1*1) in an MCU.

MCU – Minimum Coded Unit, which is comprised by blocks from each component

Page 29: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022
Page 30: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

SOS

2 2 2

NsTd NsTa

NsCs

SOS Ls Ns Ss Se Ah Al

Cs1 Td1 Ta1 Cs Td Ta

TISO0860-93/d022

Scan header

Component-specificationparameters

Scan component-specification parameters

Figure B.4 – Scan header syntaxNs: Number of image components in scanTdj: DC entropy coding table destination selectorTaj: AC entropy coding table destination selector

Page 31: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Values

Parameter Size (bits) Sequential DCT Progressive DCT Lossless

Baseline Extended

Ls 16 6 + 2 * Ns

Ns 8 1-4

Csj 8 0-255

Tdj 4 0-1 0-3 0-3 0-3

Taj 4 0-1 0-3 0-3 0

Ss 8 0 0 0-63 1-7

Se 8 63 63 Ss-63 0

Ah 4 0 0 0-13 0

Al 4 0 0 0-13 0-15

Page 32: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

DQT

Q 1Q Q0 63Lq Pq Tq

TISO0880-93/d024

DQT

Define quantization table segment

Multiple (t = 1, ..., n)

Figure B.6 – Quantization table syntaxTq: Quantization table destination identifier - o for DC 1 for AC

Qk: Quantization table element

Page 33: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Values

Parameter Size (bits) Sequential DCT Progressive DCT Lossless

Baseline Extended

Lq 16 Undefined

Pq 4 0 0, 1 0, 1 Undefined

Tq 4 0-3 Undefined

Qk 8, 16 1-255, 1-65535 Undefined

Page 34: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

DHT

TISO0890-93/d025

DHT Lh Tc Th L 1 L 2 L 16

Define Huffman table segment

Symbol-lengthassignment

Multiple (t = 1, ..., n)Symbol-length assignment parameters

Figure B.7 – Huffman table syntax

V1,1 V1,2 V1,L1V2,1 V2,2 V2,L2

V16,1 V16,2 V16,L16

Tc: Table class, DC or AC?Th: Huffman table destination identifier

Li: Number of Huffman codes of length iVi,j: Value associated with each Huffman code(symbol)

(length)

(codeword)

Page 35: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

(ITU-T81 Annex C)

Page 36: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

In common case, there are four huffman tables:

– Luminance’s DC huffman table

– Luminance’s AC huffman table

– Chrominance’s DC huffman table

– Chrominance’s AC huffman table

Page 37: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Values

Parameter Size (bits) Sequential DCT Progressive DCT Lossless

Baseline Extended

Lh 16

Tc 4 0, 1 0

Th 4 0, 1 0-3

Li 8 0-255

Vi, j 8 0-255

Page 38: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

DRI

• Ri – Specifies the number of MCU in the restart

interval.

• After the number of MCU reaches Ri, a RST marker

is inserted and the MCU counter is reset.

• You can resynchronize to the next RST marker when

bit error or packet loss occurred during image

transmission and the number of lost MCU can be

known from the difference of RST id.

Page 39: Jpeg Decoder - 國立臺灣大學itct/hw/jpeg_2016/JPEG Baseline... · Jpeg Decoder Baseline Sequential DCT-based. ... TdNs TaNs Cs Ns SOS Ls Ns Ss Se Ah Al Cs1 Td1 Ta1 Cs Td Ta TISO0860-93/d022

Other Marker

• APP, COM, DNL, DHP, EXP …• Skip it by using the length field after the marker