Jpeg and Mpeg
-
Upload
rohitsharma -
Category
Documents
-
view
218 -
download
0
Transcript of Jpeg and Mpeg
-
8/12/2019 Jpeg and Mpeg
1/68
Introduction to JPEG andMPEG
Ingemar J. Cox
University College London
-
8/12/2019 Jpeg and Mpeg
2/68
UCLAdastra
lParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 2
Outline Elementary information theory
Lossless compression
Quantization
Fundamentals of images
Discrete Cosine Transform (DCT)
JPEG
MPEG-1, MPEG-2
-
8/12/2019 Jpeg and Mpeg
3/68
UCLAdastra
lParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 3
Bibliography D. MacKay, Information Theory, Inference and learning
Algorithms, Cambridge University Press, 2003.
http://www.inference.phy.cam.ac.uk/itprnn/book.html
W. B. Pennebaker and J. L. Mitchell, JPEG Still Image
Data Compression Standard, Chapman Hall, 1993
(ISBN 0-442-01272-1).
G. K. Wallace, The JPEG Still-Picture CompressionStandard, IEEE Trans. On Consumer Electronics, 38,
1, 18-34, 1992.
http://en.wikipedia.org/wiki/JPEG
-
8/12/2019 Jpeg and Mpeg
4/68
UCLAdastra
lParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 4
Bibliography http://en.wikipedia.org/wiki/MPEG-2
T. Sikora, MPEG Digital Video-Coding
Standards, IEEE Signal Processing Magazine,82-100, September 1997
http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2 -
8/12/2019 Jpeg and Mpeg
5/68
Elementary Information Theory
-
8/12/2019 Jpeg and Mpeg
6/68
UCLAdastra
lParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 6
Elementary Information Theory How much information does a symbol convey?
Intuitively, the more unpredictable or surprising
it is, the more information is conveyed.
Conversely, if we strongly expected something,
and it occurs, we have not learnt very much
-
8/12/2019 Jpeg and Mpeg
7/68
UCLAdastra
lParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 7
Elementary Information Theory If p is the probability that a symbol will occur
Then the amount of information, I, conveyed is:
The information, I, is measured in bits
It is the optimum code length for the symbol
pI
1log 2
-
8/12/2019 Jpeg and Mpeg
8/68
UCLAdastra
lParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 8
Elementary Information Theory The entropy, H, is the average information per
symbol
Provides a lower bound on the compression
that can be achieved
))(
1(log)( 2
spspH
s
-
8/12/2019 Jpeg and Mpeg
9/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 9
Elementary Information theory A simple example. Suppose we need to
transmit four possible weather conditions:
1. Sunny2. Cloudy
3. Rainy
4. Snowy
If all conditions are equally likely, p(s)=0.25,
and H=2
i.e. we need a minimum of 2 bits per symbol
-
8/12/2019 Jpeg and Mpeg
10/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 10
Elementary information theory Suppose instead that it is:
1. Sunny 0.5 of the time
2. Cloudy 0.25 of the time3. Rainy 0.125 of the time, and
4. Snowy 0.125 of the time
Then the entropy is
75.175.05.05.0
3125.02225.015.0
125.0
1log125.02
25.0
1log25.0
5.0
1log5.0 222
H
H
H
-
8/12/2019 Jpeg and Mpeg
11/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 11
Elementary Information Theory Variable length codewords
Huffman codeinteger code lengths
Arithmetic codesnon-integer code lengths
-
8/12/2019 Jpeg and Mpeg
12/68
-
8/12/2019 Jpeg and Mpeg
13/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 13
Elementary Information Theory Previous illustration is an example of a lossless
code
I.e. we are able to recover the information exactly
-
8/12/2019 Jpeg and Mpeg
14/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 14
Elementary Information Theory Note that we have assumed that each symbol
is independent of the other symbols
I.e. the current symbol provides no information
regarding the next symbol
-
8/12/2019 Jpeg and Mpeg
15/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 15
Quantization Quantization is the process of approximating a
continuous (or range of values) by a (much)
smaller range of values
Where Round(y) rounds y to the nearest integer
is the quantization stepsize
5.0Round),( x
xQ
-
8/12/2019 Jpeg and Mpeg
16/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 16
Quantization Example: =2
0 1-3 -2 -1 2 3 4 5-5 -4
0-1 1 2-2
0-2 2 4-4
-
8/12/2019 Jpeg and Mpeg
17/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 17
Quantization Quantization plays an important role in lossy
compression
This is where the loss happens
-
8/12/2019 Jpeg and Mpeg
18/68
Fundamentals of Images
-
8/12/2019 Jpeg and Mpeg
19/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 19
Fundamentals of images An image consists of pixels (picture elements)
Each pixel represents luminance (and colour)
Typically, 8-bits per pixel
-
8/12/2019 Jpeg and Mpeg
20/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 20
Fundamentals of images Colour
Colour spaces (representations)
RGB (red-green-blue)
CMY (cyan-magenta-yellow)
YUV
Y = 0.3R+0.6G+0.1B (luminance)
U=R-Y
V=B-Y
Greyscale
Binary
-
8/12/2019 Jpeg and Mpeg
21/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 21
Fundamentals of images A TV frame is about 640x480 pixels
If each pixels is represented by 8-bits for each
colour, then the total image size is 640480*3=921,600 bytes or 7.4Mbits
At 30 frames per second, this would be
220Mbits/second
-
8/12/2019 Jpeg and Mpeg
22/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 22
Fundamentals of images Do we need all these bits?
-
8/12/2019 Jpeg and Mpeg
23/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 23
Fundamentals of images Here is an image represented with 8-bits per
pixel
-
8/12/2019 Jpeg and Mpeg
24/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 24
Fundamentals of images Here is the same image at 7-bits per pixel
-
8/12/2019 Jpeg and Mpeg
25/68
UCLAdastr
alParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 25
Fundamentals of images And at 6-bits per pixel
-
8/12/2019 Jpeg and Mpeg
26/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 26
Fundamentals of images And at 5-bits per pixel
-
8/12/2019 Jpeg and Mpeg
27/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 27
Fundamentals of images And at 4-bits per pixel
-
8/12/2019 Jpeg and Mpeg
28/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 28
Fundamentals of images Do we need all these bits?
No!
The previous example illustrated the eyessensitivity to luminance
We can build a perceptual model
Only code what is important to the human visualsystem (HVS)
Usually a function of spatial frequency
-
8/12/2019 Jpeg and Mpeg
29/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 29
Fundamentals of Images Just as audio has temporal frequencies
Images have spatial frequencies
Transforms
Fourier transform
Discrete cosine transform
Wavelet transform Hadamard transform
-
8/12/2019 Jpeg and Mpeg
30/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 30
Discrete cosine transform Forward DCT
Inverse DCT
1
0
)5.0(8
cos)(2)()(
N
n
nunsuCuS
)5.0(8
cos)(2
)()(
1
0
nu
uSuC
nsN
u
-
8/12/2019 Jpeg and Mpeg
31/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 31
Basis functions DC term
-
8/12/2019 Jpeg and Mpeg
32/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 32
Basis functions First term
-
8/12/2019 Jpeg and Mpeg
33/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 33
Basis functions Second term
-
8/12/2019 Jpeg and Mpeg
34/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 34
Basis functions Third term
-
8/12/2019 Jpeg and Mpeg
35/68
UCLAdast
ralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 35
Basis functions Fourth term
-
8/12/2019 Jpeg and Mpeg
36/68
UCLAdast
ralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 36
Basis functions Fifth term
-
8/12/2019 Jpeg and Mpeg
37/68
UCLAdast
ralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 37
Basis functions Sixth term
-
8/12/2019 Jpeg and Mpeg
38/68
UCLAdast
ralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 38
Basis functions Seventh term
-
8/12/2019 Jpeg and Mpeg
39/68
DCT Example
-
8/12/2019 Jpeg and Mpeg
40/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 40
Example Signal
-
8/12/2019 Jpeg and Mpeg
41/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 41
Example DCT coefficients are:
4.2426
0
-3.1543
0
0
0 -0.2242
0
-
8/12/2019 Jpeg and Mpeg
42/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 42
Example: DCT decomposition DC term
-
8/12/2019 Jpeg and Mpeg
43/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 43
Example: DCT decomposition 2ndAC term
-
8/12/2019 Jpeg and Mpeg
44/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 44
Example: DCT decomposition 6thAC term
-
8/12/2019 Jpeg and Mpeg
45/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 45
Example: summation of DCT terms First two non-zero coefficients
-
8/12/2019 Jpeg and Mpeg
46/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 46
Example: summation of DCT terms All 3 non-zero coefficients
-
8/12/2019 Jpeg and Mpeg
47/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 47
Example
What if we quantize DCT coefficients?
=1
Quantized DCT coefficients are:
4
0
-3
0
0
0
0
0
-
8/12/2019 Jpeg and Mpeg
48/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 48
Example
Approximate reconstruction
-
8/12/2019 Jpeg and Mpeg
49/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 49
Example
Exact reconstruction
-
8/12/2019 Jpeg and Mpeg
50/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 50
2-D DCT Transform
Let i(x,y) represent an image with N rows and
M columns
Its DCT I(u,v) is given by
where
M
x
N
y
vyuxyxivCuCvuI
1 1 16
)12(cos
16
)12(cos),()()(
4
1),(
2
1)0( C 1)( uC
-
8/12/2019 Jpeg and Mpeg
51/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 51
Fundamentals of images
Discrete cosine transform
Coefficients are approximately uncorrelated
Except DC term
C.f. original 88 pixel block
Concentrates more power in the low frequency
coefficients
Computationally efficient
Block-based DCT
Compute DCT on 88 blocks of pixels
-
8/12/2019 Jpeg and Mpeg
52/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 52
Fundamentals of images
Basis functions for the 88 DCT (courtesy
Wikipedia)
-
8/12/2019 Jpeg and Mpeg
53/68
Fundamentals of JPEG
-
8/12/2019 Jpeg and Mpeg
54/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 54
Fundamentals of JPEG
DCT Quantizer Entropy coder
IDCT Dequantizer Entropy
decoder
Compressed
image data
Encoder
Decoder
-
8/12/2019 Jpeg and Mpeg
55/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 55
Fundamentals of JPEG
JPEG works on 88 blocks
Extract 88 block of pixels
Convert to DCT domain
Quantize each coefficient Different stepsize for each coefficient
Based on sensitivity of human visual system
Order coefficients in zig-zag order
Entropy code the quantized values
-
8/12/2019 Jpeg and Mpeg
56/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 56
Fundamentals of JPEG
A common quantization table is
16 11 10 16 24 40 51 61
12 12 14 19 26 58 60 5514 13 16 24 40 57 69 56
14 17 22 29 51 87 80 62
18 22 37 56 68 109 103 7724 35 55 64 81 104 113 92
49 64 78 87 103 121 120 101
72 92 95 98 112 100 103 99
-
8/12/2019 Jpeg and Mpeg
57/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 57
Fundamentals of JPEG
Zig-zag ordering
0 1 5 6 14 15 27 28
2 4 7 13 16 26 29 42
3 8 12 17 25 30 41 43
9 11 18 24 31 40 44 53
10 19 23 32 39 45 52 5420 22 33 38 46 51 55 60
21 34 37 47 50 56 59 61
35 36 48 49 57 58 62 63
-
8/12/2019 Jpeg and Mpeg
58/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 58
Fundamentals of JPEG
Entropy coding
Run length encoding followed by
Huffman
Arithmetic
DC term treated separately
Differential Pulse Code Modulation (DPCM)
2-step process
1. Convert zig-zag sequence to a symbol sequence
2. Convert symbols to a data stream
-
8/12/2019 Jpeg and Mpeg
59/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 59
Fundamentals of JPEG
Modes
Sequential
Progressive
Spectral selection
Send lower frequency coefficients first
Successive approximation
Send lower precision first, and subsequently refine
Lossless Hierarchical
Send low resolution image first
-
8/12/2019 Jpeg and Mpeg
60/68
Fundamentals of MPEG-1/2
-
8/12/2019 Jpeg and Mpeg
61/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 61
Fundamentals of MPEG
A sequence of 2D images
Temporal correlation as well as spatial
correlation
TV broadcast
Frame-based
Field-based
-
8/12/2019 Jpeg and Mpeg
62/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 62
MPEG
Moving Picture Experts Group
Standard for video compression
Similarities with JPEG
-
8/12/2019 Jpeg and Mpeg
63/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 63
MPEG
Design is a compromise between
Bit rate
Encoder/decoder complexity
Random access capability
-
8/12/2019 Jpeg and Mpeg
64/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 64
MPEG
Images
Spatial redundancy
Perceptual redundancy
Video
Spatial redundancy
Intraframe coding
Temporal redundancy Interframe coding
Perceptual redundancy
-
8/12/2019 Jpeg and Mpeg
65/68
UCLAdas
tralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 65
MPEG
Consider a sequence of n frames of video.
It consists of:
I-frames P-frames
B-frames
A sequence of one I-frame followed by P- and
B-frames is known as a GOP Group of Pictures
E.g. IBBPBBPBBPBBP
-
8/12/2019 Jpeg and Mpeg
66/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 66
MPEG
I-frames
Intraframe coded
No motion compensation
P-frames Interframe coded
Motion compensation
Based on past frames only
B-frames Interframe coded
Motion compensation
Based on past and future frames
-
8/12/2019 Jpeg and Mpeg
67/68
UCLAdastralParkPostgraduateCampus
Nov 27th 2006 Ingemar J. Cox 67
MPEG
Motion-compensated prediction
Divide current frame, i, into disjoint 1616
macroblocks
Search a window in previous frame, i-1, for closestmatch
Calculate the prediction error
For each of the four 88 blocks in the macroblock,
perform DCT-based coding
Transmit motion vector + entropy coded prediction
error (lossy coding)
-
8/12/2019 Jpeg and Mpeg
68/68
UCLAdastralParkPostgraduateCampus
MPEG
Like JPEG, the DC term is treated separately
DPCM
B-frame compression high Need buffer and delay