8/16/2019 module8-0
1/270
Digital Sign
Module 8:
8/16/2019 module8-0
2/270
Module Overview:
◮
Module 8.1: Introduction to Images and Image Processing◮ Module 8.2: Affine Transforms
◮ Module 8.3: 2D Fourier Analysis
◮ Module 8.4: Image Filters
◮ Module 8.5: Image Compression
◮ Module 8.6: The JPEG Compression Standard
8
8/16/2019 module8-0
3/270
Digital Sign
Module 8.1:
8/16/2019 module8-0
4/270
Overview:
◮ Images as multidimensional digital signals
◮ 2D signal representations
◮ Basic signals and operators
8.1
8/16/2019 module8-0
5/270
Overview:
◮ Images as multidimensional digital signals
◮ 2D signal representations
◮ Basic signals and operators
8.1
8/16/2019 module8-0
6/270
Overview:
◮ Images as multidimensional digital signals
◮ 2D signal representations
◮ Basic signals and operators
8.1
8/16/2019 module8-0
7/270
Please meet ...
8.1
8/16/2019 module8-0
8/270
Digital images
◮ two-dimensional signal x [n1, n2], n1, n2 ∈ Z
◮ indices locate a point on a grid → pixel
◮ grid is usually regularly spaced
◮ values x [n1, n2] refer to the pixel’s appearance
8.1
8/16/2019 module8-0
9/270
Digital images
◮ two-dimensional signal x [n1, n2], n1, n2 ∈ Z
◮ indices locate a point on a grid → pixel
◮ grid is usually regularly spaced
◮ values x [n1, n2] refer to the pixel’s appearance
8.1
8/16/2019 module8-0
10/270
8/16/2019 module8-0
11/270
Digital images
◮ two-dimensional signal x [n1, n2], n1, n2 ∈ Z
◮ indices locate a point on a grid → pixel
◮ grid is usually regularly spaced
◮ values x [n1, n2] refer to the pixel’s appearance
8.1
8/16/2019 module8-0
12/270
Digital images: grayscale vs color
◮ grayscale images: scalar pixel values
◮ color images: multidimensional pixel values in a color space (RGB, HSV
◮ we can consider the single components separately:
8.1
8/16/2019 module8-0
13/270
Digital images: grayscale vs color
◮ grayscale images: scalar pixel values
◮ color images: multidimensional pixel values in a color space (RGB, HSV
◮ we can consider the single components separately:
8.1
8/16/2019 module8-0
14/270
Digital images: grayscale vs color
◮ grayscale images: scalar pixel values
◮ color images: multidimensional pixel values in a color space (RGB, HSV
◮ we can consider the single components separately:
= + +
8.1
8/16/2019 module8-0
15/270
Image processing
From one to two dimensions...
◮ something still works
◮ something breaks down
◮ something is new
8.1
8/16/2019 module8-0
16/270
Image processing
From one to two dimensions...
◮ something still works
◮ something breaks down
◮ something is new
8.1
8/16/2019 module8-0
17/270
Image processing
From one to two dimensions...
◮ something still works
◮ something breaks down
◮ something is new
8.1
I i
8/16/2019 module8-0
18/270
Image processing
What works:
◮ linearity, convolution
◮ Fourier transform
◮ interpolation, sampling
8.1
I i
8/16/2019 module8-0
19/270
Image processing
What works:
◮ linearity, convolution
◮ Fourier transform
◮ interpolation, sampling
8.1
I i
8/16/2019 module8-0
20/270
Image processing
What works:
◮ linearity, convolution
◮ Fourier transform
◮ interpolation, sampling
8.1
I i
8/16/2019 module8-0
21/270
Image processing
What breaks down:
◮ Fourier analysis less relevant
◮ filter design hard, IIRs rare
◮ linear operators only mildly useful
8.1
I i
8/16/2019 module8-0
22/270
Image processing
What breaks down:
◮ Fourier analysis less relevant
◮ filter design hard, IIRs rare
◮ linear operators only mildly useful
8.1
Image processing
8/16/2019 module8-0
23/270
Image processing
What breaks down:
◮ Fourier analysis less relevant
◮ filter design hard, IIRs rare
◮ linear operators only mildly useful
8.1
Image processing
8/16/2019 module8-0
24/270
Image processing
What’s new:◮ new manipulations: affine transforms
◮ images are finite-support signals
◮ images are (most often) available in their entirety → causality loses mea
◮ images are very specialized signals, designed for a very specific processinhuman brain! Lots of semantics that is extremely hard to deal with
8.1
Image processing
8/16/2019 module8-0
25/270
Image processing
What’s new:◮ new manipulations: affine transforms
◮ images are finite-support signals
◮ images are (most often) available in their entirety → causality loses mea
◮ images are very specialized signals, designed for a very specific processinhuman brain! Lots of semantics that is extremely hard to deal with
8.1
Image processing
8/16/2019 module8-0
26/270
Image processing
What’s new:◮ new manipulations: affine transforms
◮ images are finite-support signals
◮ images are (most often) available in their entirety → causality loses mea
◮ images are very specialized signals, designed for a very specific processinhuman brain! Lots of semantics that is extremely hard to deal with
8.1
Image processing
8/16/2019 module8-0
27/270
Image processing
What’s new:◮ new manipulations: affine transforms
◮ images are finite-support signals
◮ images are (most often) available in their entirety → causality loses mea
◮ images are very specialized signals, designed for a very specific processinhuman brain! Lots of semantics that is extremely hard to deal with
8.1
2D signal processing: the basics
8/16/2019 module8-0
28/270
2D signal processing: the basics
A two-dimensional discrete-space signal:
x [n1, n2], n1, n2 ∈ Z
8.1
2D signals: Cartesian representation
8/16/2019 module8-0
29/270
2D signals: Cartesian representation
n1 n2
h[n1, n2]
8.1
2D signals: support representation
8/16/2019 module8-0
30/270
2D signals: support representation
◮ just show coordinates of nonzerosamples
◮ amplitude may be written alongexplicitly
◮ example:
δ [n1, n2] =
1 if n1 = n2 = 0
0 otherwise.
1
−5 0
−5
0
5
n1
n 2
8.1
2D signals: image representation
8/16/2019 module8-0
31/270
2D signals: image representation
◮ medium has a certain dynamic range(paper, screen)
◮ image values are quantized (usually to8 bits, or 256 levels)
◮ the eye does the interpolation in spaceprovided the pixel density is highenough
0 85 170 255 30
85
170
255
340
425
510
n1
n 2
8.1
Why 2D?
8/16/2019 module8-0
32/270
Why 2D?
◮ images could be unrolled (printers, fax)
◮ but what about spatial correlation?
8.1
Why 2D?
8/16/2019 module8-0
33/270
Why 2D?
◮ images could be unrolled (printers, fax)
◮ but what about spatial correlation?
8.1
2D vs raster scan
8/16/2019 module8-0
34/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
35/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
36/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
37/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
38/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
39/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
40/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
2D vs raster scan
8/16/2019 module8-0
41/270
−20 −10 0 10 20−20
−10
0
10
20
0 205 410 615 820 1025 1230 14350
1
8.1
Basic 2D signals: the impulse
8/16/2019 module8-0
42/270
δ [n1, n2] =
1 if n1 = n2 = 0
0 otherwise.
−5 0
−5
0
5
8.1
Basic 2D signals: the rect
8/16/2019 module8-0
43/270
rect
n1
2N 1,
n2
2N 2
=
1 if |n1|
8/16/2019 module8-0
44/270
x [n1, n2] = x 1[n1]x 2[n2]
8.1
Separable signals
8/16/2019 module8-0
45/270
δ [n1, n2] = δ [n1]δ [n1]
rect n1
2N 1 ,
n2
2N 2
= rect n1
2N 1
rect n2
2N 2
.
8.1
Separable signals
8/16/2019 module8-0
46/270
δ [n1, n2] = δ [n1]δ [n1]
rect n1
2N 1 ,
n2
2N 2
= rect n1
2N 1
rect n2
2N 2
.
8.1
Nonseparable signal
8/16/2019 module8-0
47/270
x [n1, n2] =
1 if |n1| + |n2|
8/16/2019 module8-0
48/270
x [n1, n2] = rect
n1
2N 1,
n2
2N 2
−rect
n1
2M 1,
n2
2M 2
−5 0
−5
0
5
8.1
2D convolution
8/16/2019 module8-0
49/270
x [n1, n2] ∗ h[n1, n2] =∞
k 1=−∞
∞k 2=−∞
x [k 1, k 2]h[n1 − k 1, n2 − k
8.1
2D convolution for separable signals
8/16/2019 module8-0
50/270
If h[n1, n2] = h1[n1]h2[n2]:
x [n1, n2] ∗ h[n1, n2] =∞
k 1=−∞
h1[n1 − k 1]∞
k 2=−∞
x [k 1, k 2]h2[n2 −
= h1[n1] ∗ (h2[n2] ∗ x [n1, n2]).
8.1
2D convolution for separable signals
8/16/2019 module8-0
51/270
If h[n1, n2] is an M 1 ×M 2 finite-support signal:
◮ non-separable convolution: M 1M 2 operations per output sample
◮ separable convolution: M 1 + M 2 operations per output sample!
8.1
8/16/2019 module8-0
52/270
END OF MODULE 8.1
8/16/2019 module8-0
53/270
Digital Sign
Module 8.2: Ima
Overview:
8/16/2019 module8-0
54/270
◮ Affine transforms
◮ Bilinear interpolation
8.2
Overview:
8/16/2019 module8-0
55/270
◮ Affine transforms
◮ Bilinear interpolation
8.2
Affine transforms
8/16/2019 module8-0
56/270
mapping R2 → R2 that reshapes the coordinate system:
t ′1t ′2
=
a11 a12a21 a22
t 1t 2
−
d 1d 2
t ′1t ′2
= A
t 1t 2
− d
8.2
Affine transforms
8/16/2019 module8-0
57/270
mapping R2 → R2 that reshapes the coordinate system:
t ′1t ′2
=
a11 a12a21 a22
t 1t 2
−
d 1d 2
t ′1t ′2
= A
t 1t 2
− d
8.2
Translation
8/16/2019 module8-0
58/270
A =
1 00 1
= I
d = d 1d
2
0
0
8.2
Translation
8/16/2019 module8-0
59/270
A =
1 00 1
= I
d = d 1d
2
0
0
8.2
Scaling
8/16/2019 module8-0
60/270
A =
a1 00 a2
d = 0
if a1 = a2 the aspect ratio is preserved
0
0
8.2
Scaling
8/16/2019 module8-0
61/270
A =
a1 00 a2
d = 0
if a
1 = a
2 the aspect ratio
is preserved
0
0
8.2
Rotation
8/16/2019 module8-0
62/270
A =
cos θ − sin θsin θ cos θ
d = 0
0
0
8 2
Rotation
8/16/2019 module8-0
63/270
A =
cos θ − sin θsin θ cos θ
d = 0
0
0
8 2
Rotation
8/16/2019 module8-0
64/270
A =
cos θ − sin θsin θ cos θ
d = 0
0
0
8 2
Flips
8/16/2019 module8-0
65/270
Horizontal:
A =−1 00 1
d = 0
Vertical:
A =
1 00 −1
d = 00
0
8 2
Flips
8/16/2019 module8-0
66/270
Horizontal:
A =−1 00 1
d = 0
Vertical:
A =
1 00 −1
d = 00
0
8 2
Shear
8/16/2019 module8-0
67/270
Horizontal:
A =
1 s
0 1
d = 0
Vertical:
A =
1 0s 1
d = 00
0
8 2
Shear
8/16/2019 module8-0
68/270
Horizontal:
A =1 s
0 1
d = 0
Vertical:
A =
1 0s 1
d = 00
0
8 2
Affine transforms in discrete-space
8/16/2019 module8-0
69/270
t ′1t ′2
= A
n1n2
− d ∈ R2 = Z2
8 2
Solution for images
8/16/2019 module8-0
70/270
◮ apply the inverse transform: t 1t 2
= A−1
m1 + d 1m2 + d 2
;
◮ interpolate from the original grid point to the “mid-point”
(t 1, t 2) = (η1 + τ 1, η2 + τ 2), η1,2 ∈ Z, 0 ≤ τ 1,2
8/16/2019 module8-0
71/270
◮ apply the inverse transform: t 1t 2
= A−1
m1 + d 1m2 + d 2
;
◮ interpolate from the original grid point to the “mid-point”
(t 1, t 2) = (η1 + τ 1, η2 + τ 2), η1,2 ∈ Z, 0 ≤ τ 1,2
8/16/2019 module8-0
72/270
x [η1 + 1, η2 + 1]
x [η1 , η2]
η1 η1 + 1
η2
η2 + 1
8 2
Bilinear Interpolation
8/16/2019 module8-0
73/270
x [η1 + 1, η2 + 1]
x [η1 , η2]
(t 1, t 2)
η1 η1 + 1
η2
η2 + 1
8 2
Bilinear Interpolation
8/16/2019 module8-0
74/270
τ 1
τ 2
η1 η1 + 1
η2
η2 + 1
8 2
Bilinear Interpolation
8/16/2019 module8-0
75/270
τ 1
η1 η1 + 1
η2
η2 + 1
8 2
Bilinear Interpolation
8/16/2019 module8-0
76/270
τ 1
τ 2
η1 η1 + 1
η2
η2 + 1
8.2
Bilinear Interpolation
8/16/2019 module8-0
77/270
If we use a first-order interpolator:
y [m1, m2] = (1 − τ 1)(1− τ 2)x [η1, η2] + τ 1(1− τ 2)x [η1 + 1, η
+ (1 − τ 1)τ 2x [η1, η2 + 1] + τ 1τ 2x [η1 + 1, η2 + 1]
8.2
Shearing
8/16/2019 module8-0
78/270
8.2
8/16/2019 module8-0
79/270
END OF MODULE 8.2
8/16/2019 module8-0
80/270
Digital Sign
Module 8.3: F
Overview:
8/16/2019 module8-0
81/270
◮ DFT
◮ Magnitude and phase
8.3
Overview:
8/16/2019 module8-0
82/270
◮ DFT
◮ Magnitude and phase
8.3
2D DFT
8/16/2019 module8-0
83/270
X [k 1, k 2] =
N 1−1n1=0
N 2−1n2=0
x [n1, n2]e − j 2π
N 1n
1k
1 e − j 2π
N 2n
2k
2
x [n1, n2] = 1
N 1N 2
N 1−1
k 1=0
N 2−1
k 2=0
X [k 1, k 2]e j 2π
N 1n1k 1 e
j 2πN 2
n2k 2
8.3
2D DFT
8/16/2019 module8-0
84/270
X [k 1, k 2] =
N 1−1n1=0
N 2−1n2=0
x [n1, n2]e − j 2π
N 1n
1k
1 e − j 2π
N 2n
2k
2
x [n1, n2] = 1
N 1N 2
N 1−1
k 1=0
N 2−1
k 2=0
X [k 1, k 2]e j 2π
N 1n1k 1 e
j 2πN 2
n2k 2
8.3
2D-DFT Basis Vectors
8/16/2019 module8-0
85/270
There are N 1N 2 orthogonal basis vectors for an N 1 × N 2 imag
w k 1,k 2 [n1, n2] = e j 2πN 1
n1k 1 e j 2π
N 2n2k 2
for n1, k 1 = 0, 1, . . . , N 1 − 1 and n2, k 2 = 0, 1, . . . , N 2 − 1
8.3
2D-DFT basis vectors for k 1 = 1,
k 2 = 0 (real part)
255
8/16/2019 module8-0
86/270
0 2550
55
8.3
2D-DFT basis vectors for k 1 = 0,
k 2 = 1 (real part)
255
8/16/2019 module8-0
87/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 2,
k 2 = 0 (real part)
255
8/16/2019 module8-0
88/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 3,
k 2 = 0 (real part)
255
8/16/2019 module8-0
89/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 0,
k 2 = 3 (real part)
255
8/16/2019 module8-0
90/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 30,
k 2 = 0 (real part)
255
8/16/2019 module8-0
91/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 1,
k 2 = 1 (real part)
255
8/16/2019 module8-0
92/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 2,
k 2 = 7 (real part)
255
8/16/2019 module8-0
93/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 5,
k 2 = 250 (real part)
255
8/16/2019 module8-0
94/270
0 2550
8.3
2D-DFT basis vectors for k 1 = 3,
k 2 = 230 (real part)
255
8/16/2019 module8-0
95/270
0 2550
8.3
2D DFT
8/16/2019 module8-0
96/270
2D-DFT basis functions are separable, and so is the 2D-DFT:
X [k 1, k 2] =N 1−1n1=0
N 2−1n2=0
x [n1, n2]e − j 2π
N 1n1k 1 e
− j 2πN 2
n2k 2
◮ 1D-DFT along n2 (the columns)
◮ 1D-DFT along n1 (the rows)
8.3
2D DFT
8/16/2019 module8-0
97/270
2D-DFT basis functions are separable, and so is the 2D-DFT:
X [k 1, k 2] =N 1−1n1=0
N 2−1n2=0
x [n1, n2]e − j 2π
N 2n2k 2
e − j 2π
N 1n1k 1
◮ 1D-DFT along n2 (the columns)
◮ 1D-DFT along n1 (the rows)
8.3
2D DFT
8/16/2019 module8-0
98/270
2D-DFT basis functions are separable, and so is the 2D-DFT:
X [k 1, k 2] =N 1−
1n1=0
N 2−1n2=0
x [n1, n2]e − j 2π
N 2n2k 2
e − j 2πN 1 n1k 1
◮ 1D-DFT along n2 (the columns)
◮
1D-DFT along n1 (the rows)
8.3
2D DFT
8/16/2019 module8-0
99/270
2D-DFT basis functions are separable, and so is the 2D-DFT:
X [k 1, k 2] =N 1−
1n1=0
N 2−1n2=0
x [n1, n2]e − j 2π
N 2n2k 2
e − j 2πN 1 n1k 1
◮ 1D-DFT along n2 (the columns)
◮ 1D-DFT along n1 (the rows)
8.3
2D DFT in matrix form
◮ finite support 2D signal can be written as a matrix x
8/16/2019 module8-0
100/270
◮ finite-support 2D signal can be written as a matrix x
◮ N 1 × N 2 image is an N 2 × N 1 matrix (n1 spans the columns, n2 spans th
◮ recall also the N × N DFT matrix (Module 4.2):
WN =
1 1 1 1 . . . 1
1 W 1N W 2N W
3 . . . W N −1N 1 W 2N W
4 W 6N . . . W 2(N −1)N
. . .1 W N −1
N W
2(N −1)N
W 3(N −1)N
. . . W (N −1)2
N
8.3
2D DFT in matrix form
◮ finite support 2D signal can be written as a matrix x
8/16/2019 module8-0
101/270
◮ finite-support 2D signal can be written as a matrix x
◮ N 1 × N 2 image is an N 2 × N 1 matrix (n1 spans the columns, n2 spans th
◮ recall also the N × N DFT matrix (Module 4.2):
WN =
1 1 1 1 . . . 1
1 W 1N W 2N W
3 . . . W N −1N 1 W 2N W
4 W 6N . . . W 2(N −1)N
. . .1 W N −1
N W
2(N −1)N
W 3(N −1)N
. . . W (N −1)2
N
8.3
2D DFT in matrix form
◮ finite support 2D signal can be written as a matrix x
8/16/2019 module8-0
102/270
◮ finite-support 2D signal can be written as a matrix x
◮ N 1 × N 2 image is an N 2 × N 1 matrix (n1 spans the columns, n2 spans th
◮ recall also the N × N DFT matrix (Module 4.2):
WN =
1 1 1 1 . . . 1
1 W 1N W 2N W
3 . . . W N −1N 1 W 2N W
4 W 6N . . . W 2(N −1)N
. . .1 W N −1
N W
2(N −1)N
W 3(N −1)N
. . . W (N −1)2
N
8.3
2D DFT in matrix form
8/16/2019 module8-0
103/270
X [k 1, k 2] =
N 1−1
n1=0
N 2−1
n2=0
x [n1, n2]e − j 2π
N 2n2k 2
e
− j 2πN 1
n1k 1
8.3
2D DFT in matrix form
N 1
N 1
8/16/2019 module8-0
104/270
X [k 1, k 2] =
N 1−1
n1=0
N 2−1
n2=0x [n1, n2]e
− j 2πN 2
n2k 2
e − j 2π
N 1n1k 1
V = WN 2 x
V ∈ RN 2×N 1
8.3
2D DFT in matrix form
N1 1
N2 1
8/16/2019 module8-0
105/270
X [k 1, k 2] =
N 1−1
n1=0
N 2−1
n2=0x [n1, n2]e
− j 2πN 2
n2k 2
e − j 2π
N 1n1k 1
V = WN 2 x
V ∈ RN 2×N 1
X = VWN 1
X ∈ RN 2×N 1
8.3
2D DFT in matrix form
N1−1
N2−1
8/16/2019 module8-0
106/270
X [k 1, k 2] =
N 1−1
n1=0
N 2−1
n2=0x [n1, n2]e
− j 2πN 2
n2k 2
e − j 2π
N 1n1k 1
V = WN 2 x
V ∈ RN 2×N 1
X = VWN 1
X ∈ RN 2×N 1
X = WN 2 xWN 1
8.3
How does a 2D-DFT look like?
8/16/2019 module8-0
107/270
◮
try to show the magnitude as an image◮ problem: the range is too big for the grayscale range of paper or screen
◮ try to normalize: |X ′[n1, n2]| = |X [n1, n2]|/ max |X [n1, n2]|
◮ but it doesn’t work...
8.3
DFT coefficients sorted by magnitude
8/16/2019 module8-0
108/270
0 5122
101
103
105
107
8.3
Dealing with HDR images
8/16/2019 module8-0
109/270
if the image is high dynamic range we need to compress the levels◮ remove flagrant outliers (e.g. X [0, 0] =
x [n1, n2])
◮ use a nonlinear mapping: e.g. y = x 1/3 after normalization (x ≤ 1)
8.3
Dealing with HDR images
8/16/2019 module8-0
110/270
if the image is high dynamic range we need to compress the levels◮ remove flagrant outliers (e.g. X [0, 0] =
x [n1, n2])
◮ use a nonlinear mapping: e.g. y = x 1/3 after normalization (x ≤ 1)
8.3
How does a 2D-DFT look like?
510 510
8/16/2019 module8-0
111/270
0 85 170 255 340 425 5100
85
170
255
340
425
0 85 170 255 340
85
170
255
340
425
8.3
DFT magnitude doesn’t carry much information
8/16/2019 module8-0
112/270
8.3
DFT phase, on the other hand...
8/16/2019 module8-0
113/270
8.3
Image frequency analysis
8/16/2019 module8-0
114/270
◮ most of the information is contained in image’s edges
◮ edges are points of abrupt change in signal’s values
◮ edges are a “space-domain” feature → not captured by DFT’s magnitud
◮ phase alignment is important for reproducing edges
8.3
8/16/2019 module8-0
115/270
END OF MODULE 8.3
8/16/2019 module8-0
116/270
Digital Sign
Mod
8/16/2019 module8-0
117/270
Overview:
8/16/2019 module8-0
118/270
◮ Filters for image processing
◮ Classification
◮ Examples
8.4
Overview:
8/16/2019 module8-0
119/270
◮ Filters for image processing
◮ Classification
◮ Examples
8.4
Analogies with 1D filters
◮ li it
8/16/2019 module8-0
120/270
◮ linearity
◮
space invariance◮ impulse response
◮ frequency response
◮ stability
◮ 2D CCDE
8.4
The problem with LSI operators
◮ interesting images contain lots of semantics: different information in diff
8/16/2019 module8-0
121/270
◮ interesting images contain lots of semantics : different information in diff
◮ space-invariant filters process everything in the same way
◮ but we should process things differently
• edges
• gradients
• textures
• ...
8.4
The problem with LSI operators
◮ interesting images contain lots of semantics: different information in diff
8/16/2019 module8-0
122/270
◮ interesting images contain lots of semantics : different information in diff
◮ space-invariant filters process everything in the same way
◮ but we should process things differently
• edges
• gradients
• textures
• ...
8.4
8/16/2019 module8-0
123/270
The problem with LSI operators
◮ interesting images contain lots of semantics : different information in diff
8/16/2019 module8-0
124/270
g g
◮ space-invariant filters process everything in the same way
◮ but we should process things differently
• edges
• gradients
• textures
• ...
8.4
The problem with LSI operators
◮ interesting images contain lots of semantics : different information in diff
8/16/2019 module8-0
125/270
g g
◮ space-invariant filters process everything in the same way
◮ but we should process things differently
• edges
• gradients
• textures
• ...
8.4
The problem with LSI operators
◮ interesting images contain lots of semantics : different information in diff
8/16/2019 module8-0
126/270
g g
◮ space-invariant filters process everything in the same way
◮ but we should process things differently
• edges
• gradients
• textures
• ...
8.4
The problem with LSI operators
◮ interesting images contain lots of semantics : different information in diff
8/16/2019 module8-0
127/270
g g
◮ space-invariant filters process everything in the same way
◮ but we should process things differently
• edges
• gradients
• textures
• ...
8.4
Filter types
8/16/2019 module8-0
128/270
◮ IIR, FIR
◮ causal or noncausal
◮ highpass, lowpass, ...
• lowpass → image smoothing
• highpass → enhancement, edge detection
8.4
Filter types
8/16/2019 module8-0
129/270
◮ IIR, FIR
◮ causal or noncausal
◮ highpass, lowpass, ...
• lowpass → image smoothing
• highpass → enhancement, edge detection
8.4
Filter types
8/16/2019 module8-0
130/270
◮ IIR, FIR
◮ causal or noncausal
◮ highpass, lowpass, ...
• lowpass → image smoothing
• highpass → enhancement, edge detection
8.4
Filter types
8/16/2019 module8-0
131/270
◮ IIR, FIR
◮ causal or noncausal
◮ highpass, lowpass, ...
• lowpass → image smoothing
• highpass → enhancement, edge detection
8.4
Filter types
8/16/2019 module8-0
132/270
◮ IIR, FIR
◮ causal or noncausal
◮ highpass, lowpass, ...
• lowpass → image smoothing
• highpass → enhancement, edge detection
8.4
The problems with 2D IIRs
8/16/2019 module8-0
133/270
◮ nonlinear phase (edges!)
◮ border effects
◮ stability: the fundamental theorem of algebra doesn’t hold in multiple di
◮ computability
8.4
The problems with 2D IIRs
8/16/2019 module8-0
134/270
◮ nonlinear phase (edges!)
◮ border effects
◮ stability: the fundamental theorem of algebra doesn’t hold in multiple di
◮ computability
8.4
The problems with 2D IIRs
8/16/2019 module8-0
135/270
◮ nonlinear phase (edges!)
◮ border effects
◮ stability: the fundamental theorem of algebra doesn’t hold in multiple di
◮ computability
8.4
8/16/2019 module8-0
136/270
A noncomputable CCDE
y [n1, n2] = a0y [n1 + 1, n2] + a1y [n1, n2 − 1] + a2y [n1 − 1, n2] + a3y [n1, n2 + 1] +
2
8/16/2019 module8-0
137/270
y [0, 0]
y [1, 0]
y [0, 1]
y [−1, 0]
y [0,−1]
−2 −1 0 1 2
−2
−
1
0
1
2
8.4
A noncomputable CCDE
y [n1, n2] = a0y [n1 + 1, n2] + a1y [n1, n2 − 1] + a2y [n1 − 1, n2] + a3y [n1, n2 + 1] +
2
8/16/2019 module8-0
138/270
−2 −1 0 1 2
−2
−1
0
1
2
8.4
Practical FIR filters
◮ generally zero centered (causality not an issue) ⇒
8/16/2019 module8-0
139/270
ge e a y e o ce te ed (causa ty ot a ssue) ⇒odd number of taps in both directions
◮ per-sample complexity:
• M 1M 2 for nonseparable impulse responses
• M 1 + M 2 for separable impulse responses
◮ obviously always stable
8.4
Practical FIR filters
◮ generally zero centered (causality not an issue) ⇒
8/16/2019 module8-0
140/270
g y ( y )odd number of taps in both directions
◮ per-sample complexity:
• M 1M 2 for nonseparable impulse responses
• M 1 + M 2 for separable impulse responses
◮ obviously always stable
8.4
Practical FIR filters
◮ generally zero centered (causality not an issue) ⇒
8/16/2019 module8-0
141/270
g y ( y )odd number of taps in both directions
◮ per-sample complexity:
• M 1M 2 for nonseparable impulse responses
• M 1 + M 2 for separable impulse responses
◮ obviously always stable
8.4
Practical FIR filters
◮ generally zero centered (causality not an issue) ⇒
8/16/2019 module8-0
142/270
g y ( y )odd number of taps in both directions
◮ per-sample complexity:
• M 1M 2 for nonseparable impulse responses
• M 1 + M 2 for separable impulse responses
◮ obviously always stable
8.4
Practical FIR filters
◮ generally zero centered (causality not an issue) ⇒
8/16/2019 module8-0
143/270
( )odd number of taps in both directions
◮ per-sample complexity:
• M 1M 2 for nonseparable impulse responses
• M 1 + M 2 for separable impulse responses
◮ obviously always stable
8.4
Moving Average
[ ]1
N N
[ k k ]
8/16/2019 module8-0
144/270
y [n1, n2] = 1
(2N + 1)2
k 1=−N
k 2=−N
x [n1 − k 1, n2 − k 2]
h[n1, n2] = 1
(2N + 1)2rect
n1
2N ,
n2
2N
8.4
Moving Average
[ ] 1
N N
[ k k ]
8/16/2019 module8-0
145/270
y [n1, n2] =
(2N + 1)2
k 1=−N
k 2=−N
x [n1 − k 1, n2 − k 2]
h[n1, n2] = 1
(2N + 1)2rect
n1
2N ,
n2
2N
8.4
Moving Average
8/16/2019 module8-0
146/270
h[n1, n2] = 1
9
1 1 11 1 1
1 1 1
8.4
Moving Average
8/16/2019 module8-0
147/270
11× 11 MA 51 × 51 MA
8.4
Gaussian Blur
8/16/2019 module8-0
148/270
h[n1, n2] = 12πσ2
e −n2
1+n2
22σ2 , |n1, n2|
8/16/2019 module8-0
149/270
n1 n2
8.4
8/16/2019 module8-0
150/270
Gaussian Blur
8/16/2019 module8-0
151/270
σ = 1.8, 11 × 11 blur σ = 8.7, 51 × 51 blu
8.4
Sobel filter
approximation of the first derivative in the horizontal direction:
s [n1 n2] =−1 0 1
2 0 2
8/16/2019 module8-0
152/270
s o [n1, n2] = −2 0 2−1 0 1
separability and structure:
s o [n1, n2] =1
21
−1 0 1
8.4
Sobel filter
approximation of the first derivative in the horizontal direction:
s [n1 n2] =−1 0 1−2 0 2
8/16/2019 module8-0
153/270
s o [n1, n2] = −2 0 2−1 0 1
separability and structure:
s o [n1, n2] =1
21
−1 0 1
8.4
Sobel filter
approximation of the first derivative in the vertical direction:
8/16/2019 module8-0
154/270
pp
s v [n1, n2] =
−1 −2 10 0 0
1 2 1
8.4
Sobel filter
8/16/2019 module8-0
155/270
horizontal Sobel filter vertical Sobel filter
8.4
Sobel operator
approximation for the square magnitude of the gradient:
8/16/2019 module8-0
156/270
|∇x [n1, n2]|2 = |s o [n1, n2] ∗ x [n1, n2]|2 + |s v [n1, n2] ∗ x [n1, n2]
(“operator” because it’s nonlinear)
8.4
Gradient approximation for edge detection
8/16/2019 module8-0
157/270
Sobel operator thresholeded Sobel oper
8.4
Laplacian operator
Laplacian of a function in continuous-space:
8/16/2019 module8-0
158/270
Laplacian of a function in continuous-space:
∆f (t 1, t 2) = ∂ 2f
∂ t 21+
∂ 2f
∂ t 22
8.4
Laplacian operator
approximating the Laplacian; start with a Taylor expansion
f (t + τ ) =∞
0
f (n)(t )
n! τ n
8/16/2019 module8-0
159/270
n=0and compute the expansion in (t + τ ) and (t − τ ):
f (t + τ ) = f (t ) + f ′(t )τ + 1
2f ′′(t )τ 2
f (t − τ ) = f (t )− f ′(t )τ + 12
f ′′(t )τ 2
8.4
Laplacian operator
by rearranging terms:1
8/16/2019 module8-0
160/270
f ′′
(t ) = 1
τ 2 (f (t − τ ) − 2f (t ) + f (t + τ ))
which, on the discrete grid, is the FIR h[n] =
1 −2 1
8.4
Laplacian
summing the horizontal and vertical components:
8/16/2019 module8-0
161/270
h[n1, n2] =
0 1 01 −4 1
0 1 0
8.4
Laplacian
If we use the diagonals too:
8/16/2019 module8-0
162/270
If we use the diagonals too:
h[n1, n2] =
1 1 11 −8 11 1 1
8.4
Laplacian for Edge Detection
8/16/2019 module8-0
163/270
Laplacian operator thresholeded Laplacian op
8.4
8/16/2019 module8-0
164/270
END OF MODULE 8.4
8/16/2019 module8-0
165/270
Digital SignModule 8.5: Im
Overview:
8/16/2019 module8-0
166/270
◮ Redundancy in natural images
◮ Image coding ingredients
8.5
Overview:
8/16/2019 module8-0
167/270
◮ Redundancy in natural images
◮ Image coding ingredients
8.5
A thought experiment
◮ consider all possible 256 × 256, 8bpp images
8/16/2019 module8-0
168/270
◮ each image is 524,288 bits
◮ total number of possible images: 2524,288 ≈ 10157,826
◮ number of atoms in the universe: 1082
8.5
A thought experiment
◮ consider all possible 256 × 256, 8bpp images
8/16/2019 module8-0
169/270
◮ each image is 524,288 bits
◮ total number of possible images: 2524,288 ≈ 10157,826
◮ number of atoms in the universe: 1082
8.5
A thought experiment
◮ consider all possible 256 × 256, 8bpp images
8/16/2019 module8-0
170/270
◮ each image is 524,288 bits
◮ total number of possible images: 2524,288 ≈ 10157,826
◮ number of atoms in the universe: 1082
8.5
A thought experiment
◮ consider all possible 256 × 256, 8bpp images
8/16/2019 module8-0
171/270
◮ each image is 524,288 bits
◮ total number of possible images: 2524,288 ≈ 10157,826
◮ number of atoms in the universe: 1082
8.5
How many bits per image?
Another thought experiment
◮ take all images in the world and list them in an “encyclopedia of images
◮ to indicate an image, simply give its number
8/16/2019 module8-0
172/270
◮ on the Internet: M = 50 billion
◮ raw encoding: 524,288 bits per image
◮ enumeration-based encoding: log2 M ≈ 36 bits per image
◮ (of course, side information is HUGE)
8.5
8/16/2019 module8-0
173/270
How many bits per image?
Another thought experiment
◮ take all images in the world and list them in an “encyclopedia of images
◮ to indicate an image, simply give its number
8/16/2019 module8-0
174/270
◮ on the Internet: M = 50 billion
◮ raw encoding: 524,288 bits per image
◮ enumeration-based encoding: log2 M ≈ 36 bits per image
◮ (of course, side information is HUGE)
8.5
How many bits per image?
Another thought experiment
◮ take all images in the world and list them in an “encyclopedia of images
◮ to indicate an image, simply give its number
8/16/2019 module8-0
175/270
◮ on the Internet: M = 50 billion
◮ raw encoding: 524,288 bits per image
◮ enumeration-based encoding: log2 M ≈ 36 bits per image
◮ (of course, side information is HUGE)
8.5
How many bits per image?
Another thought experiment
◮ take all images in the world and list them in an “encyclopedia of images
◮ to indicate an image, simply give its number
8/16/2019 module8-0
176/270
◮ on the Internet: M = 50 billion
◮ raw encoding: 524,288 bits per image
◮ enumeration-based encoding: log2 M ≈ 36 bits per image
◮ (of course, side information is HUGE)
8.5
How many bits per image?
Another thought experiment
◮ take all images in the world and list them in an “encyclopedia of images
◮ to indicate an image, simply give its number
8/16/2019 module8-0
177/270
◮ on the Internet: M = 50 billion
◮ raw encoding: 524,288 bits per image
◮ enumeration-based encoding: log2 M ≈ 36 bits per image
◮ (of course, side information is HUGE)
8.5
Compression
Another approach:
◮ exploit “physical” redundancy
8/16/2019 module8-0
178/270
◮ allocate bits for things that matter (e.g. edges)
◮ use psychovisual experiments to find out what matters
◮ some information is discarded: lossy compression
8.5
Compression
Another approach:
◮ exploit “physical” redundancy
8/16/2019 module8-0
179/270
◮ allocate bits for things that matter (e.g. edges)
◮ use psychovisual experiments to find out what matters
◮ some information is discarded: lossy compression
8.5
Compression
Another approach:
◮ exploit “physical” redundancy
8/16/2019 module8-0
180/270
◮ allocate bits for things that matter (e.g. edges)
◮ use psychovisual experiments to find out what matters
◮ some information is discarded: lossy compression
8.5
Compression
Another approach:
◮ exploit “physical” redundancy
8/16/2019 module8-0
181/270
◮ allocate bits for things that matter (e.g. edges)
◮ use psychovisual experiments to find out what matters
◮ some information is discarded: lossy compression
8.5
Key ingredients
◮ compressing at block level
◮ ( )
8/16/2019 module8-0
182/270
◮ using a suitable transform (i.e., a change of basis)
◮ smart quantization
◮ entropy coding
8.5
Key ingredients
◮ compressing at block level
◮ bl f ( h f b )
8/16/2019 module8-0
183/270
◮ using a suitable transform (i.e., a change of basis)
◮ smart quantization
◮ entropy coding
8.5
Key ingredients
◮ compressing at block level
◮ i i bl f (i h f b i )
8/16/2019 module8-0
184/270
◮ using a suitable transform (i.e., a change of basis)
◮ smart quantization
◮ entropy coding
8.5
Key ingredients
◮ compressing at block level
◮ i i bl f (i h f b i )
8/16/2019 module8-0
185/270
◮ using a suitable transform (i.e., a change of basis)
◮ smart quantization
◮ entropy coding
8.5
Compressing at pixel level
◮ reduce number bits per pixel
8/16/2019 module8-0
186/270
◮ equivalent to coarser quantization
◮ in the limit, 1bpp
8.5
Compressing at pixel level
◮ reduce number bits per pixel
8/16/2019 module8-0
187/270
◮ equivalent to coarser quantization
◮ in the limit, 1bpp
8.5
Compressing at pixel level
◮ d b bit i l
8/16/2019 module8-0
188/270
◮ reduce number bits per pixel
◮ equivalent to coarser quantization
◮ in the limit, 1bpp
8.5
Compressing at block level
◮ divide the image in blocks
◮ code the average value with 8 bits
8/16/2019 module8-0
189/270
◮ code the average value with 8 bits
◮ 3× 3 blocks at 8 bits per block gives lessthan 1bpp
8.5
Compressing at block level
◮ divide the image in blocks
◮ code the average value with 8 bits
8/16/2019 module8-0
190/270
◮ code the average value with 8 bits
◮ 3× 3 blocks at 8 bits per block gives lessthan 1bpp
8.5
Compressing at block level
◮ divide the image in blocks
8/16/2019 module8-0
191/270
◮ code the average value with 8 bits
◮ 3× 3 blocks at 8 bits per block gives lessthan 1bpp
8.5
Compressing at block level
◮
exploit the local spatial correlation
8/16/2019 module8-0
192/270
exploit the local spatial correlation◮ compress remote regions independently
8.5
Compressing at block level
◮
exploit the local spatial correlation
8/16/2019 module8-0
193/270
exploit the local spatial correlation◮ compress remote regions independently
8.5
Transform coding
A simple example:
◮ take a DT signal, assume R bits persample
◮ storing the signal requires NR bits
8/16/2019 module8-0
194/270
storing the signal requires NR bits
◮ now you take the DFT and it looks likethis
◮ in theory, we can just code the twononzero DFT coefficients!
0 5 10 15 20
8.5
Transform coding
A simple example:
◮ take a DT signal, assume R bits persample
◮ storing the signal requires NR bits
8/16/2019 module8-0
195/270
storing the signal requires NR bits
◮ now you take the DFT and it looks likethis
◮ in theory, we can just code the twononzero DFT coefficients!
0 5 10 15 20
8.5
Transform coding
A simple example:
◮ take a DT signal, assume R bits persample
◮ storing the signal requires NR bits
8/16/2019 module8-0
196/270
storing the signal requires NR bits
◮ now you take the DFT and it looks likethis
◮ in theory, we can just code the twononzero DFT coefficients!
0 5 10 15 20
8.5
Transform coding
A simple example:
◮ take a DT signal, assume R bits persample
◮ storing the signal requires NR bits
8/16/2019 module8-0
197/270
storing the signal requires NR bits
◮ now you take the DFT and it looks likethis
◮ in theory, we can just code the twononzero DFT coefficients!
0 5 10 15 20
8.5
Transform coding
Ideally, we would like a transform that:
◮ captures the important features of an image block in a few coefficients
i ffi i t t t
8/16/2019 module8-0
198/270
◮ is efficient to compute
◮ answer: the Discrete Cosine Transform
8.5
Transform coding
Ideally, we would like a transform that:
◮ captures the important features of an image block in a few coefficients
i ffi i t t t
8/16/2019 module8-0
199/270
◮ is efficient to compute
◮ answer: the Discrete Cosine Transform
8.5
Transform coding
Ideally, we would like a transform that:
◮ captures the important features of an image block in a few coefficients
◮ is efficient to comp te
8/16/2019 module8-0
200/270
◮ is efficient to compute
◮ answer: the Discrete Cosine Transform
8.5
2D-DCT
C [k 1, k 2] =N −1 N −1
x [n1, n2]cos
πN
n1 + 1
2
k 1
cos
πN
n2 +
8/16/2019 module8-0
201/270
[ 1, 2]n1=0
n2=0
[ 1, 2]
N
1 +
2
1
N
2 +
8.5
DCT basis vectors for an 8 × 8 image
8/16/2019 module8-0
202/270
8.5
Smart quantization
◮ deadzone
bl (fi )
8/16/2019 module8-0
203/270
◮ variable step (fine to coarse)
8.5
Smart quantization
◮ deadzone
i bl (fi )
8/16/2019 module8-0
204/270
◮ variable step (fine to coarse)
8.5
Quantization
Standard quantization:1
2x̂ [n]
00
8/16/2019 module8-0
205/270
x̂ = floor(x ) + 0.5
−1
−2
1 −1−2
11
10
8.5
Quantization
Deadzone quantization:
1
2x̂ [n]
00
01
8/16/2019 module8-0
206/270
x̂ = round(x )
−1
−2
1 −1−2
10
00
8.5
Entropy coding
◮ minimize the effort to encode a certain amount of information
◮ associate short symbols to frequent values and vice-versa
8/16/2019 module8-0
207/270
◮ if it sounds familiar it’s because it is...
8.5
Entropy coding
◮ minimize the effort to encode a certain amount of information
◮ associate short symbols to frequent values and vice-versa
8/16/2019 module8-0
208/270
◮ if it sounds familiar it’s because it is...
8.5
Entropy coding
◮ minimize the effort to encode a certain amount of information
◮ associate short symbols to frequent values and vice-versa
8/16/2019 module8-0
209/270
◮ if it sounds familiar it’s because it is...
8.5
Entropy coding
8/16/2019 module8-0
210/270
8.5
END OF MODULE 8.5
8/16/2019 module8-0
211/270
Digital Sign
Module 8 6: The JPEG Comp
8/16/2019 module8-0
212/270
Module 8.6: The JPEG Comp
Key ingredients
◮ compressing at block level
◮
using a suitable transform (i.e., a change of basis)t ti ti
8/16/2019 module8-0
213/270
◮ smart quantization
◮ entropy coding
8.6
8/16/2019 module8-0
214/270
Key ingredients
◮ split image into 8 × 8 non-overlapping blocks
◮
compute the DCT of each block◮ smart quantization
8/16/2019 module8-0
215/270
◮ smart quantization
◮ entropy coding
8.6
Key ingredients
◮ split image into 8 × 8 non-overlapping blocks
◮
compute the DCT of each block◮ quantize DCT coefficients according to psycovisually tuned tables
8/16/2019 module8-0
216/270
◮ quantize DCT coefficients according to psycovisually-tuned tables
◮ entropy coding
8.6
Key ingredients
◮ split image into 8 × 8 non-overlapping blocks
◮ compute the DCT of each block
◮ quantize DCT coefficients according to psycovisually tuned tables
8/16/2019 module8-0
217/270
◮ quantize DCT coefficients according to psycovisually-tuned tables
◮ run-length encoding and Huffman coding
8.6
DCT coefficients of image blocks (detail)
8/16/2019 module8-0
218/270
8.6
DCT coefficients of image blocks (detail)
8/16/2019 module8-0
219/270
8.6
Smart quantization
◮ most coefficients are negligible → captured by the deadzone
◮ some coefficients are more important than others
◮ find out the critical coefficients by experimentation
8/16/2019 module8-0
220/270
◮ allocate more bits (or, equivalently, finer quantization levels) to the mostcoefficients
8.6
Smart quantization
◮ most coefficients are negligible → captured by the deadzone
◮ some coefficients are more important than others
◮ find out the critical coefficients by experimentation
8/16/2019 module8-0
221/270
◮ allocate more bits (or, equivalently, finer quantization levels) to the mostcoefficients
8.6
Smart quantization
◮ most coefficients are negligible → captured by the deadzone
◮ some coefficients are more important than others
◮ find out the critical coefficients by experimentation
8/16/2019 module8-0
222/270
◮ allocate more bits (or, equivalently, finer quantization levels) to the mostcoefficients
8.6
Smart quantization
◮ most coefficients are negligible → captured by the deadzone
◮ some coefficients are more important than others
◮ find out the critical coefficients by experimentation
8/16/2019 module8-0
223/270
◮ allocate more bits (or, equivalently, finer quantization levels) to the mostcoefficients
8.6
Psychovisually-tuned quantization table
ĉ [k 1, k 2] = round(c [k 1, k 2]/Q [k 1, k 2])
Q
16 11 10 16 24 40 51 6112 12 14 19 26 58 60 55
14 13 16 24 40 57 69 5614 17 22 29 51 87 80 62
8/16/2019 module8-0
224/270
Q =
18 22 37 56 68 109 103 7724 35 55 64 81 104 113 9249 64 78 87 103 121 120 10172 92 95 98 112 100 103 99
8.6
Advantages of nonuniform bit allocation
8/16/2019 module8-0
225/270
uniform tuned
8.6
Advantages of nonuniform bit allocation (detail)
8/16/2019 module8-0
226/270
uniform tuned
8.6
Efficient coding
◮ most coefficients are small, decreasing with index
◮ use zigzag scan to maximize ordering
◮ ti ti ill t l i f
8/16/2019 module8-0
227/270
◮ quantization will create long series of zeros
8.6
Efficient coding
◮ most coefficients are small, decreasing with index
◮ use zigzag scan to maximize ordering
◮ quantization will create long series of zeros
8/16/2019 module8-0
228/270
◮ quantization will create long series of zeros
8.6
Efficient coding
◮ most coefficients are small, decreasing with index
◮ use zigzag scan to maximize ordering
◮ quantization will create long series of zeros
8/16/2019 module8-0
229/270
◮ quantization will create long series of zeros
8.6
Zigzag scan
8/16/2019 module8-0
230/270
8.6
Example
100 −60 0 6 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
13 −1 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
8/16/2019 module8-0
231/270
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
100, -60, 0, 0, 0, 0, 6, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 00, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
8.6
Example
100 −60 0 6 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
13 −1 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
8/16/2019 module8-0
232/270
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
100, -60, 0, 0, 0, 0, 6, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 00, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
8.6
Runlength encoding
Each nonzero value is encoded as the triple
[(r , s ), c ]
◮ r is the runlength i.e. the number of zeros before the current value◮ s is the size i.e. the number of bits needed to encode the value
8/16/2019 module8-0
233/270
s is the size i.e. the number of bits needed to encode the value
◮ c is the actual value
◮ (0, 0) indicates that from now on it’s only zeros (end of block)
8.6
Runlength encoding
Each nonzero value is encoded as the triple
[(r , s ), c ]
◮ r is the runlength i.e. the number of zeros before the current value◮ s is the size i.e. the number of bits needed to encode the value
8/16/2019 module8-0
234/270
◮ c is the actual value
◮ (0, 0) indicates that from now on it’s only zeros (end of block)
8.6
Runlength encoding
Each nonzero value is encoded as the triple
[(r , s ), c ]
◮
r is the runlength i.e. the number of zeros before the current value◮ s is the size i.e. the number of bits needed to encode the value
8/16/2019 module8-0
235/270
◮ c is the actual value
◮ (0, 0) indicates that from now on it’s only zeros (end of block)
8.6
Runlength encoding
Each nonzero value is encoded as the triple
[(r , s ), c ]
◮
r is the runlength i.e. the number of zeros before the current value◮ s is the size i.e. the number of bits needed to encode the value
8/16/2019 module8-0
236/270
◮ c is the actual value
◮ (0, 0) indicates that from now on it’s only zeros (end of block)
8.6
Example
100 −60 0 6 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
13 −1 0 0 0 0 0 0
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
8/16/2019 module8-0
237/270
0 0 0 0 0 0 0 0
[(0, 7), 100], [(0, 6), −60], [(4, 3), 6], [(3, 4), 13], [(8, 1), −1], [(0
8.6
Example
100 −60 0 6 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
13 −1 0 0 0 0 0 0
0 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
8/16/2019 module8-0
238/270
0 0 0 0 0 0 0 0
[(0, 7), 100], [(0, 6), −60], [(4, 3), 6], [(3, 4), 13], [(8, 1), −1], [(0
8.6
The runlength-size pairs
◮ by design, (r , s ) ∈ A with |A| = 256
◮ in theory, 8 bits per pair
◮ some pairs are much more common than others!
8/16/2019 module8-0
239/270
◮ a lot of space can be saved by being smart
8.6
The runlength-size pairs
◮ by design, (r , s ) ∈ A with |A| = 256
◮ in theory, 8 bits per pair
◮ some pairs are much more common than others!
8/16/2019 module8-0
240/270
◮ a lot of space can be saved by being smart
8.6
The runlength-size pairs
◮ by design, (r , s ) ∈ A with |A| = 256
◮ in theory, 8 bits per pair
◮ some pairs are much more common than others!
8/16/2019 module8-0
241/270
◮ a lot of space can be saved by being smart
8.6
The runlength-size pairs
◮ by design, (r , s ) ∈ A with |A| = 256
◮ in theory, 8 bits per pair
◮ some pairs are much more common than others!
8/16/2019 module8-0
242/270
◮ a lot of space can be saved by being smart
8.6
Variable-length encoding
great idea: shorter binary sequences for common symbols
8/16/2019 module8-0
243/270
8.6
Variable-length encoding
however: if symbols have different lengths, we must know how to pa
◮ in English, spaces separate words → extra symbol (wasteful)
◮ in Morse code, pauses separate letters and words (wasteful)
◮ can we do away with separators?
8/16/2019 module8-0
244/270
◮ can we do away with separators?
8.6
Variable-length encoding
however: if symbols have different lengths, we must know how to pa
◮ in English, spaces separate words → extra symbol (wasteful)
◮ in Morse code, pauses separate letters and words (wasteful)
◮ can we do away with separators?
8/16/2019 module8-0
245/270
◮ can we do away with separators?
8.6
Variable-length encoding
however: if symbols have different lengths, we must know how to pa
◮ in English, spaces separate words → extra symbol (wasteful)
◮ in Morse code, pauses separate letters and words (wasteful)
◮ can we do away with separators?
8/16/2019 module8-0
246/270
◮ can we do away with separators?
8.6
Prefix-free codes
◮ no valid sequence can be the beginning of another valid sequence
◮
can parse a bitstream sequentially with no look-ahead◮ extremely easy to understand graphically...
8/16/2019 module8-0
247/270
8.6
Prefix-free codes
◮ no valid sequence can be the beginning of another valid sequence
◮
can parse a bitstream sequentially with no look-ahead◮ extremely easy to understand graphically...
8/16/2019 module8-0
248/270
8.6
Prefix-free codes
◮ no valid sequence can be the beginning of another valid sequence
◮
can parse a bitstream sequentially with no look-ahead◮ extremely easy to understand graphically...
8/16/2019 module8-0
249/270
8.6
Prefix-free code
A0
1
0
C0
D1
B1
8/16/2019 module8-0
250/270
001100110101100
8.6
Prefix-free code
A0
1
0
C0
D1
B1
8/16/2019 module8-0
251/270
001100110101100
8.6
Prefix-free code
A0
1
0
C0
D1
B1
8/16/2019 module8-0
252/270
001100110101100
A
8.6
Prefix-free code
A0
1
0
C0
D1
B1
8/16/2019 module8-0
253/270
001100110101100
AA
8.6
Prefix-free code
A0
1
0
C0
D1
B1
8/16/2019 module8-0
254/270
001100110101100
AAB
8.6
Prefix-free code
A0
1
0
C0
D1
B1
8/16/2019 module8-0
Top Related