Tree-based Filter Banks and based Filter Banks and...
Transcript of Tree-based Filter Banks and based Filter Banks and...
ENEE630 P tENEE630 P t 11
TreeTree--based Filter Banks andbased Filter Banks and
ENEE630 PartENEE630 Part--11
TreeTree--based Filter Banks and based Filter Banks and MultiresolutionMultiresolution AnalysisAnalysis
CECE DepartmentUniv. of Maryland, College Park
Updated 10/2012 by Prof. Min Wu. bb eng md ed (select ENEE630) min @eng md ed
M. Wu: ENEE630 Advanced Signal Processing (Fall 2010)
bb.eng.umd.edu (select ENEE630); [email protected]
Dynamic Range of Original Dynamic Range of Original andand SubbandSubband SignalsSignalsand and SubbandSubband SignalsSignals
– Can assign more bits to represent coarse info
– Allocate remaining bits, if available, to finer details (via proper quantization)
M. Wu: ENEE630 Advanced Signal Processing [8]
Figures from Gonzalez/ Woods DIP 3/e book website.
Brief Note on Brief Note on SubbandSubband and Wavelet Codingand Wavelet Coding
The octave (“dyadic”) frequency partition can reflect the logarithmatic characteristics in human perceptiong p p
Wavelet coding and subband coding have many similarities (e.g. from filter bank perspectives)( g p p )– Traditionally subband coding uses filters that have little
overlap to isolate different bands
– Wavelet transform imposes smoothness conditions on the filters that usually represent a set of basis generated by shifting and scaling (“dilation”) of a “mother wavelet” functionshifting and scaling ( dilation ) of a mother wavelet function
– Wavelet can be motivated from overcoming the poor time-domain localization of short-time FT
Explore more in Proj#1. See PPV Book Chapter 11
M. Wu: ENEE630 Advanced Signal Processing [9]
Review and Examples of BasisReview and Examples of Basis
Standard basis vectors
00
110
301
636
Standard basis images
1
01013
006
13
g
1000
00100
30010
20001
20322
Example: representing a vector with different basis
10
501
353
1 1
111
4
22
22
22
22
224
M. Wu: ENEE631 Digital Image Processing (Spring 2010) Lec10 – Unitary Transform [10]
TimeTime--Freq (or SpaceFreq (or Space--Freq) InterpretationsFreq) Interpretations– Inverse transf. represents a signal as a linear combination of basis vectors– Forward transf. determines combination coeff. by projecting signal onto basisE.g. Standard Basis (for data samples); Fourier Basis; Wavelet Basis
M. Wu: ENEE630 Advanced Signal Processing [11]
Figures from Gonzalez/ Woods DIP 2/e book website.
Recall: Matrix/Vector Form of DFT Recall: Matrix/Vector Form of DFT
1
0)(1)(
N
n
nkNWnz
NkZ
{ z(n) } { Z(k) }n k = 0 1 N 1 W = exp{ j2 / N }
1
0)(1)(
N
k
nkNWkZ
Nnz
01/2
004)
n, k = 0, 1, …, N-1, WN = exp{ - j2 / N } ~ complex conjugate of primitive Nth root of unity
by M
.Wu
© 2
00
azZ T
*)0(1 1 11)0(
Slid
es (c
reat
ed
za
a
zz
eeeeee
NZZ
T
TNNjNjNj
NNjNjNj
*
*1
0
/)1(22/42/22
/)1(2/22/2
)2()1(
1 1
1)2()1(
)0()0(1 1 11)0(
ZZz
CP
ENEE
631
S aNzeeeNZT
NNNjNNjNNj
1/)1(2/)1(22/)1(2 )1(1)1( 2
)1(
)1()0(
)1(
)2()1(
1 1
1
)1(
)2()1(
110
/)1(2/)1(22/)1(2
/)1(22/42/22
/)1(2/22/2
2 NZ
ZZ
aaa
NZ
ZZ
eeeeee
N
N
zz
N
NNjNNjNNj
NNjNjNj
NNjNjNj
UM
C
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [13]
)(
)1(1)1( /)1(2/)1(22/)1(2 2 NZeeeNz NNjNNjNNj
Example of 1Example of 1--D DCT: N = 8D DCT: N = 8From Ken Lam’s DCT talk 2001 (HK Polytech)
2001
)
1.0
0.0
1.0
0.0
100
0
100
0
ed b
y M
.Wu
©
-1.0
1.0
0.0
-1.0
1.0
0.0
-100
100
0
100
0
-100u=0 u=0 to 4
n
z(n)
1 S
lides
(cre
ate 0.0
-1.0
1.0
0.0
-1.0
1.0
0
-100
100 100
0
-100u=0 to 1 u=0 to 5Original signal
MC
P E
NE
E63
Z(k)
0.0
-1.0
1.0
0.0
-1.0
1.0 100
0
-100
100
0
-100u=0 to 2 u=0 to 6
U
k
Transform coeff.
1.0
0.0
-1.0
1.0
0.0
-1.0
0
-100
100
0
-100u=0 to 3 u=0 to 7
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [14]
Basis vectors Reconstructions
Haar TransformHaar Transform 1x
(0,0)
(0,1)
Haar basis functions: index by (p, q)– Scaling captures info. at different freq.
T l i i f diff
2001
)
(1,1)
– Translation captures info. at different locations
– Transition at each scale p is localized died
by
M.W
u ©
(1,2)
(2,1)according to q
Haar transform H ~ orthogonal
– Sample Haar function to obtain1 S
lides
(cre
ate
– Sample Haar function to obtain transform matrix
– Filter bank representation filtering and downsamplingM
CP
EN
EE
63
filtering and downsampling
– Relatively poor energy compaction Equiv. filter response doesn’t have good
cutoff & stopband attenuation
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [15]
ff p
=> Basis images of 2-D Haar transform
Compare Basis Images of DCT and HaarCompare Basis Images of DCT and Haar
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [16]
See also: Jain’s Fig.5.2 pp136UMCP ENEE631 Slides (created by M.Wu © 2001)
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [19]
Compressive SensingCompressive Sensing
Downsampling as a data compression tool– For bandlimited signals. Considered uniform sampling so farFor bandlimited signals. Considered uniform sampling so far
More general case of “sparsity” in some domainE.g. non-zero coeff. at a small # of frequencies but over a broad support
of frequency?
H l h i d d li ?– How to leverage such sparsity to get reduced average sampling rate?– Can we sample at non-equally spaced intervals?– How to deal with real-world issues e.g. approx. but not exactly sparse?g pp y p
Ref: IEEE Signal Processing Magazine: Lecture Notes on Compressive Sensing Ref: IEEE Signal Processing Magazine: Lecture Notes on Compressive Sensing (2007); Special Issue on Compressive Sensing (2008);(2007); Special Issue on Compressive Sensing (2008);
UMD ENEE630 Advanced Signal Processing (F'10) Discussions [20]
ENEE698A Fall 2008 Graduate Seminar: ENEE698A Fall 2008 Graduate Seminar: http://terpconnect.umd.edu/~dikpal/enee698a.htmlhttp://terpconnect.umd.edu/~dikpal/enee698a.html
L1 vs. L2 OptimizationL1 vs. L2 Optimizationfor Sparse Signalfor Sparse Signalfor Sparse Signalfor Sparse Signal
UMD ENEE630 Advanced Signal Processing (F'10) Discussions [21]
( Fig. from Candes-Wakins SPM’08 article)
Example: Tomography problemExample: Tomography problem
Logan-Shepp phantomtest image
Sampling in the frequency planeAlong 22 radial lines with 512
samples on each
Minimum energy reconstruction
Reconstruction by minimizingtotal variation
UMD ENEE630 Advanced Signal Processing (F'10) Discussions [22]
Slide source: by Dikpal Reddy, ENEE698A, http://terpconnect.umd.edu/~dikpal/enee698a.html
2004
)
A Close Look at Wavelet TransformA Close Look at Wavelet Transform
ed b
y M
.Wu
©
Haar Transform Haar Transform –– unitaryunitary1 S
lides
(cre
ate
yy
Orthonormal Wavelet FiltersOrthonormal Wavelet Filters
MC
P E
NE
E63
Biorthogonal Wavelet FiltersBiorthogonal Wavelet FiltersU
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [33]
Construction of Haar FunctionsConstruction of Haar Functions Unique decomposition of integer k (p, q)
– k = 0, …, N-1 with N = 2n, 0 <= p <= n-1q = 0 1 (for p=0); 1 <= q <= 2p (for p>0) k = 2p + q 1
power of 2
01/2
004)
– q = 0, 1 (for p=0); 1 <= q <= 2p (for p>0)
e.g., k=0 k=1 k=2 k=3 k=4 …(0,0) (0,1) (1,1) (1,2) (2,1) …
k = 2p + q – 1“remainder”
by M
.Wu
© 2
00
hk(x) = h p,q(x) for x [0,1] ]1,0[for 1)()( 000 xxhxh
1x
Slid
es (c
reat
ed
(0,0)
(0,1)
221for 21
],[)()(
21
2/
0,00
q-xq-N
N
ppp
CP
ENEE
631
S
(1,1)
22
for 21)()( 2
12/
,
qxq-N
xhxh ppp
qpk
UM
C
(1,2)
(2,1)
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [34]
]1,0[other for 0 x
( , )
More on Wavelets (1)More on Wavelets (1)20
04)
Linear expansion of a function via an expansion set
ed b
y M
.Wu
©
– Form basis functions if the expansion is unique
Orthogonal basis
1 S
lides
(cre
ate g
Non-orthogonal basis – Coefficients are computed with
MC
P E
NE
E63 a set of dual-basis
Discrete Wavelet TransformWavelet expansion gives a setU – Wavelet expansion gives a set of 2-parameter basis functions and expansion coefficients:scale and translation
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [17]
scale and translation
More on Wavelets (2)More on Wavelets (2)20
04)
1st generation wavelet systems:– Scaling and translation of a generating wavelet (“mother wavelet”)
Multiresolution conditions:
ed b
y M
.Wu
© Multiresolution conditions:
– Use a set of basic expansion signals with half width and translated in half step size to represent a larger class of signals than the original expansion set (the “scaling function”)
f
1 S
lides
(cre
ate
Represent a signal by combining scaling functions and wavelets
MC
P E
NE
E63
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [18]
Orthonormal FiltersOrthonormal Filters Equiv. to projecting input signal to orthonormal basis
Energy preservation property2001
)
Energy preservation property– Convenient for quantizer design
MSE by transform domain quantizer is same as reconstruction MSE in image domained
by
M.W
u ©
g
Shortcomings: “coefficient expansion”– Linear filtering with N-element input & M-element filter1
Slid
es (c
reat
e
g p (N+M-1)-element output (N+M)/2 after downsample
– Length of output per stage grows ~ undesirable for compression
MC
P E
NE
E63
Solutions to coefficient expansion– Symmetrically extended input (circular convolution) &
S i fil
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [36]
Symmetric filter
Solutions to Coefficient ExpansionSolutions to Coefficient Expansion Circular convolution in place of linear con ol tion Circular convolution in place of linear convolution
– Periodic extension of input signal– Problem: artifacts by large discontinuity at borders
2001
)
Symmetric extension of input– Reduce border artifacts (note the signal length doubled with symmetry)– Problem: output at each stage may not be symmetric F U it h (IEEEed
by
M.W
u ©
Problem: output at each stage may not be symmetric From Usevitch (IEEE Sig.Proc. Mag. 9/01)
1 S
lides
(cre
ate
MC
P E
NE
E63
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [37]
Solutions to Coefficient Expansion (cont’d)Solutions to Coefficient Expansion (cont’d)
Symmetric extension + symmetric filters– No coefficient expansion and little artifacts20
01)
No coefficient expansion and little artifacts– Symmetric filter (or asymmetric filter) => “linear phase filters”
(no phase distortion except by delays)
ed b
y M
.Wu
©
Problem– Only one set of linear phase filters for real FIR orthogonal wavelets1
Slid
es (c
reat
e
Only one set of linear phase filters for real FIR orthogonal wavelets Haar filters: (1, 1) & (1,-1)
do not give good energy compaction
MC
P E
NE
E63
Ref: review ENEE630 discussions on FIR perfect reconstruction Qudrature Mirror Filters (QMF) for 2-channel filter banks.
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [38]
Biorthogonal WaveletsBiorthogonal WaveletsFrom Usevitch (IEEE Sig.Proc. Mag. 9/01)
“Biorthogonal”– Basis in forward and inverse transf.
t th b t i ll2001
)
are not the same but give overall perfect reconstruction (PR) recall EE630 PR filterbank
N t i t th lit f t fed b
y M
.Wu
©
– No strict orthogonality for transf. filters so energy is not preserved But could be close to orthogonal
filters’ performance1 S
lides
(cre
ate
f p f
Advantage– Covers a much broader class of filters
i l di t i filt th t li i t ffi i t iMC
P E
NE
E63
including symmetric filters that eliminate coefficient expansion
Commonly used filters for compression– 9/7 biorthogonal symmetric filter
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [39]
g y– Efficient implementation: Lifting approach (ref: Swelden’s tutorial)
Smoothness Conditions on Wavelet FilterSmoothness Conditions on Wavelet Filter20
04)
– Ensure the low band coefficients obtained by recursive filtering can provide a smooth approximation of the original signal
ed b
y M
.Wu
©
1 S
lides
(cre
ate
MC
P E
NE
E63
U
M. Wu: ENEE630 Advanced Signal Processing (Fall'09) 10/14/2009 [40]
From M. Vetterli’s wavelet/filter-bank paper