Wavelet Packets Shortcomings of standard orthogonal

(bi-orthogonal) multi-resolution structure of DWT,

Insufficient flexibility for the analysis of signal under various application requirements

Wavelet Packets Signal components with wide range of

frequencies, are packed into details D1

Desire to focus on signal components with spectral bandwidths that are sufficiently narrow at high and low frequency ranges.

Wavelet Packets Adaptive Decomposition

There are application areas such as compression or pattern recognition/ classification problems where it is highly desirable to identify wavelets with certain properties suitable for the requirements of a given applications including the following

Application Requirements

Demand for High Resolution both in Time and in Frequency

Signal decomposition and approximation by a few number of large amplitude coefficients for high compression,

Discriminatory separation of different classes in pattern recognition and classifications problems,

High efficiency of computations,

Beyond Orthogonality Compression Requirements

Slow rate of decay Fast rate of decay

Coeffs 2Coeffs 1

Orthogonal Expansion Non-orthogonal Expansion

Dictionary GenerationRedundancy and “Best Basis”

To use redundancy in wavelet packet to generate a dictionary that is composed of large number of basis functions ( basis functions that are not necessarily orthogonal) and as such to have a higher degree of freedom for selecting wavelets suitable for a given application

Time-Frequency Partitioning in Standard DWT

Constant Q factor

Fc F Fc / F = constant

Phase-plane Diagram of standard DWT and Frequency Partitioning

Perfect Reconstruction QMF Filterbank

High Pass

Low Pass Low pass

High pass

Perfect Reconstrution Filter bank

Wavelet Subspace W

Standard QMF Structure of DWT

Wavelet Packet Structure Extended QMF structure Decomposition of signal i.e. splitting

of signal into high and low resolution, is applied to high resolution details as shown below.

This is followed similarly during the synthesis stage

Subspace Illustration of Two Channel Wavelet Packet

Wj-1,0

Vj-1

Vj

Wj-1,1

Wavelet Packets

High Pass

Low Pass

2

2

2

Low pass

High pass

Wavelet Packets -Analysis Stages

2

High freq detailsHigh pass

Low pass 2

2

V space of the signal

Standard and Wavelet Packet Structure

Frequency Partitioning

Wavelet Packets Tree

Binary Tree of DWT in Wavelet Packet

Wavelet, scaling Basis at WP Cells

Spectrum of Basis inWP

Redundancy in Wavelet Packet Decomposition

Wavelet packet decomposition of a signal, results in a considerable redundancy and an increase in the number of wavelet bases in which the size of the library will grow rapidly when the number of scale levels is increased. Redundancy results in a substantial increase of both computational and storage costs

Basis Selection Algorithm A pruning algorithm is needed for

selecting a subset of nodes for signal representation considered suitable for a given application and reducing computational costs. These nodes should provide a sufficiently accurate approximation to a given signal

Basis Selection Algorithm a selection criteria is needed in

which information cost of each node in the expansion tree is utilized as the basis for retaining or discarding the node. Different cost functions may result in different approximations ( see below).

Basis Designation Indices Parent node of a pair of children

nodes A library of bases functions that are

indexed by three parameters (instead of two parameters of scale and translation as used in standard multiresolution structure) as follows. scale index j, spatial location (translation) index k frequency bands indexed by n.

Alternative Decomposition, Orthogonal or non-orthogonal

A signal can be decomposed using orthogonal bases functions that reside at children nodes at a given scale.

However this may not be a ‘best representation for a given application though an orthogonal representation..

Computational cost of signal decomposition by wavelet packet is of order O(Nlog(N)) as compared with standard wavelet decomposition which is of order of N ( i.e linear cost).

Frequency bandwidths of the details and pproximation at a given scale are equal in size, similar to FFT having an order of computational complexity of Nlog(N).

Dictionary Construction, Best Basis Selection

Cost Functions as Criteria for Best Basis Selection

Cost functions are defined in accordance with the requirements of a particular application.

They are often described in terms of concentration of information contained in the coefficients

An Example: entropy-based cot function

Properties of Cost Functions ‘Additive property’ Under additive property, total cost is

given by the sum of costs of individual nodes in a tree. It relates information cost of a node to information costs of individual coefficients at that node. C({xi}= C{ (xi)}= C(xi) C(0)=0

Different Criteria and Cost Function Entropy Measure

Entropy is a measure of uncertainty in predicting a particular outcome of an experiment

Pdf of random variable x Low Entropy High Entropy

x x

Entropy Measure We define Shannon-Weaver entropy

of a sequence of x={xj, j=1,2,…n} as following.

x={xj}, H(x)= - pj log pj pj=||xj||2/ ||x||2 and p.log p=0 if p=0

Coefficients at a node are used as xj.

Entropy Measure Under an entropy criteria for best

bases selection, wavelet packet nodes with the coefficients that are distributed across a narrow dynamic range are selected, leading to high compression rate for signal representation

Comparison with Karhunen-Loeve transform of PCA

Symmetric Entropy Measure Symmetric Entropy and Discriminatory

Classification. x={xi}, H(x)= -[ pj log pj/ qj + qj log qj/

pj ] pj=||xj||2/ ||x||2 and p.log p=0 if p=0 qj=||yj||2/ ||y||2 and q.log q=0 if q=0 coefficients at wavelet packet nodes are

used for x={xi} and y={yj}

Log of Energy Criteria

M(x) = log|xj|2, log0=0 Log of energy may be interpreted as

the entropy of so called Gauss-Markov process, a stochastic process composed of N uncorrelated Gaussian random variables with variances as

21 =|x1|2, …..2N =|xN |2

Log of Energy Criteria Minimizing log energy, finds a best

basis and best approximation for signal representation using minimum variance criteria similar to principal component analysis (PCA) and Karhunen-Lo’ve Transform (KLT)

Concentration in l p norm, p<2.

For this norm, we choose p<2 and set M(x)=||{x}||p

Maximization of this norm results in selecting a few large amplitude coefficients with high concentration of signal energy in few coefficients. That is, in a binary tree of WP structure, nodes are selected that have a few large amplitude coefficients leading to a peaky pdf

Secondary Feature of WT and l p norm In signal expansion by wavelets, it is also observed

that often large amplitude coefficients are large across several neighboring scales as well as neighboring coefficients at the given scale.

This occurs at points of sudden changes and singularities where clustering of coefficients is observed both at within scale and cross-scales.

This property of wavelets, is often referred to as a secondary feature of wavelet analysis.

Other Cost Function, Selection Rules. “Informative Wavelets”, “Dictionary Projection Pursue”’ “Local Discriminatory Basis” selection

criteria “Matching Pursue”On items 1 and 3 , papers have been

included in ‘Archives of Project Reports’ posted on course web site

Best Bases and Best Tree Best tree’ is a tree in the binary array of the

signal decomposition that corresponds to minimal cost function.

And ‘best basis’ are the basis set that belong to the nodes of the ‘best tree’. For example in entropy-based basis selection, best basis are wavelets(and scaling functions) of those nodes having minimal entropy of the coefficients at the node.Best Tree: Refer to Matlab Wavemenu for Illustrations

Illustrative Examples of basis

0 50 100 150 200 250 300-2

-1

0

1

2

wp(1

:5)

Wavelet Packet wavelets generated by wpfun("db5",100,5)

0 50 100 150 200 250 3000

10

20

30

40

fft(w

p(1:

10)

0 50 100 150 200 250 3000

10

20

30

fft(w

p(11

:20)

Illustrative Examples of WP Basis

0 500 1000 1500 2000 2500 3000-10

-5

0

5

10wp

(1:5

)

Wavelet Packet wavelets generated by wpfun("db45",100,5)

0 500 1000 1500 2000 2500 30000

500

1000

fft(w

p(1:

10)

0 500 1000 1500 2000 2500 30000

500

1000

1500

2000

fft(w

p(11

:20)

On the Illustrative Examples of WP Basis

Note Db 45 (of longer length) generates higher oscillatory wavelets (higher # of oscillations) as compared with those of Db 5 (of short length). Also note equi-frequency bandwidth of different wavelets of the same scale.

Translation of wavelets are also seen easily for different wavelets indicating the shift in the location of center of the wavelets.

Note time span of illustrations are all equal to the length of wavelet at lowest scale