Designs of Orthogonal Filter Banks and Orthogonal Cosine ...jyan/Publications/Thesis slides.pdf ·...

Designs of Orthogonal Filter Banks andOrthogonal Cosine-Modulated Filter Banks

Jie Yan

Department of Electrical and Computer EngineeringUniversity of Victoria

April 16, 2010

1 / 45

OUTLINE

1 INTRODUCTION

2 LS DESIGN OF ORTHOGONAL FILTER BANKS ANDWAVELETS

3 MIMINAX DESIGN OF ORTHOGONAL FILTER BANKS ANDWAVELETS

4 DESIGN OF ORTHOGONAL COSINE-MODULATED FILTERBANKS

5 CONCLUSIONS AND FUTURE RESEARCH

2 / 45

1. INTRODUCTION

A two-channel conjugate quadrature (CQ) filter bank

H1(z) = −z−(N−1)H0(−z−1)G0(z) = H1(−z)G1(z) = −H0(−z)

where H0(z) =∑N−1

n=0 hnz−n

0 ( )H z

1( )H z

2

2

2

2

0 ( )G z

1( )G z}Analysis Filter Bank

}Synthesis Filter Bank

3 / 45

Two-Channel Orthogonal Filter Banks

Perfect reconstruction (PR) condition

N−1−2m∑n=0

hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2

Vanishing moment (VM) requirement:

A CQ filter has L vanishing moments if

N−1∑n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1

4 / 45

Two-Channel Orthogonal Filter Banks Cont’d

A least squares (LS) design of CQ lowpass filter H0(z) having LVMs

minimize∫ �

!a

∣H0(ej!)∣2d!

subject to: PR condition and VM requirement

The LS problem above can be expressed as

minimize hTQh

subject to:N−1−2m∑

n=0

hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2

N−1∑n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1

5 / 45

Two-Channel Orthogonal Filter Banks Cont’d

A minimax design minimizes the maximum instantaneous powerof H0(z) over its stopband

minimize maximize!a≤!≤�

∣H0(ej!)∣

subject to: PR condition and VM requirement

The minimax problem can be further cast as

minimize �

subject to: ∥T(!) ⋅ h∥ ≤ � for ! ∈ ΩN−1−2m∑

n=0

hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2

N−1∑n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1

6 / 45

Orthogonal Cosine-Modulated Filter Banks

An orthogonal cosine-modulated (OCM) filter bank

hk(n) = 2h(n) cos[�

M

(k +

12

)(n− D

2

)+ (−1)k�

4

]fk(n) = 2h(n) cos

[�

M

(k +

12

)(n− D

2

)− (−1)k�

4

]for 0 ≤ k ≤ M − 1 and 0 ≤ n ≤ N − 1

x(n) y(n)H0(z)

HM-1(z)

M M

M M

M M

. . .

. . .

H1(z)

F0(z)

FM-1(z)

F1(z)

+

} }

7 / 45

Orthogonal Cosine-Modulated Filter Banks Cont’d

An M-channel OCM filter bank is uniquely characterized by itsprototype filter (PF)The design of the PF of an OCM filter bank can be formulated as

minimize∫ �

!s

∣H0(ej!)∣2d!

subject to: PR condition

As the PF has linear phase, h is symmetrical. The design problemcan be reduced to

minimize e2(h) = hT Phsubject to: al,n(h) = hTQl,nh− cn = 0

for 0 ≤ n ≤ m− 1 and 0 ≤ l ≤ M/2− 1

where the design variables are reduced by half toh = [h0 h1 ⋅ ⋅ ⋅ hN/2−1]T .

8 / 45

Overview and Contribution of the Thesis

Overview

We have formulated three nonconvex optimization problems

LS design of CQ filter banks

Minimax design of CQ filter banks

Design of OCM filter banks

Contribution of the thesis

Several improved local design methods for the three problems

Several strategies proposed for potentially GLOBAL solutions ofthe three problems

9 / 45

Global Design Method at a Glance

Multiple local solutions exist for a nonconvex problem

Algorithms in finding a locally optimal solution are available

Start the local design algorithm from a good initial point

How do we secure such a good initial point?

10 / 45

2. LS DESIGN OF ORTHOGONAL FILTER BANKS AND WAVELETS

A least squares (LS) design of a conjugate quadrature (CQ) filter

of length-N with L vanishing moments (VMs) can be cast as

minimize hTQh

subject to:N−1−2m∑

n=0

hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2

N−1∑n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1

11 / 45

Local LS Design of CQ Filter Banks

An effective direct design method is recently proposed by

W.-S. Lu and T. Hinamoto

Based on the direct design technique, we develop two local

methods

Sequential convex-programming (SCP) method

Sequential quadratic-programming (SQP) method

Both methods produce improved local designs than the directmethod

12 / 45

Local LS Design of CQ Filter Banks Cont’d

Sequential Convex-Programming Method

Suppose we are in the kth iteration to compute �h so that hk+1 = hk + �h

reduces the filter’s stopband energy and better satisfies the constraints, then

hTk+1Qhk+1 = �T

h Q�h + 2�Th Qhk + hT

k Qhk

N−1∑n=0

(−1)n ⋅ nl ⋅ (�h)n = −N−1∑n=0

(−1)n ⋅ nl ⋅ (hk)n

N−1−2m∑n=0

(hk)n(�h)n+2m +

N−1−2m∑n=0

(hk)n+2m(�h)n

≈ �m −N−1−2m∑

n=0

(hk)n(hk)n+2m

13 / 45


With h bounded to be small, the kth iteration assumes the form

minimize �Th Q�h + �T

h gk

subject to: Ak�h = −ak

C�h ≤ b

By using SVD to remove the equality constraint, the problem is reduced to

minimize xTQx + xT gk

subject to: Cx ≤ b

We modify the problem to make it always feasible as

minimize xTQx + xT gk

subject to: Fx ≤ a

which is a convex QP problem.

14 / 45


Sequential Quadratic-Programming Method

The design problem is a general nonlinear optimization problem

minimize f (h)

subject to: ai(h) = 0 for i = 1, 2, ..., p

By using the first-order necessary conditions of a local minimizer,

the problem can be reduced to

minimize12�T

h Wk�h + �Th gk

subject to: Ak�h = −ak

∣∣�h∣∣ is small

15 / 45


where

Wk = ∇2hf (hk)−

p∑i=1

(�k)i∇2hai(hk) (13a)

Ak =[∇ha1(hk) ∇ha2(hk) ⋅ ⋅ ⋅ ∇hap(hk)

]T(13b)

gk = ∇hf (hk) (13c)

ak =[

a1(hk) a2(hk) ⋅ ⋅ ⋅ ap(hk)]T (13d)

By removing the equality constraint using the SVD or QR decomposition, theproblem assumes the form of a QP problem. Once the minimizer �∗h is found,the next iterate is set to

hk+1 = hk + �∗h , �k+1 = (AkATk )−1Ak(Wk�

∗h + gk)

16 / 45

Global LS Design of Low-Order CQ Filter Banks

The LS design problem is a polynomial optimization problem

(POP)

Two recent breakthroughs in solving POPs

Global solutions of POPs are made available by Lasserre’s

method

Sparse SDP relaxation is proposed for global solutions of

POPs of relatively larger scales

MATLAB toolbox SparsePOP and GloptiPoly can be used to

find global solutions of POPs, but only for POPs of limited sizes

17 / 45

Global LS Design of Low-Order CQ Filter Banks Cont’d

Example: Design a globally optimal LS CQ filter with N = 6, L = 2 and!a = 0.56�

MATLAB toolbox GloptiPoly and SparsePOP are utilized toproduce the globally optimal solution

h(6,2)LS =

⎡⎢⎢⎢⎢⎢⎢⎣

0.332680987886290.806895914548490.45986215652386−0.13501431772967−0.08543638600240

0.03522516035714

⎤⎥⎥⎥⎥⎥⎥⎦However, GloptiPoly and SparsePOP fail to work as long as thefilter length N is greater than or equal to 18

18 / 45

Global LS Design of High-Order CQ Filter Banks

A common pattern shared among globally optimal low-order impulseresponses.

0 0.2 0.4 0.6 0.8 1

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

N = 6, L = 2N = 8, L = 2N = 10, L = 2

19 / 45

Global LS Design of High-Order CQ Filter Banks Cont’d

h6: Globally optimal impulse response when N = 6hzp

8 : Impulse response generated by zero-padding h6h8: Globally optimal impulse response when N = 8

0 0.2 0.4 0.6 0.8 1

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

h6 (N=6, L=2)

h8zp

h8 (N=8, L=2)

Generate initial point by zero-padding!20 / 45


Global design strategy in brief:

1 Design a globally optimal CQ filter of short length, say 4, using

e.g. GloptiPoly

2 Generating an impulse response for higher order design by

zero-padding

3 Apply the SCP or SQP method with the zero-padded impulse

response as the initial point to obtain the optimal impulse

response of higher order

4 Follow this concept in an iterative way, until desired filter length

is reached

21 / 45


The designs obtained are quite likely to be globally optimal because:1 Zero-padded initial point sufficiently close to the global minimizer.2 The local design methods are known to converge to a nearby

minimizer.

22 / 45

Design Examples

Potentially globally optimal design of an LS CQ filter with N = 96, L = 3and ! = 0.56�

0 0.2 0.4 0.6 0.8 1−120

−100

−80

−60

−40

−20

0

Normalized frequency

23 / 45

Design Examples Cont’d

ComparisonsGlobal design

Global design based on SCPEnergy in stopband 1.18103e-9

Largest eq. error 1e-14

Local design

Local design based on SCPEnergy in stopband 3.15564e-9

Largest eq. error 1e-14

24 / 45


Zero-pole plots

−1 −0.5 0 0.5 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

4 95

Real Part

Imag

inar

y Pa

rt

−1.5 −1 −0.5 0 0.5 1 1.5

−1

−0.5

0

0.5

1

3 95

Real PartIm

agin

ary

Part

Global design Local design

The globally optimal LS CQ filter possesses minimum phase

25 / 45

3. MIMINAX DESIGN OF ORTHOGONAL FILTER BANKS ANDWAVELETS

A minimax design of a conjugate quadrature (CQ) filter of length-N

with L vanishing moments (VMs) can be cast as

minimize �

subject to: ∥T(!) ⋅ h∥ ≤ � for ! ∈ ΩN−1−2m∑

n=0

hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2

N−1∑n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1

26 / 45

Local Minimax Design of CQ Filter Banks

Like the LS design, an effective direct design method is recently

proposed by W.-S. Lu and T. Hinamoto

Based on the direct design technique, we develop an improved

method named the SCP-GN method

The SCP-GN method can achieve convergence at a small

tolerance ", by implementing two techniques

1 Constructing Ω by locating magnitude-response peaks

2 A Gauss-Newton method with adaptively controlled weights

27 / 45

Local Minimax Design of CQ Filter Banks Cont’d

In the kth iteration, the problem assumes the form

minimize �

subject to: ∥T(!)(hk + �h)∥ ≤ � for ! ∈ Ω

Ak�h = −ak

C�h ≤ b

By using SVD of matrix Ak to remove the equality constraints, the problem can bereduced to

minimize �

subject to: ∥Tk(!)x + ek(!)∥ ≤ � for ! ∈ Ω

Cx ≤ b

As a technical remedy to make the above problem to be always feasible, we modifythe problem as

minimize �

subject to: ∥Tk(!)x + ek(!)∥ ≤ � for ! ∈ Ω

Fx ≤ a

which is a second-order cone programming (SOCP) problem for which efficientsolvers such as SeDuMi exist. 28 / 45

Global Minimax Design of Low-Order CQ Filter Banks

Example: Design a globally optimal minimax CQ filter with N = 4,

L = 1 and !a = 0.56�

Since the Minimax design problem is a POP, GloptiPoly and

SparsePOP can be used to produce the globally optimal solution

h(4,1)minimax =

⎡⎢⎢⎣0.482962821735310.836516231382340.22414405492402−0.12940935473280

⎤⎥⎥⎦However, GloptiPoly and SparsePOP fail to work as long as the

filter length N is greater than or equal to 6

29 / 45

Global Minimax Design of High-Order CQ Filter Banks

Method 1Globally optimal minimax impulse responses appear to exhibit a

pattern similar to that in the LS caseThus, we proposed method 1 in spirit similar to that utilized in the

global LS designs by passing the zero-padded impulse response

as the initial point for the SCP-GN local method in each round ofiteration

Method 2We simply pass the impulse response of the globally optimal LS

filter as an initial point for the SCP-GN method to design an

optimal minimax filter with the same design specifications

30 / 45

Design Examples

Potentially globally optimal design of a minimax CQ filter with N = 96,L = 3 and ! = 0.56�

0 0.2 0.4 0.6 0.8 1

−100

−80

−60

−40

−20

0


31 / 45


ComparisonsGlobal design

Global design based on Method 1Maximum instantaneous

energy in stopband 6.75750e-9Largest eq. error <1e-15

Local design

Local design based on SCP-GNMaximum instantaneous

energy in stopband 1.81165e-8Largest eq. error 2.9e-14

32 / 45


Zero-pole plots

−1 −0.5 0 0.5 1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

4 95

Real Part

Imag

inar

y Pa

rt

−1 −0.5 0 0.5 1 1.5

−1

−0.5

0

0.5

1

4 95

Real Part

Imag

inar

y Pa

rt


The globally optimal minimax CQ filter possesses minimum phase

33 / 45

4. DESIGN OF ORTHOGONAL COSINE-MODULATED FILTERBANKS

We have formulated the design of the prototype filter (PF) of an

orthogonal cosine-modulated (OCM) filter bank as

minimize e2(h) = hT Phsubject to: al,n(h) = hTQl,nh− cn = 0

for 0 ≤ n ≤ m− 1 and 0 ≤ l ≤ M/2− 1

34 / 45

Local Design of OCM Filter Banks

We improved an effective direct design method proposed by

W.-S. Lu, T. Saramäki and R. Bregovic

Gauss-Newton method with adaptively controlled weights was

applied for the algorithm to converge to a highly accurate solution

35 / 45

Local Design of OCM Filter Banks Cont’d

Suppose we are in the kth iteration to compute � so that hk+1 = hk + �reduces the PF’s stopband energy and better satisfies the PR conditions.Then,

hTk+1Phk+1 = �T P� + 2�T Phk + hT

k Phk

al,n(hk + �) ≈ al,n(hk) + gTl,n(hk)� = 0

for 0 ≤ n ≤ m− 1 and 0 ≤ l ≤ M/2− 1

And the kth iteration assumes the form

minimize �T P� + �Tbk

subject to: Gk� = −ak

∥�∥ is small

36 / 45

Local Design of OCM Filter Banks Cont’d

The equality constraint can be eliminated via SVD of Gk = UΣV as

� = Ve� + �s (18)

Thus, the problem can be cast as

minimize �T Pk� + �T bk

subject to: ∣∣�∣∣ is small

The Gauss-Newton technique with adaptively controlled weights is used as

a post-processing step to achieve convergence at a small tolerance.

37 / 45

Global Design of Low-Order OCM Filter Banks

Example: Design a globally optimal OCM filter bank with M = 2, m = 1and � = 1

GloptiPoly and SparsePOP can be used to produce the globallyoptimal solution

h(2,1) =

⎡⎢⎢⎣0.2359234169663530.4408402673665810.4408402673665810.235923416966353

⎤⎥⎥⎦The software was found to work only for the following cases:a) M = 2, 1 ≤ m ≤ 5;b) M = 4, 1 ≤ m ≤ 3;c) M = 6, m = 1;d) M = 8, m = 1.

38 / 45

Global Design of High-Order OCM Filter Banks

Two observations:1 For a fixed M, the impulse responses with different m exhibit a

similar pattern and are close to each other2 For m = 1, the impulse responses with different M also exhibit a

similar shape.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

m=1,M=2m=2,M=2m=3,M=2m=4,M=2m=1,M=4m=2,M=4

39 / 45

Global Design of High-Order OCM Filter Banks Cont’d

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

m=1,M=4

h0zp

m=2,M=4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

m=1,M=2

h0int

m=1,M=4

Effect of zero-padding when M = 4 Effect of linear interpolation when m = 1

40 / 45


An improvement in initial point when m = 1, by downshifting hint0 by a

constant value d computed using the Gauss-Newton method withadaptively controlled weights.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

h0int

h0

h

d

41 / 45


An order-recursive algorithm in brief

1 Obtaining a low-order global design;

2 Using zero-padding/linear interpolation in conjuction of the G-N

method with adaptively controlled weights of the impulse response

to produce a desirable initial point for PF of slightly increased

order, and carrying out the design by a locally optimal method;

3 Repeating step 2 until the filter order reaches the targeted value.

42 / 45

Design Examples

Design of an OCM filter bank with m = 20, M = 4 and � = 1. Shownbelow are the impulse responses of the PF from global and localdesign, respectively.

0 0.2 0.4 0.6 0.8 1−160

−140

−120

−100

−80

−60

−40

−20

0

20

Normalized frequency0 0.2 0.4 0.6 0.8 1

−200

−180

−160

−140

−120

−100

−80

−60

−40

−20

0

20



43 / 45


Performance comparison for OCM filter banks with m = 20, M = 4

and � = 1

Global design Local designEnergy in stopband 8.226e-13 6.585e-10

Largest eq. error 1.839e-15 2.297e-10

By comparing the OCM filter banks reported in the literature, the

OCM filter bank designed using our proposed algorithm offers the

BEST performance, because it is a globally optimal design.

44 / 45

5. CONCLUSIONS AND FUTURE RESEARCH

We have investigated three design problems,

1 LS design of orthogonal filter banks and wavelets

2 Minimax design of orthogonal filter banks and wavelet

3 Design of OCM filter banks

Improved local design methods for the three problems

Several strategies proposed for GLOBAL designs of the

three design scenarios

Future research

Theoretical proof of the global optimality of our proposed method

45 / 45

Designs of Orthogonal Filter Banks and Orthogonal Cosine ...jyan/Publications/Thesis slides.pdf ·...

Documents

Transcript of Designs of Orthogonal Filter Banks and Orthogonal Cosine ...jyan/Publications/Thesis slides.pdf ·...