Pyramid coding and subband codingee290t/sp04/lectures/lec5.pdf · Wavelet bases ( )() ()() () 2 2 2...

1

Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 1

Multiresolution coding and wavelets

Predictive (closed-loop) pyramidsOpen-loop (“Laplacian”) pyramidsDiscrete Wavelet Transform (DWT)Quadrature mirror filters and conjugate quadrature filtersLifting and reversible wavelet transformWavelet theoryEmbedded zero-tree wavelet (EZW) coding


Interpolation error coding, I

Interpolator

InterpolatorSubsampling

• • •

• • •

Input picture

Reconstructed picture

Q

Q

+

+

-

- +

+

+

+


• • • •

Coder includes

Decoder

Sample encoded in current stage

Previously coded sample

2


Interpolation error coding, II

signals to be encoded

original image


Predictive pyramid, I

Interpolator

Subsampling

Q

Q

+

+-

- +

+

+

+

InterpolatorInterpolator

Filtering

Filtering

Subsampling

Input picture


• • •

• • •• • • •

Coder includes

Decoder

Sample encoded in current stage

3


Predictive pyramid, II

Number of samples to be encoded =

1+1N

+1

N 2 + ...⎛ ⎝

⎞ ⎠ =

NN −1

x number of original image samples

subsampling factor


Predictive pyramid, III

original image

signals to be encoded

4


Comparison:interpolation error coding vs. pyramid

Resolution layer #0, interpolated to original size for display

Interpolation Error Coding Pyramid





5







Resolution layer #3

=(original)


6


Open-loop pyramid (Laplacian pyramid)

Interpolator

Interpolator

Subsampling

Q+

Q+ -

-

Filtering

Filtering

Interpolator


Input picture


• • • • • • • •

Receiver Transmitter

[Burt, Adelson, 1983]


When multiresolution coding was a new idea . . .

This manuscript is okay if compared to some of the weaker papers.[. . .] however, I doubt that anyone will ever use this algorithm again.

Anonymous reviewer of Burt and Adelson‘s original paper, ca. 1982

7


Cascaded analysis / synthesis filterbanks

0h0g

1g

1h

0g

1g

0h

1h


Discrete Wavelet Transform

Recursive application of a two-band filter bank to the lowpass band of the previous stage yields octave band splitting:

Same concept can be derived from wavelet theory: Discrete Wavelet Transform (DWT)

frequency

8


2-d Discrete Wavelet Transform

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

...etc


2-d Discrete Wavelet Transform example

9





10





11


Two-channel filterbank

Aliasing cancellation if :0 1

1 0

( ) ( )( ) ( )

g z h zg z h z

= −− = −

[ ]

[ ] ( )

0 0 1 1

0 0 1 1

1ˆ( ) ( ) ( ) ( ) ( ) ( )21 ( ) ( ) ( ) ( )2

x z h z g z h z g z x z

h z g z h z g z x z

= +

+ − + − −Aliasing

2 2

2 2

0h

1h

0g

1g

( )x z ( )x̂ z


Example: two-channel filter bank with perfect reconstruction

Impulse responses, analysis filters:Lowpass Highpass

Impulse responses, synthesis filtersLowpass Highpass

Mandatory in JPEG2000Frequency responses:

|g |0

|h |0

1|h |

1|g |

π2

π

1

2

00

Frequency

Freq

uenc

y re

spon

se

1 1 3 1 1, , , ,4 2 2 2 4− −⎛ ⎞

⎜ ⎟⎝ ⎠

1 1 1, ,4 2 4

−⎛ ⎞⎜ ⎟⎝ ⎠

1 1 3 1 1, , , ,4 2 2 2 4

−⎛ ⎞⎜ ⎟⎝ ⎠

1 1 1, ,4 2 4

⎛ ⎞⎜ ⎟⎝ ⎠

“Biorthogonal 5/3 filters”“LeGall filters”

12


frequency ω

Classical quadrature mirror filters (QMF)

QMFs achieve aliasing cancellation by choosing

Highpass band is the mirror image of the lowpass band in the frequency domainNeed to design only one prototype filter

1 0

1 0

( ) ( )( ) ( )

h z h zg z g z

= −= − = −

Example:16-tap QMF filterbank

[Croisier, Esteban, Galand, 1976]


Conjugate quadrature filters

Achieve aliasing cancelation by

Impulse responses

Orthonormal subband transform!Perfect reconstruction: find power complementary prototype filter

( )( ) ( )

10 0

1 11 1

( ) ( )

( )

h z g z f z

h z g z zf z

−

− −

= ≡

= = − [Smith, Barnwell, 1986]

Prototype filter

[ ] [ ] [ ][ ] [ ] ( ) ( )

0 0

11 1 1 1k

h k g k f k

h k g k f k+

= − =

= − = − − +⎡ ⎤⎣ ⎦

( ) ( )2 22F Fω ω π+ ± =

13


Lifting

Analysis filters

L “lifting steps”First step can be interpreted as prediction of odd samples from the even samples

K0

1λ 2λ 1Lλ − LλΣ Σ

Σ Σ

K1

[ ]even samples 2x n

[ ]odd samples

2 1x n +

0low band y

1high band y

[Sweldens 1996]


Lifting (cont.)

Synthesis filters

Perfect reconstruction (biorthogonality) is directly built into lifting structurePowerful for both implementation and filter/wavelet design


Σ Σ[ ]even samples 2x n

[ ]odd samples

2 1x n +

0low band y

1high band y

10K −

11K −

-

-

-

-

14


Example: lifting implementation of 5/3 filters


( )12

z− + 114z −+

Σ

Σ

1/2[ ]

odd samples 2 1x n +

0low band y

1high band y

Verify by considering response to unit impulse in even and odd input channel.


Reversible subband transform

Observation: lifting operators can be nonlinear.Incorporate the necessary rounding into lifting operator:

Used in JPEG2000 as part of 5/3 biorthogonal wavelet transform

K0


Σ Σ

K1


[ ]odd samples

2 1x n +

0low band y

1high band y

15


Wavelet bases

( ) ( )

( )( ) ( ) ( )

( )

2

2

2

Consider Hilbert space of finite-energy functions .

Wavelet basis for : family of linearly independent functions

2 2

that span . Hence any signal

m m mn

x t

t t nψ ψ− −

=

= −

∈

x

x

L

L

L ( )( ) [ ] ( )

2 can be written as

m mn

m n

y n ψ∞ ∞

=−∞ =−∞

= ∑ ∑x

L

“mother wavelet”


Multi-resolution analysis

( ) ( ) ( ) ( ) ( ) ( )

( ) ( )( ) { }

( ) ( )

2 1 0 1 2 2

2

0

Upward compl

Nested subspaces

eteness

Downward complete

ness

Self-similari t fy i

m

m

m

m

V V V V V

V

V

x t V

− −

∈

∈

⊂ ⊂ ⊂ ⊂ ⊂ ⊂ ⊂

=

=

∈

0

… …

∪∩

Z

Z

L

L

( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ){ } ( )

0 0

0

f 2

iff for all

There exists a "scaling function" with integer translates -

such that forms an orthonormal basis f

Translation i

or

nvaria

nce

mm

n

n n

x t V

x t V x t n V n

t t t n

V

ϕ ϕ ϕ

ϕ

−

∈

∈

∈ − ∈ ∈

=

Z

16


Multiresolution Fourier analysis

ω

( ){ } ( )span pn

p

nVϕ =

( ){ } ( )11span p pn n

Vϕ − −=

( ){ } ( )22span p pn n

Vϕ − −=

( ){ } ( )33span p pn n

Vϕ − −=

( ){ } ( )span pn

p

nWψ =

( ){ } ( )11span p pn n

Wψ − −=

( ){ } ( )22span p pn n

Wψ − −=

( ){ } ( )33span p pn n

Wψ − −=


Relation to subband filters( ) ( )

( ) [ ] ( ) ( )

( )

[ ] ( )

[ ] ( ) ( )

1

0 1

10 0

0 00

linear combinationof scaling functions in

Since , recursive definition of scaling function

2 2

Orthonormality

n ,

nn n

n

V

V V

t g n t g n t nϕ ϕ ϕ

δ ϕ ϕ

−

−

∞ ∞−

=−∞ =−∞

⊂

= = −

=

∑ ∑

[ ] ( ) ( ) [ ] ( ) ( )

[ ] [ ] ( ) ( ) [ ] [ ][ ]0

1 10 0 2

1 10 0 0 0

,

k unit norm and orthogonalto its 2-translates: correspondsto synthesis lowpass filter oforthonormal subband tra

2 , 2

i j ni j

i ji j i

g

g i t g j t dt

g i g j n g i g i n

ϕ ϕ

ϕ ϕ

∞− −

+−∞

− −

⎛ ⎞= ⎜ ⎟

⎝ ⎠

= − = −

∑ ∑∫

∑ ∑

nsform

17


Wavelets from scaling functions( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

( ){ } ( ) ( )

( ) [ ] ( ) ( )

1

1

0 0 1

11

linear combinationof scaling functions

is orthogonal complement of in

and

Orthonormal wavelet basis for

p p p

p p p p p

n

nn

W V V

W V W V V

W V

t g n t

ψ

ψ ϕ

−

−

−

∞−

=−∞

⊥ ∪ =

⊂

= ∑

( )

[ ] ( )

[ ] ( ) ( )( ){ } ( ){ } ( )

1

1

11 0

0 0 1

in

2 2

Using conjugate quadrature high-pass synthesis filter

The mutually orthonormal

1 1

Easy

functions, and , together span .

to e

nn

n

n

n n n

V

g n t n

g n g n

Vϕ

ϕ

ψ

−

−

∞

∞

∈

+

∈

=−

= −

= − − −⎡ ⎤⎣ ⎦

∑

Z Z

( )( ){ } ( )2

,

xtend to dilated versions of to construct orthonormal wavelet basis

for .

mn n m

tψ

ψ∈Z

L


Calculating wavelet coefficients for a continuous signal

Signal synthesis by discrete filter bank

Signal analysis by analysis filters h0[k], h1[k]Discrete wavelet transform

( ) ( ) ( ) [ ] ( ) ( ) [ ] ( ) ( )

( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) [ ] ( )

( ) ( ) ( )

( ) [ ] ( )

( ) ( ) ( )1 1 1 1

0 0 0 0 00 0

1 1 1 1

0 1 1 1 10 1

Suppose continuous signal

Write as superposition of and

nn n

n ni j

x t V w t W

x t y n t n y n V

x t V w t W

x t y i y j

ϕ ϕ

ϕ ψ

∈ ∈

∈ ∈

∈ ∈

= − = ∈

∈ ∈

= +

∑ ∑

∑ ∑

Z Z

Z Z

( ) ( ) [ ] [ ] ( ) [ ] [ ]( ) [ ]00

0 1 10 0 1 1 2 2n

n i j

y n

y n g n i y j g n iϕ∈ ∈ ∈

⎛ ⎞= − + −⎜ ⎟

⎝ ⎠∑ ∑ ∑

Z Z Z

18


0h0g

1g

1h

0g

1g

0h

1h

Discrete Wavelet Transform

( ) [ ]00y n

( ) [ ]10y n

( ) [ ]11y n

( ) [ ]20y n

( ) [ ]21y n

( ) [ ]10y n

( ) [ ]00y n

Sampling( )x t

( )Interpolation

tϕ

( )x t


Different wavelets

Haar2/2 coeffs.

Daubechies8/8

Symlets8/8

Cohen-Daubechies-Feauveau17/11

[Gonzalez, Woods, 2001]

19


Daubechies orthonormal 8-tap filters



8-tap Symlets


20


Biorthogonal Cohen-Daubechies-Feauveau 17/11 wavelets



Wavelet compression results

Original512x512

8bpp

0.074 bpp 0.048 bpp

Errorimages

enlarged


21


Embedded zero-tree wavelet algorithmX X X XX X X XX X X XX X X X

X X X X

X

X

X X X X

X X X XX X X XX X X XX X X X

X

X X X X

X X X XX X X XX X X XX X X X

Idea: Conditional coding of all descendants (incl. children)Coefficient magnitude > threshold: significant coefficientsFour cases

ZTR: zero-tree, coefficient and all descendants are not significantIZ: coefficient is not significant, but some descendants are significantPOS: positive significantNEG: negative significant

„Parent“

„Children“

„Descendants“


Embedded zero-tree wavelet algorithm (cont.)

For the highest bands, ZTR and IZ symbols are merged into one symbol ZSuccessive approximation quantization and encoding

Initial „dominant“ pass• Set initial threshold T, determine significant coefficients• Arithmetic coding of symbols ZTR, IZ, POS, NEG

Subordinate pass• Refine magnitude of all coefficients found significant so far by one bit

(subdivide magnitude bin by two)• Arithmetic coding of sequence of zeros and ones.

Repeat dominant pass• Omit previously found significant coefficients• Decrease threshold by factor of 2, determine new significant

coefficients• Arithmetic coding of symbols ZTR, IZ, POS, NEG

Repeat subordinate and dominate passes, until bit budget is exhausted.

22


Embedded zero-tree wavelet algorithm (cont.)

Decoding: bitstream can be truncated to yield a coarser approximation: „embedded“ representationFurther details: J. M. Shapiro, „Embedded image coding using zerotrees of wavelet coefficients,“ IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3445-3462, December 1993.Enhancement SPIHT coder: A. Said, A., W. A. Pearlman, „A new, fast, and efficient image codec based on set partitioning in hierarchical trees,“ IEEE Transactions on Circuits and Systems for Video Technology, vol. 63 , pp. 243-250, June 1996.


Summary:multiresolution and subband coding

Resolution pyramids with subsampling 2:1 horizontally and verticallyPredictive pyramids: quantization error feedback („closed loop“)Transform pyramids: no quantization error feedback („open loop“)Pyramids: overcomplete representation of the imageCritically sampled subband decomposition: number of samples not increasedDiscrete Wavelet Transform = cascaded dyadic subband splitsQuadrature mirror filters and conjugate quadrature filters: aliasing cancellationLifting: powerful for implementation and wavelet constructionLifting allows reversible wavelet transform Zero-trees: exploit statistical dependencies across subbands

Pyramid coding and subband codingee290t/sp04/lectures/lec5.pdf · Wavelet bases ( )() ()() () 2 2 2...

Documents

Transcript of Pyramid coding and subband codingee290t/sp04/lectures/lec5.pdf · Wavelet bases ( )() ()() () 2 2 2...