Wavelets and Signal Processing: A Match Made in Heaven

Martin Vetterli, EPFL, 23.01.2015with Y.Barbotin, T.Blu, P.Marziliano, H.Pan, R.Parhizkar

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

2

Outline

• A Tale of Two Communities:

– Signal processors

– Harmonic analysts

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions3

Speech Processing: Subband Coding

This looks quite boring…

4

Claude Galand at Work!

5

Time Frequency Methods in Signal Processing

• Crochiere, Esteban, Galand 1976

• Short-time Fourier transform

• Perfect reconstruction filter banks

• Transform coding and KLT

• Subband speech and image coding

6

Harmonic Analysis: Beautiful and…Esoteric ?

Weierstrass function (1872)7

Harmonic Analysis: Haar and Fourier bases

• Heisenberg uncertainty, Gabor expansion

• Balian-Low

8

The Meeting of the Minds

• Compactly supported wavelets: Ingrid Daubechies

• Multiresolution analysis: Stéphane Mallat, Yves Meyer

• Many Contributors: Stromberg, Lemarié, Battle, Cohen, ….

• Local cosine bases: Coifman, Meyer, Malvar

9

The Meeting of the Minds

Strömberg wavelet (1983)10

Algorithms• Wavelets based on filter banks

• Orthogonal • Biorthogonal • Multidimensional

• Wavelets packets• Adaptive bases

• Mallat’s algorithm

Roy Lichtenstein: Magnifying Glass, 1963

11

Outline

• A Tale of Two Communities

• The Golden Age:

– When Wavelets Were Going to Cure Cancer

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

12

Wavelets are looking for applications…… but applications were not waiting for wavelets!

• Wavelets are beautiful• They don’t have to be necessary useful!

• Wavelets create a framework• Many disparate constructions have a common interpretation

• Wavelets and time-frequency-scale is a way of thinking about problems

13

A few stories

• Bell Laboratories Murray Hill• Is a Daubechies’ filter a filter?

• New York Times Science Section• Image compression will be improved, maybe a hundred fold!

• Wright Patterson Airforce Base, Ohio• Speech compression will be improved 10 times!

14

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast– The Story of JPEG 2000

– Contributions of Wavelets

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

15

JPEG vs JPEG 2000

JPEG• Based on KLT• Theory from 60’s• Block based• Fast DCT

JPEG 2000• Based on wavelets• Theory from 80’s• No blocks• Fast WT

16

And the Winner is…

• JPEG is used in 98% of the cases• Mobile phones, digital

cameras

• JPEG 2000• Used in frame based digital

cinema

• Improvement of 1-2 dB does not justify change of standards

• Patent situation is murky…

From wikipedia page of JPEG2000: « However, the JPEG committee has acknowledged that undeclared submarine patents may still present a hazard. »

17

An example patent…

• Goupillaud, Morlet and Grossman patent

18

Main Contributions of Wavelets to SP:

Use of more general norms (getting rid of the tyranny of SNR ;)

Simple and powerful non-linear approximation for piecewise smooth functions

19

Thus: a piecewise smooth signal expands as:

• phase changes randomize signs, but not decay• a singularity influence only L wavelets at each scale• wavelet coefficients decay fast

Applications: Denoising

original

wavelet13.8 dB

noisy

countourlets15.4 dB

21

Lessons learned for signal processing

There is life beyond SNR

• Other norms are key (l1, TV, Sobolev)

There are exotic spaces that are actually useful

• Besov spaces

Sparsity is a key principle

• It helps in more ways than we thought

• It is critical for inverse problems

• It helps regularization

Pseudo diagonalization

…

and all this with solid theory and efficient algorithms!22

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as Way of Life

– Maximally compact sequences

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

23

Time-Frequency Tilings

Fourier

Wavelets

24

Maximally compact sequencesContinuous time

• Time and frequency spreads: 2nd moments• Heisenberg uncertainty bound• Gaussians are maximally compact

Discrete time• Time and frequency spreads?• Uncertainty bound?• What are the minimizers?

25

Localization for Discrete Sequences

• Option 1: Extension from analog signals

The signal is periodic in the freq. domain but the frequency definitions are not!

• Option 2: Use the first trigonometric moment (circular statistics)

Localization for Discrete Sequences

• Option 1: Extension from analog signals

Uncertainty Principle for Discrete Sequences

Heisenberg: For sequences with :

Maximally Compact Sequences

• The most compact sequence in time for a given frequency spread:

Theorem: For finding maximally compact sequences, solve the SDP:

The solution to above SDP is rank-1 and decomposed to .

Maximally Compact Sequences, Example

• Example:

The image part with relationship ID rId2 was not found in the file.

Theorem: Fourier transform of maximally compact sequences are Mathieu functions.

New Uncertainty Bounds for Sequences

There is a gap!

A New Benchmark

Uncertainty Principle for sequences

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

– Sampling 2.0

• A Community of Interest and an Interesting Community

• Conclusions

36

Given a class of objects, like a class of functions (e.g. bandlimited, SISS)And given a sampling device, as usual to acquire the real world– Smoothing kernel or low pass filter– Regular, uniform sampling

Obvious question:When is there a countable representation?When does a minimum number of samples uniquely specify the function?

sampling kernel

An example from my garden...The sampling question:

37

From Analog to Digital… and back!

38

Shannon’s Theorem… a Bit of History

Whittaker

Nyquist

Kotelnikov

Whittaker

Raabe

Gabor

Shannon

19151928

1933

1935

1938

1946

1948

1949

Someya

39

Classic Case: Subspaces

Shannon bandlimited case

or 1/T degrees of freedom per unit time

But: a single discontinuity, and no more sampling theorem…

Are there other signals with finite number of degrees of freedom per unit of time that allow exact sampling results?

40

Examples of Non-bandlimited Signals

41

Classic Cases and Beyond…

Sampling theory beyond Shannon?– Shannon: bandlimitedness is sufficient but not necessary– Shannon bandwidth and shift-invariant subspaces: dimension of subspaceIs there a sampling theory beyond subspaces?– Finite rate of innovation: Similar to Shannon rate of information– Non-linear set up– Position information is key!Thus, develop a sampling theory for classes of

non-bandlimited but sparse signals!

t

x(t)

t0x1

t1

x0…

xN-1

tN-1

Generic, continuous-time sparse signal 42

Signals with Finite Rate of Innovation

The set up:

For a sparse input, like a weighted sum of Diracs

– One-to-one map yn ⇔ x(t)?

– Efficient algorithm?

– Stable reconstruction?

– Robustness to noise?

– Optimality of recovery?

sampling kernel

43

The sampling theorem (VMB02)

For the class of periodic FRI signals which includes– Sequences of Diracs– Non-uniform or free knot splines– Piecewise polynomials– …

There are sampling schemes with sampling at the rate of innovation with perfect recovery and polynomial complexity

Variations: finite length, 2D, local kernels etc

44

What’s Maybe Surprising….

Bandlimited

Manifold

45

Current challenges…

The Tyranny of the pixel!

The

Viol

in, F

elix

Val

lott

on

46

The good old super-resolution problem….

Up-sampled image with mask regularizerUp-sampled image without mask regularizer 47

The irony of it all…

As a signal processor:• I went from l2(Z) to L2(R)Meanwhile:• Compressed sensing. • Beautiful theory of sparsity in RN!

48

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

49

The Wavelet and SP crowd

I.Daubechies, Y.Meyer, R.Coifman, D.Donoho, A.Cohen, S.Mallat, M.Unser, R.Malvar,M.Smith, P.P.Vaidyanathan, A.Aldroubi, H.G.Feichtinger, K.H.Groechenig and many others!

50

My Wavelet and SP gang

T. Blu, O. Rioul, C. Herley, J. Kovacevik, K. Ramchandran, T. Nguyen, A. Ortega, V. Goyal, R. Parhizkar, P. Marziliano, Y. Barbotin, H.Pan, and many more…. 51

• Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions: Foundations of Signal Processing

52

It has changed my view of the (signal processing) world!

Wavelets brought a new understanding of known methods

«Understanding is a lot like sex. It's got a practical purpose, but that's not why people do it normally». Frank Oppenheimer

Wavelets raised high expectations and produced new methods and successes

«As new developments emerge in any field, itsimportant to let them mature withoutgenerating unrealistic expectations so that the beautiful and important aspects have time and good soil in which to blossom”. Al Oppenheim

53

It has changed my view of the (signal processing) world!

54

55

56