Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at...

Simple and Practical Algorithm for the SparseFourier Transform

Haitham Hassanieh Piotr Indyk Dina Katabi Eric Price

MIT

2012-01-19

Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 1 / 19

Outline

1 Introduction

2 Algorithm

3 Experiments


The Dicrete Fourier Transform

Discrete Fourier transform: given x ∈ Cn, find

xi =∑

xjωij

Fundamental toolI Compression (audio, image, video)I Signal processingI Data analysisI ...

FFT: O(n log n) time.


Sparse Fourier Transform

Often the Fourier transform is dominated by a small number of“peaks”

I Precisely the reason to use for compression.

If most of mass in k locations, can we compute FFT faster?


Previous work

Boolean cube: [KM92], [GL89]. What about C?[Mansour-92]: kc logc n.Long list of other work [GGIMS02, AGS03, Iwen10, Aka10]Fastest is [Gilbert-Muthukrishnan-Strauss-05]: k log4 n.

I All have poor constants, many logs.I Need n/k > 40,000 or ω(log3 n) to beat FFTW.I Our goal: beat FFTW for smaller n/k in theory and practice.I Result: n/k > 2,000 or ω(log n) to beat FFTW.


Our result

Simple, practical algorithm with good constants.Compute the k -sparse Fourier transform in O(

√kn log3/2 n) time.

Get x ′ with approximation error

‖x ′ − x‖2∞ ≤1k‖x − xk‖22

If x is sparse, recover it exactly.Caveats:

I Additional ‖x‖2/nΘ(1) error.I n must be a power of 2.


Structure of this section

If x is k -sparse with known support S, find xS exactly inO(k log2 n) time.

In general, estimate x approximately in O(√

nk) time.


IntuitionOriginal signal (time) Original signal (freq)

Cutoff signal (time) Cutoff signal (freq)

Cutoff signal (time) Cutoff signal, subsampled (freq)


n-dimensional DFT

n-dimensional DFTof first B terms.

B-dimensional DFTof first B terms.

FrameworkCutoff signal (time) Cutoff subsampled signals (freq)

“Hashes” into B buckets in B log B time.Issues:

I “Hashing” needs a random hash functionF Access x ′t = ω−atxσt , so x ′t = xσ−1t+a [GMS-05]

I CollisionsF Have B > 4k , repeat O(log n) times and take median. [Count-Sketch,

CCF02]I LeakageI Finding the support. [Porat-Strauss-12], talk at 9:45am.


LeakageCutoff signal (time) Cutoff, subsampled signals (freq)

Let Fi =

{1 i < B0 otherwise

be the “boxcar” filter. (Used in

[GGIMS02,GMS05])Observe

DFT(F ·x ,B) = subsample(DFT(F ·x ,n),B) = subsample(F∗x ,B).

DFT F of boxcar filter is sinc, decays as 1/i .Need a better filter F !


Filters

0 20 40 60 80 1000.0

0.5

1.0

1.5

2.0 Filter (time)

Bin0

5

10

15

20

25 Filter (freq)

Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :

I supp(F ) ∈ [0,B]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].

Gaussians:I Standard deviation σ = B/

√log n

B ·√

log n

I DFT has σ = (n/B)√

log n

(n/B)/√

log n

I Nontrivial leakage into O(log n) buckets.I But likely trivial contribution to correct bucket.


Filters

0 20 40 60 80 1000.0

0.2

0.4

0.6

0.8

1.0 Filter (time)

Bin0

2

4

6

8

10

12

14

16

18 Filter (freq)


I supp(F ) ∈ [0,B]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].

Gaussians:I Standard deviation σ = B/

√log n

B ·√

log n

I DFT has σ = (n/B)√

log n

(n/B)/√

log n

I Nontrivial leakage into O(log n) buckets.I But likely trivial contribution to correct bucket.


Filters

0 50 100 150 2000.0

0.2

0.4

0.6

0.8

1.0 Filter (time)

Bin0

10

20

30

40

50

60

70

80 Filter (freq)


I supp(F ) ∈ [0,B log n]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].

Gaussians:I Standard deviation σ =��B/

√log n B ·

√log n

I DFT has σ =��(n/B)

√log n (n/B)/

√log n

I Nontrivial leakage into 0 buckets.I But likely trivial contribution to correct bucket.


Filters

0 50 100 150 2000.0

0.2

0.4

0.6

0.8

1.0 Filter (time)

Bin0

10

20

30

40

50

60

70

80 Filter (freq)

Let G be Gaussian with σ = B√

log nH be box-car filter of length n/B.

Use F = G ∗ H.I F = G · H, so supp(F ) ⊂ [0,B log n].I |F | < 1/nΘ(1) outside −n/B,n/B.I |F | = 1± 1/nΘ(1) within n/2B,n/B.

Hashes correctly to one bucket, leaks to at most 1 bucket.Replace Gaussians with “Dolph-Chebyshev window functions”:factor 2 improvement.


Filters

0 50 100 150 2000.02

0.00

0.02

0.04

0.06

0.08

0.10 Filter (time)

Bin0.0

0.2

0.4

0.6

0.8

1.0

1.2 Filter (freq)

Let G be Gaussian with σ = B√

log nH be box-car filter of length n/B.

Use F = G ∗ H.I F = G · H, so supp(F ) ⊂ [0,B log n].I |F | < 1/nΘ(1) outside −n/B,n/B.I |F | = 1± 1/nΘ(1) within n/2B,n/B.

Hashes correctly to one bucket, leaks to at most 1 bucket.Replace Gaussians with “Dolph-Chebyshev window functions”:factor 2 improvement.


Algorithm to estimate xS

For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.

To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total

(nkB

+ B log n) log n =√

nk log3/2 n

time.


Algorithm in general

For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.

To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total

(nkB


nk log3/2 n

time.



For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.To find S: choose all that map to the top 2k values.

nk/B candidates to update at each iteration: total

(nkB


nk log3/2 n

time.



For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total

(nkB


nk log3/2 n

time.


Empirical Performance: runtime

0.001

0.01

0.1

1

10

214

215

216

217

218

219

220

221

222

223

224

225

226

Ru

n T

ime

(se

c)

Signal Size (n)

Run Time vs Signal Size (k=50)

sFFT 1.0

sFFT 2.0

FFTW

FFTW OPT

AAFFT 0.9

0.01

0.1

1

10

100 1000

Ru

n T

ime

(se

c)

Number of Non-Zero Frequencies (k)

Run Time vs Sparsity (n=222

)

sFFT 1.0

sFFT 2.0

FFTW

FFTW OPT

AAFFT 0.9

Compare to FFTW, previous best sublinear algorithm (AAFFT).

Offer a heuristic that improves time to O(n1/3k2/3).I Filter from [Mansour ’92].I Can’t rerandomize, might miss elements.

Faster than FFTW for n/k > 2,000.Faster than AAFFT for n/k < 1,000,000.


Empirical Performance: noise

1e-07

1e-06

1e-05

0.0001

0.001

0.01

0.1

1

-20 0 20 40 60 80 100

Ave

rage

L1

Err

or p

er E

nrty

SNR (dB)

Robustness vs SNR (n=222, k=50)

sFFT 1.0

sFFT 2.0

AAFFT 0.9

Just like in Count-Sketch, algorithm is noise tolerant.


Conclusions

Roughly: fastest algorithm for n/k ∈ [2× 103,106].Recent improvements [HIKP12b?]

I O(k log n) for exactly sparse xI O(k log n

k log n) for approximation.I Beats FFTW for n/k > 400 (in the exact case).


Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at...

Documents

Transcript of Simple and Practical Algorithm for the Sparse Fourier ...Hashes correctly to one bucket, leaks to at...