Parallel Fast Fourier Transform

Ryan Liu

IntroductionIntroductionThe Discrete Fourier Transform could be

applied in science and engineering.Examples:

◦Voice recognition◦Image processing

Discrete Fourier Transform (DFT): O(n2)Fast Fourier Transform (FFT): O(n log n)

2

Fourier AnalysisFourier AnalysisFourier analysis: Represent continuous functions by

potentially infinite series of sine and cosine functions.Fourier Series: A function can be expressed as the sum

of a series of sines and cosines:

3

Fourier TransformFourier TransformFourier Series can be generalized to derive the Fourier

TransformForward Fourier Transform:

Inverse Fourier Transform:

Note:

4

dkexfkF ikx2)()(

dkekFxf ikx2)()(

)sin()cos( xixexi

Fourier TransformFourier TransformFourier Transform maps a time series into the

series of frequencies that composed the time series.

Inverse Fourier Transform maps the series of frequencies back into the corresponding time series.

The two functions are inverses of each other.

5

Discrete Fourier TransformDiscrete Fourier Transform Maps a sequence over time to another sequence over frequency. The Discrete Fourier Transform (DFT):

k = 0, … N -1

n = 0, … N -1

Let k represent the discrete time signal, and Fn represent discrete frequency transform function.

6

1

0

2N

n

knN

i

nk efX

1

0

21 N

k

knN

i

kn eXN

f

Speech example of DFTSpeech example of DFT“Angora cats are furrier…”

7

Signal

Frequencyand amplitude

DFT ComputationDFT Computation n elements vector x. DFT matrix vector product Fnx

◦ fi,j = wnij for , j < n, and wn is the primitive nth root of unity.

8

)/2sin()/2cos(2

NiNew N

i

n

1

0

N

n

knNnk wfX

1

0

1 N

k

knNkn wX

Nf

i0

DFT Example1DFT Example1 DFT of vector (2,3)

◦ The primitive square root of unity for w2 is -1

◦ The inverse of DFT

9

1

5

3

2

11

11

1

0

112

012

102

002

x

x

ww

ww

3

2

1

5

11

11

2

1

1

0

112

012

102

002

x

x

ww

ww

DFT Example2DFT Example2 DFT of vector (1,2,4,3) The primitive 4th root of unity for w4 is i

10

i

i

ii

ii

x

x

x

x

wwww

wwww

wwww

wwww

3

0

3

10

3

4

2

1

11

1111

11

1111

3

2

1

0

94

64

34

04

64

44

24

04

34

24

14

04

04

04

04

04

DFT Example2DFT Example2 Inverse DFT

11

i

i

ii

ii

x

x

x

x

wwww

wwww

wwww

wwww

3

0

3

10

11

1111

11

1111

4

1

4

1

3

2

1

0

94

64

34

04

64

44

24

04

34

24

14

04

04

04

04

04

3

4

2

1

12

16

8

4

4

1

Fast Fourier TransformFast Fourier Transform DFT requires O(n2) time to process for n samples:

So, using DFT is not a best way in practice.

Fast Fourier Transform:

◦ Produces exactly the same result as the DFT.

◦ Time complexity O(n log(n)).

◦ divide-and-conquer strategy.

12

FFTFFT Recursively breaks down a DFT of any composite size N = N1N2

into many smaller DFTs of sizes N1 and N2, along with O(N) multiplication.

It’s to divide the transform into two pieces of size N/2 at each step.

13

Parallel FFTParallel FFT Algorithm: recursive

14

FFT(1,2,4,3)

FFT(1,4)

FFT(1)

FFT(2,3)

FFT(4) FFT(3)FFT(2)

Parallel FFTParallel FFT

15

Tracking the flow of data values.

Phases of Parallel FFTPhases of Parallel FFT Phase 1: Processes permute the input sequence.

Phase 2:

◦ First log n – log p iterations of FFT

◦ No message passing is required

Phase 3:

◦ Final log p iterations

◦ In each iteration every process swaps values with partner.

16

Time Complexity AnalysisTime Complexity Analysis FFT time complexity O(n log n).

Parallel FFT

◦ Each process controls n/p elements

◦ The overall communication time complexity is O( (n/p) log p)

◦ Computational time complexity of parallel FFT is O(n log n/p)

17

ReferenceReference Quinn,M.J (2004). Parallel programming in C with MPI and OpenMP Chu, E., & George. A., (2000). Serial and Parallel Fast Fourier Transform

Algorithms Bi,G & Zeng, Y. (2003). Transforms and Fast Algorithm for Signal

Analysis and Representation. The Fast Fourier Transform. (n.d.). Retrieved from

http://www.dspguide.com/ch12/2.htm Chu, E., & George, A., (1999). Inside the FFT black box: serial and

parallel fast Fourier transform algorithms. Boca Raton, Fla.: CRC Press, 1999.

18