Fast Fourier Transform - cs.upc.edujordicf/Teaching/AP2/pdf/14_FastFourierTransform.pdf ·...

Fast Fourier Transform

Jordi Cortadella and Jordi Petit

Department of Computer Science

Why Fourier Transform?

Divide & Conquer © Dept. CS, UPC 2

Polynomials: coefficient representation


Polynomials: point-value representation

• Fundamental Theorem (Gauss): A degree 𝑛 polynomial with complex coefficients has exactly 𝑛 complex roots.

• Corollary: A degree 𝑛 − 1 polymonial 𝐴(𝑥) is uniquely identified by its evaluation at 𝑛 distinct values of 𝑥.


Polynomials: point-value representation


Conversion between both representations


representation addition multiplication evaluation

coefficient O(𝑛) O(𝑛2) O(𝑛)

point-value O(𝑛) O(𝑛) O(𝑛2)

𝑎0, 𝑎1, ⋯ , 𝑎𝑛−1 𝑥0, 𝑦0 , ⋯ , 𝑥𝑛−1, 𝑦𝑛−1

evaluation

interpolationCoefficient representation Point-value representation

Could we have an efficient algorithm to move from coefficientto point-value representation and vice versa?

From coefficients to point-values


Evaluation by divide-and-conquer• Credits: based on the intuitive explanation by Dasgupta,

Papadimitriou and Vazinari, Algorithms, McGraw-Hill, 2008.

• We want to evaluate 𝐴(𝑥) at 𝑛 different points. Let us choose them to be positive-negative pairs: ±𝑥0, ±𝑥1, … , ±𝑥 Τ𝑛 2−1

• The computations for 𝐴(𝑥𝑖) and 𝐴(−𝑥𝑖) overlap a lot.

• Split the polynomial into odd and even powers

3 + 4𝑥 + 6𝑥2 + 2𝑥3 + 𝑥4 + 10𝑥5 = 3 + 6𝑥2 + 𝑥4 + 𝑥 4 + 2𝑥2 + 10𝑥4

• The terms in parenthesis are polynomials in 𝑥2:

𝐴 𝑥 = 𝐴𝑒 𝑥2 + 𝑥𝐴𝑜(𝑥2)


Evaluation by divide-and-conquer

• The calculations needed for 𝐴 𝑥𝑖 can be reused for computing 𝐴 −𝑥𝑖 .

𝐴 𝑥𝑖 = 𝐴𝑒 𝑥𝑖2 + 𝑥𝑖𝐴𝑜 𝑥𝑖

2

𝐴 −𝑥𝑖 = 𝐴𝑒 𝑥𝑖2 − 𝑥𝑖𝐴𝑜 𝑥𝑖

2

• Evaluating 𝐴 𝑥 at 𝑛 paired points

±𝑥0, ±𝑥1, … , ±𝑥 Τ𝑛 2−1

reduces to evaluating 𝐴𝑒 𝑥 and 𝐴𝑜(𝑥) at just𝑛/2 points: 𝑥0

2, ⋯ , 𝑥𝑛/2−12




+𝑥0 −𝑥0 +𝑥1 −𝑥1 +𝑥𝑛/2−1 −𝑥𝑛/2−1⋯Evaluate: 𝐴 𝑥degree ≤ 𝑛 − 1

Evaluate:𝐴𝑒 𝑥 and 𝐴𝑜(𝑥)degree ≤ 𝑛/2 − 1

𝑥02 𝑥1

2 𝑥𝑛/2−12⋯

𝑇 𝑛 = 2 ⋅ 𝑇 𝑛/2 + O 𝑛 = O(𝑛 log 𝑛)

If we could recurse, we would get a running time:

But can we recurse?



+𝑥0 −𝑥0 +𝑥1 −𝑥1 +𝑥𝑛/2−1 −𝑥𝑛/2−1⋯Evaluate: 𝐴 𝑥degree ≤ 𝑛 − 1

Evaluate:𝐴𝑒 𝑥 and 𝐴𝑜(𝑥)degree ≤ 𝑛/2 − 1

𝑥02 𝑥1

2 𝑥𝑛/2−12⋯

The problem: ? We need 𝑥02 and 𝑥1

2 to be a plus-minus pair.But a square cannot be negative !



+𝑥0 −𝑥0 +𝑥1 −𝑥1 +𝑥2 −𝑥2 +𝑥3 −𝑥3

𝑥02 𝑥1

2 𝑥22 𝑥3

2

𝑥04 𝑥2

4

𝑥08

+ 𝑖 − 𝑖

+1

+1 −1

+1 −1

+1 −1

−𝑖+𝑖

+𝑖 −𝑖 + −𝑖 − −𝑖

Note:

𝑖 = ±1

21 + 𝑖

−𝑖 = ±1

2(1 − 𝑖)

Complex numbers: review


𝑟

𝜃

𝑎

𝑏

Imaginary

Real

𝑧 = 𝑎 + 𝑏𝑖 𝑧 = 𝑟(cos 𝜃 + 𝑖 sin 𝜃) = 𝑟𝑒𝑖𝜃

Polar coordinates: (𝑟, 𝜃)

Length: 𝑟 = 𝑎2 + 𝑏2

Angle 𝜃 ∈ 0,2𝜋 : cos 𝜃 =𝑎

𝑟, sin 𝜃 =

𝑏

𝑟

𝜃 can always be reduced modulo 2𝜋

Some examples:

Number −1 𝑖 5 + 5𝑖

Polar coords (1, 𝜋) (1, 𝜋/2) (5 2, Τ𝜋 4)

Complex numbers: multiplication


(𝑟1, 𝜃1)

(𝑟2, 𝜃2)

(𝑟1𝑟2, 𝜃1 + 𝜃2)

𝑟1, 𝜃1 × 𝑟2, 𝜃2 = (𝑟1𝑟2, 𝜃1 + 𝜃2)

For any 𝑧 = 𝑟, 𝜃 :

−𝑧 = (𝑟, 𝜃 + 𝜋), since −1 = 1, 𝜋

If 𝑧 is on the unit circle, then 𝑧𝑛 = (1, 𝑛𝜃)

Complex numbers: the 𝑛th roots of unity


Angle 2𝜋/𝑛

4𝜋/𝑛

2𝜋

𝑛+ 𝜋

Solutions to the equation 𝑧𝑛 = 1(𝑛 = 16)

Solutions are 𝑧 = 1, 𝜃 ,for 𝜃 a multiple of 2𝜋/𝑛

All roots are plus-minus paired:

− 1, 𝜃 = (1, 𝜃 + 𝜋)

Divide-and-conquer step


Evaluate 𝐴(𝑥) at 𝑛th roots of unityEvaluate 𝐴𝑒(𝑥) and 𝐴𝑜 𝑥at Τ𝑛 2 nd roots of unity

Divide-and-conquer steps


Roots of unity for 𝑛 = 8


+ 𝑖

− 𝑖

+1−1

+𝑖

−𝑖

+ −𝑖

− −𝑖


function FFT(𝑨,𝝎)Inputs: 𝑨 = (𝒂𝟎, 𝒂𝟏, … , 𝒂𝒏−𝟏), for 𝒏 a power of 2

𝝎: A primitive 𝒏th root of unity

Output: 𝑨 𝟏 , 𝑨 𝝎 , 𝑨 𝝎𝟐 , … , 𝑨 𝝎𝒏−𝟏

if 𝝎=1: return 𝐴

𝑨𝒆 𝝎𝟎 , 𝑨𝒆 𝝎𝟐 , … , 𝑨𝒆(𝝎𝒏−𝟐) = FFT(𝑨𝒆, 𝝎

𝟐)

𝑨𝒐 𝝎𝟎 , 𝑨𝒐 𝝎𝟐 , … , 𝑨𝒐(𝝎𝒏−𝟐) = FFT(𝑨𝒐, 𝝎

𝟐)

for 𝒌 = 𝟎 to 𝒏 − 𝟏:𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐(𝝎

𝟐𝒌)

return 𝑨 𝟏 , 𝑨 𝝎 , 𝑨 𝝎𝟐 , … , 𝑨(𝝎𝒏−𝟏)


Fast Fourier Transform: example with 𝑛 = 4

function FFT((𝒂𝟎, 𝒂𝟏, 𝒂𝟐, 𝒂𝟑),𝝎)

𝑨𝒆(𝝎𝟎), 𝑨𝒆(𝝎

𝟐) = FFT( 𝒂𝟎, 𝒂𝟐 , 𝝎𝟐)

𝑨𝒐 𝝎𝟎 , 𝑨𝒐(𝝎𝟐) = FFT( 𝒂𝟏, 𝒂𝟑 , 𝝎𝟐)

𝑨 𝝎𝟎 = 𝑨𝒆 𝝎𝟎 +𝝎𝟎𝑨𝒐 𝝎𝟎

𝑨 𝝎𝟏 = 𝑨𝒆 𝝎𝟐 +𝝎𝟏𝑨𝒐 𝝎𝟐

𝑨 𝝎𝟐 = 𝑨𝒆 𝝎𝟒 +𝝎𝟐𝑨𝒐 𝝎𝟒 = 𝑨𝒆 𝝎𝟎 −𝝎𝟎𝑨𝒐 𝝎𝟎

𝑨 𝝎𝟑 = 𝑨𝒆 𝝎𝟔 +𝝎𝟑𝑨𝒐 𝝎𝟔 = 𝑨𝒆 𝝎𝟐 −𝝎𝟏𝑨𝒐 𝝎𝟐

return 𝑨 𝟏 , 𝑨 𝝎 ,𝑨 𝝎𝟐 , 𝑨(𝝎𝟑)


𝝎𝟎 = 𝟏𝝎𝟏 = 𝒊𝝎𝟐 = −𝟏𝝎𝟑 = −𝒊


function FFT(𝒂,𝝎)Inputs: 𝒂 = (𝒂𝟎, 𝒂𝟏, … , 𝒂𝒏−𝟏), for 𝒏 a power of 2

𝝎: A primitive 𝒏th root of unity

Output: 𝒂 𝟏 , 𝒂 𝝎 , 𝒂 𝝎𝟐 , … , 𝒂 𝝎𝒏−𝟏

if 𝝎=1: return a

𝒔𝟎, 𝒔𝟏, … , 𝒔 Τ𝒏 𝟐−𝟏 = FFT( 𝒂𝟎, 𝒂𝟐, … , 𝒂𝒏−𝟐 , 𝝎𝟐)

𝒔𝟎′ , 𝒔𝟏

′ , … , 𝒔 Τ𝒏 𝟐−𝟏′ = FFT( 𝒂𝟏, 𝒂𝟑, … , 𝒂𝒏−𝟏 , 𝝎𝟐)

for 𝒌 = 𝟎 to Τ𝒏 𝟐 − 𝟏:

𝒓𝒌 = 𝒔𝒌 +𝝎𝒌𝒔𝒌′

𝒓𝒌+ Τ𝒏 𝟐 = 𝒔𝒌 −𝝎𝒌𝒔𝒌′

return (𝒓𝟎, 𝒓𝟏, … , 𝒓𝒏−𝟏)


FFT: asymptotic complexity• The runtime of the FFT can be expressed as:

𝑇 𝑛 = 2 ⋅ 𝑇 Τ𝑛 2 + O 𝑛

• Using the master theorem we conclude:

Runtime FFT 𝑛 = O(𝑛 log 𝑛)

• Gilbert Strang (MIT, 1994):

“the most important numerical algorithm of our lifetime”.

• Reference: Cooley, James W., and John W. Tukey, 1965,“An algorithm for the machine calculation of complex Fourier series,” Math. Comput. 19: 297-301.


Unfolding the FFT


𝐹𝐹𝑇 Τ𝑛 2

𝑎0𝑎2

𝑎𝑛−2

⋮

𝐹𝐹𝑇 Τ𝑛 2

𝑎1𝑎3

𝑎𝑛−1

⋮

𝑟𝑘

𝑟𝑘+ Τ𝑛 2+

+

𝜔𝑘

𝜔𝑘+ Τ𝑛 2


Unfolding the FFT (butterfly diagram)𝑎0

𝑎4

𝑎2

𝑎6

𝑎1

𝑎5

𝑎3

𝑎7

𝐴(𝜔0)

𝐴(𝜔1)

𝐴(𝜔2)

𝐴(𝜔3)

𝐴(𝜔4)

𝐴(𝜔5)

𝐴(𝜔6)

𝐴(𝜔7)

4

4

4

4

4

4

2

2

6

6

4

2

3

7

6

5

1

000

001

010

011

100

101

110

111

000

100

010

110

001

101

011

111

Why is it called a butterfly diagram?







𝑎0, 𝑎1, ⋯ , 𝑎𝑛−1 𝑥0, 𝑦0 , ⋯ , 𝑥𝑛−1, 𝑦𝑛−1

evaluation


values = FFT( coefficients , 𝜔)

O(𝒏 𝐥𝐨𝐠𝒏)

From point-values to coefficients







𝑎0, 𝑎1, ⋯ , 𝑎𝑛−1 𝑥0, 𝑦0 , ⋯ , 𝑥𝑛−1, 𝑦𝑛−1

evaluation


values = FFT( coefficients , 𝜔)

coefficients =1

𝑛FFT( values , 𝜔−1)

O(𝒏 𝐥𝐨𝐠𝒏)

Polynomial multiplication

Input: Coefficients of two polynomials 𝐴(𝑥) and 𝐵(𝑥),of degree 𝑑𝐴 and 𝑑𝐵, respectively. Let 𝑑 = 𝑑𝐴 + 𝑑𝐵.

Output: The product 𝐶 = 𝐴 ⋅ 𝐵.

1. Selection: Pick 𝜔 = (1, Τ2𝜋 𝑛), such that 𝑛 ≥ 𝑑 + 1 and 𝑛 is a power of two.

2. Evaluation (FFT): Compute 𝐴 1 , 𝐴 𝜔 , 𝐴 𝜔2 , … , 𝐴 𝜔𝑛−1 . Compute 𝐵 1 , 𝐵 𝜔 , 𝐵 𝜔2 , … , 𝐵 𝜔𝑛−1 .

3. Multiplication: Compute 𝐶 𝜔𝑘 = 𝐴 𝜔𝑘 ⋅ 𝐵(𝜔𝑘), for all 𝑘 = 0,… , 𝑛 − 1.

4. Interpolation (inverse FFT): Recover 𝐶 𝑥 = 𝑐0 + 𝑐1𝑥 + 𝑐2𝑥

2 +⋯+ 𝑐𝑑𝑥𝑑.


FFT application in Signal Processing


Converting a signal: time domain frequency domain

Distinguishing instruments


Same note (frequency).Different timbre (spectral envelope).

Speech


Tone: distance between sidelobes (vocal cords).Sound: spectral envelope.

Spectrogram


Pronouncing “veintisiete”

Exercises

1. Consider the polynomials 1 + 𝑥 − 2𝑥2 + 𝑥3 and −1 + 𝑥2:– Choose an appropriate power of two to execute the FFT for the

polynomial multiplication. Find the value of 𝜔.

– Give the result of the FFT for 𝑥2 − 1(no need to execute the FFT).

2. Consider the polynomials −1 + 2𝑥 + 𝑥2 and 1 + 2𝑥:– Choose an appropriate power of two to execute the FFT.

Find the value of 𝜔.

– Calculate their point-value representation using the FFT (execute the FFT algorithm manually).

– Calculate the product of the point-value representations.

– Execute the inverse FFT to obtain the coefficients of the product.


Fast Fourier Transform - cs.upc.edujordicf/Teaching/AP2/pdf/14_FastFourierTransform.pdf ·...

Documents

Transcript of Fast Fourier Transform - cs.upc.edujordicf/Teaching/AP2/pdf/14_FastFourierTransform.pdf ·...