Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n...
-
Upload
teresa-preston -
Category
Documents
-
view
226 -
download
1
Transcript of Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n...
![Page 1: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/1.jpg)
Linear Algebra
Definitions Dot product of vectors A=<a1,a2,…an> and B=<b1,b2, …, bn>
A B = a1*b1 + a2*b2 + … + an * bn
The length of a vector <a1, a2, … , an> is (a12 + a2
2 + … + an2)½
A set of vectors are linearly independent if none of them can be expressed as a linear combination of the others
A set of n vectors is a basis for an n-dimensional space if they are all linearly independent
Vectors are orthogonal if their dot product = 0 (ex: <1/5½,2/5½ >, <-2/5½,1/5½ >) Two orthogonal vectors are orthonormal if their lengths are all unity
<1,0,0>
<0,1,0>
<0,0,1>
Interpretation1.We can represent any vector by a linear combination of an orthogonal set<1/5½,2/5½ > • <4,7> = 18/5½ and <-2/5½,1/5½>• <4,7> = -1/5½
(18/5½)<1/5½,2/5½ > + (-1/5½)<-2/5½,1/5½> = <20/5, 35/5> = <4,7>2.Orthogonal vectors extends the concept of: “perpendicular”
Extends Euclidean space algebra to higher dimensions
![Page 2: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/2.jpg)
Extend Linear Algebra to CalculusTwo functions, f(x) and g(x) and are orthogonal over the interval with weighting function w(x) if
If, in addition, f(x) ≠ g(x) ≠ 0 and w(x) ≠ 0
= 0 , and = 0
the functions f(x) and g(x) are said to be orthornormal
Interpretation: Sets of periodic functions are orthornormal
![Page 3: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/3.jpg)
Fourier Decomposition
• Fourier showed us that we can decompose any time domain signal into a (possibly infinite) series of waves described by a set of orthonormal functions
• Each wave has a well-defined frequency and can vary in amplitude
![Page 4: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/4.jpg)
Fourier Series• An infinite sum of sinusoids modeling a continuous signal is an
example of a Fourier Series• Each point on the Fourier number line represents a particular
sinusoid • Fourier transform: Decompose a signal into Fourier components,
perform processing, and recombine the results to solve an original problem
We use transforms to alter a problem that cannot be solved in one domain into another, where the solution is possible. Many DSP algorithms are able to compute useful results by analyzing a signals component frequencies
![Page 5: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/5.jpg)
Fourier Square Wave Synthesis
![Page 6: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/6.jpg)
Fourier DecompositionExample: decompose a signal into 9 cosine and sine waves
The decomposition models the signal’s phase and amplitude
![Page 7: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/7.jpg)
The Fourier Transform Family• Fourier Series
A possibly infinite decomposed weighted sum of sinusoidal functions that models a periodic signal
• Fourier TransformA linear operation that maps an arbitrary function with into a spectrum of its frequency components
• Discrete Fourier Transform (DFT)A Fourier Transform applied to a discrete periodic series of measurements
• Discrete Time Fourier Transform (DTFT)A Fourier Transform applied to a an aperiodic discrete series of measurements
• Fast Fourier Transform: Fast way to calculate DFT
![Page 8: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/8.jpg)
Orthonormal Sinusoids
![Page 9: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/9.jpg)
OrthonormalAttenuating/Amplifying Sinusoids
Parameters•Frequency of attenuation or amplification•Speed of attenuation or amplification
![Page 10: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/10.jpg)
Alternative Basis Functions
![Page 11: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/11.jpg)
Choosing a Basis Function Set1. Any orthonormal function set works, but all choices are not equal
2. A good choice:a.Has desirable mathematical characteristics. Those with sharp edges fail
b.Represents frequencies, amplitudes, phases. Pure sines and cosines fail
Integral: Cos (πx/3) * cos (2 πx/3)cos (πx/3) and cos (2 πx/3)
![Page 12: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/12.jpg)
Orthonormal Basis Sets (cont.)
• The winners: – Continuous version: e-j2πft = cos(2πft) – j sin(2πft) where t
is point in time, f is a frequency, polar notation models both phase angle and magnitude
– Discrete version: e-j2πfn/N = cos(2πfn/N) – j sin(2π fn/N) where n is a signal measurement, f is one of the N orthonormal functions
• Note: The angular frequency, ωm, measured in radians per second is defined as follows: ωm = 2πfF0 = 2πf/T0 measures how fast a particular basis function oscillates with respect to the fundamental frequency, F0, having period, T0. For every one time F0 rotates about the unit circle, 2πfF0
rotates f times faster.
![Page 13: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/13.jpg)
The Fourier Transform
![Page 14: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/14.jpg)
The Fourier Algorithm
• This Signal has– Two component frequencies: 1Khz, and 2Khz– The second signal has a 1350 phase difference from the first– We measure the signal at N (8) measurements per cycle (fs=8kHz)
• Fourier analysis– Multiply each measurement by the corresponding sin and cosine
values of e-2πfn/N and sum the results
– The resulting frequency array will have non zero elements at indices X[1] and X[2] as we would expect
– The phase angles are relative to the cosine of f0
• X[1] phase is -900 (-π/2) because sin(π/2) = cos(0) and cos(f0) leads by 900
• X[2] phase is +45 (π/4) because cos(2*f0) trails by that amount
• Negative frequencies X[6] and X[7] equal -X[2] and -X[1] respectively
Signal: x(t) = sin(2π*1000t) + 0.5 sin(2*2000πt+3π/4)
![Page 15: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/15.jpg)
Understanding Digital Signal Processing, Third Edition, Richard Lyons(0-13-261480-4) © Pearson Education, 2011.
![Page 16: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/16.jpg)
Understanding Digital Signal Processing, Third Edition, Richard Lyons(0-13-261480-4) © Pearson Education, 2011.
![Page 17: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/17.jpg)
Understanding Digital Signal Processing, Third Edition, Richard Lyons(0-13-261480-4) © Pearson Education, 2011.
![Page 18: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/18.jpg)
Second Example
• Comparisons with the first example– Same signal: x(t) = sin(2π*1000t) + 0.5 sin(2*2000πt+3π/4)
– Sampling starts three time units later (see above figure)
• Fourier results (We apply the same Fourier algorithm)
– The same frequency components have non zero magnitudes
– The cosine and sine sums are different, but the magnitudes are the same
– The phase of X[1] from -90 and +45; the third sample moves 1350 forward.
– The phase of X[2] goes from +45 to -45 because the third sample, because each sample is π /2 apart, and starting with the 3π/4, the third sample starts with 3 * (π /2) + 3π/4 = 9 π /4, π /2 ahead of the basis function
![Page 19: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/19.jpg)
Understanding Digital Signal Processing, Third Edition, Richard Lyons(0-13-261480-4) © Pearson Education, 2011.
![Page 20: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/20.jpg)
Example Resultsf First Example Second Example
M ϕ Real Imag M ϕ Real Imag
0 0 0 0 0 0 0 0 0
1 4 -90 0.000 -4.000
4 +45 2.8284
2.8284
2 2 +45 1.414 1.414 2 -45 1.4142
-1.4142
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0
6 2 -45 1.414 -1.414
2 +45 1.4142
1.4142
7 4 +90 0.000 4.000 4 -45 2.8284
-2.8284
NotesThe results are scaled by N/2 (4 + 2 + 2 +4 = 12
The frequency domain is symmetrical about point N/2X[0] = X[4] in this example, but not generally
![Page 21: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/21.jpg)
DFT Properties• The DFT is a linear transformation:
c1DFT(x1[n])+c2DFT(x2[n]) = DFT(c1x[n] + c2x2[n])
• The resulting magnitudes are amplified by N/2 (8/2 in this case). This means the 4 should be 1 and the 2 should be ½.
• A shift in the measurements correspond to a shift in the DFT phase calculations (as seen in the examples)
• The frequency domain is symmetric.
![Page 22: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/22.jpg)
DFT Symmetry• The resulting frequency domain is symmetric
– X[n-k] = X[n+k] when k ≠N/2 and k ≠0,
– The upper frequencies represent negative frequencies (the extension of the signal if moving back in time)
– X[0] represents zero DC bias, a non oscillating constant
– X[N/2] represents the component fs/2
Negative Frequencies Positive Frequencies
![Page 23: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/23.jpg)
DFT and its Inverse
• Definition: Continuous Fourier Transform and Inverse– Transform: X(w) = ∫-∞, ∞ x(t)e-iwtdt
– Inverse: x(t) = (1/2π)∫ -∞, ∞ X(w)eiwtdw
• Discrete Fourier Transform and Inverse– Transform: X[k] = ∑n=0,N-1 x[n] e-i2πkn/N
– Inverse: x[k] = (1/N)∑n=0,N-1 X[k] ei2 πkn/N
• Note: We can change the scale factor if we want the transform and its inverse to be completely symmetric
• Conclusion: We can use DFT to go back and forth between the time and frequency domains
![Page 24: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/24.jpg)
DFT Correlation Algorithmpublic double[] DFT( double[] time){ int N = time.length;
double fourier[] = new double[2*N], real, imag;for (int k=0; k<N; k++){ for (int i=0; i<N; i++) { real = Math.cos(2*Math.PI*k*i/N); imag = -Math.sin(2*Math.PI*k*i/N); fourier[2*k] += time[i]*real; fourier[2*k+1]+= time[i]*imag;} }return fourier;
}
Note: even indices = real part, odd indices = imaginary part
Complexity: O(N2) because of the double loop of N eachExample: For 512 samples, loops 262144 timesEvaluation: Too slow, but FFT is O(N lg N)
![Page 25: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/25.jpg)
Relation to Speech• For analysis we breakup
signal into overlapping windows
• Why? – Speech is quasi-periodic,
not periodic– Vocal musculature is
always changing– Within a small window of
time, we assume constancy, and process as if the signal were periodic
Typical Characteristics10-30 ms length1/3 to 1/2 overlap
![Page 26: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/26.jpg)
DFT Issues• Implementation difficulties
– Impact: Algorithm runs too slow, especially for real time processing– Solution: FFT algorithm
• Spectral Leakage: Occurs when a signal has frequencies that fall between those of the DFT basis functions– Result: Magnitude leaks over the entire spectrum– Solutions: Windowing and DFT padding
• Inadequate frequency resolution– Impact: DFT magnitudes represent wide ranges of frequencies– Solution: DFT zero padding
• Signal rough edges: – Impact: Inaccuracies because windows don’t have smooth transistions– Solution: Windowing
![Page 27: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/27.jpg)
DFT Zero Padding
• Each frequency domain bin represents a range of frequencies • Assume sampling rate fs = 16000, N = 512• Frequency domain 3, contains magnitudes for frequencies
between 16000/512*3 (93.75) to 16000/512*4 (125)• Each frequency bin models a range of 31.25 frequencies
• Padding the time domain frame with zeroes adds high frequency components, but does not negatively impact the result, or impact the lower frequencies of the spectrum
![Page 28: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/28.jpg)
DFT Zero Padding
![Page 29: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/29.jpg)
Understanding Digital Signal Processing, Third Edition, Richard Lyons(0-13-261480-4) © Pearson Education, 2011.
FFT zero padding can increase SNR
Definition: SNR = 10log( Psignal/Pnoise
![Page 30: Linear Algebra Definitions Dot product of vectors A= and B= A B = a 1 *b 1 + a 2 *b 2 + … + a n * b n The length of a vector is (a 1 2 + a 2 2 +](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649f315503460f94c4caab/html5/thumbnails/30.jpg)
Understanding Digital Signal Processing, Third Edition, Richard Lyons(0-13-261480-4) © Pearson Education, 2011.