ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
ESE250 – Week 5
Nyquist-Shannon Theorem
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem2
Course Map
Numbers correspond to course weeks
Today
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Where are we ?
Week 2: SamplingReceived signal is sampled & quantized q = PCM[ r ] Week 3 : CompressionQuantized Signal is Coded c =code[ q ] Week 4 : Frequency DomainSampled signal first transformed into frequency domain Q = DFT[ q ]Week 5signal oversampled & low pass filteredQ = LPF[ DFT(q+n) ]Week 6Transformed signal analyzed Using human psychoaoustic modelsWeek 7Acoustically Interesting signal is “perceptually coded”C = MP3[ Q]
OverSample
DFT LPF
DecodeProduce
r(t)
p(t)
q + n
CPerceptual
CodingStore /
Transmit
Q + N Q
Week 4 Week 5 Week 3
[Painter & Spanias. Proc.IEEE, 88(4):451–512, 2000]
3
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
ReconstructionGeneric Digital Signal Processor
Sampler(t)q
Codec
Store/Transmit Decode Produce p(t)
4
● Acquisition side: we need to:● Sample● Then process● Then store discretely● Then transmit.
● On the other end:● How should the signal be reproduced?● How fast do we send information to sound
card?● What functions do we use to interpolate
samples?● Last week: Harmonic Reconstruction
● Gets better with more terms● But need more samples to compute them● And error seems unpredictable
● This week: General reconstruction● Make some assumptions about signal● That allow us to exactly reconstruct from
samples
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Bohemian Rhapsody
● Versions sampled at different sampling rates
● 400Hz, then 1kHz, 2kHz, 3kHz.... 44kHz.● How can we describe the distortion?
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Shannon's Theorem
● Given a signal whose frequency does not exceed f
max
● We can perfectly reconstruct the signal
● From samples taken once every Ts seconds
● If 2 x fmax
≤ 1/(Ts)
the “Nyquist Rate”
● More simply: 2 x fmax
≤ fN
- the “Nyquist Frequency”
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Example 1● Consider a pure sine wave, sin(2πft), where f = 5Hz● Only one frequency component, so
fmax
= f
● Let's sample at 25Hz.
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Example 1● Let's sample at 25Hz.
– 25Hz is greater than 2 x 5Hz, so should be able to perfectly reconstruct signal
– Connect the dots – looks good!
– Now let's sample at 10Hz.
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Example 1● Let's sample at 10Hz.
– 10Hz is equal to 2 x 5Hz, so should still be able to perfectly reconstruct signal
– Connect the dots – looks good!
– Now let's sample at 6.25Hz.
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Example 1● Let's sample at 6.25Hz.
– 6.25Hz is less than 2 x 5Hz, so we should expect some problems.
– Connect the dots – what do we see?
– We'll return to this later in the lecture
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
“Connect-The-Dots?”● We're actually doing “pulse shaping”
– Formally: use Δ – functions or sinc functions
– Practically: use pulses of height (sample) and width Ts.
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Shannon's Theorem
● Given a signal whose frequency does not exceed f
max
● We can perfectly reconstruct the signal
● From samples taken once every Ts seconds
● If 2 x fmax
≤ 1/(Ts)
the “Nyquist Rate”
● More simply: 2 x fmax
≤ fN
- the “Nyquist Frequency”
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Question● Imagine we have a signal with many harmonics● DFT will yield a large number of frequencies● For perfect reconstruction, we need to store
– the amplitude
– of each frequency
– at each sample point
● OR we could just sample at 2fmax
and store
– ONE amplitude
– per sample point
● Why do we need DFT at all?
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Noise
● In reality we receive a mixed signal r(t):
r(t) = q(t) + n(t)
q(t) = signal of auditory interest
n(t) = noise (no information)
– Typically high-frequency– From recording, transmission, background
sounds● We only get to sample r(t), not q(t)!
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Example 2● Consider signal r(t):
r(t) = q(t) + n(t)– = 0 (silence)– = noise (33kHz and above)– Sample r @ 22kHz
● As in Example 1,– Higher-frequency noise
– Can alias down into the recorded signal
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Aliasing● Widely known problem● “Folding” - a manifestation
of aliasing:● “Wagon wheel” effect
● Take a signal s = sin(2π(f+δ)t)● If we sample at f● The aliased noise will be sin(2πδt)● “Folds over” as if was actually a
lower frequency.
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Example 1 Revisited● 5Hz Signal sampled at 6.25Hz.
– Folding predicts an aliased frequency of (10-6.25) = 3.75Hz
– Is this what we observe?
● Bottom line: Aliasing is undesirable.
● How do we prevent it?
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Anti-Aliasing● Given the “mixed” signal
r(t) = q(t) + n(t)
– q(t) = Auditory signal, fmax
= 22kHz
– n(t) = High-frequency noise
● We require a process to filter out the noise:– Some sort of “anti-aliasing” technique
– To “smooth away” the noise
– (which would otherwise appear in our samples)
– With some determined cut-off frequency between noise and signal
● How can we do this?
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Interlude: Visual Aliasing
● Strobe light: making water run uphill
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Shannon's Theorem
● Given a signal whose frequency does not exceed f
max
● We can perfectly reconstruct the signal
● From samples taken once every Ts seconds
● If 2 x fmax
≤ 1/(Ts)
the “Nyquist Rate”
● More simply: 2 x fmax
≤ fN
- the “Nyquist Frequency”
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Fourier's Theorem (review)
● Given a “nice” periodic signal,● We can perfectly re-construct the signal● From just its harmonics
– Lecture 4
– Lab 4 (Synthesizer)
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Clipping a non-periodic signal
● Consider a non-periodic signal
● Look at just a finite time window
● The signal is “clipped.”
t + Ts
t
t
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Clipping a non-periodic signal● Fourier Analysis gives us
the frequency content of our clipped signal
● Can use this to – perfectly reconstruct– the periodic extension
– Of the clipped signal
t
t + Ts
t
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Fourier + Shannon
● Fourier:– Over a finite time window
– We can construct an exact harmonic representation
● Shannon:– Over a finite range of
frequencies
– We can construct an exact representation in time
Clipped frequency forces finite harmonic series
Clipped time forces finite samples
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Fourier + Shannon
● Summary:
– If we have a finite sampling window T0
– And finite bandwidth fmax
– We are guaranteed a finite, exact harmonic reconstruction
● We can now solve the anti-aliasing problem
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Anti-Aliasing Revisited
● Given the “mixed” signal
r(t) = q(t) + n(t)
– q(t) = Auditory signal, fmax
= 22kHz
– n(t) = High-frequency noise
● We need to remove n(t) to be left with just the audible signal q(t).
● If we remove n(t), then no high-frequency noise to alias into our sample!
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Q + N = DFT(q) + DFT(n) = DFT(q+n)
● We can filter out the noise thus:
– Take the received time-sampled signal r(t)– Fourier: DFT(r):
● Finite sample period fmax
● So can find frequency-domain representation
– Assume q ranges from 0-fmax
Hz, n from fmax
-∞ Hz
● Can split DFT into terms < fmax
, and terms ≥ fmax
● i.e. DFT(q+n) = DFT(q) + DFT(n)
– Throw out DFT(n) terms
– Perform inverse DFT on DFT(q)
– Shannon: can now perfectly reconstruct q(t)!
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Oversampled Pre-Filter● Anti-Aliasing requires
“oversampling”
– fs > 2 x maximum noise frequency
● How do we know this maximum noise frequency ?– Microphone is band-limited; has
its own cut-off frequency– Might also add a “low-pass filter”
Low-Pass Filter
Filter Transformr Q q
Pre-Samplingr Low-Pass
ADXL001 MEMS accelerometer
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Open Questions
● How do we decide on fmax
?
– At what frequency do we distinguish between signal and noise?
● Why do CDs sample at 44.1kHz?– What is the significance of this number?
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
Big Ideas
● In order to perfectly sample a signal,
2 x fmax
≤ fs
● Sampling at too low a frequency will cause aliasing– which is detrimental to signal quality
● In order to prevent high-frequency noise from aliasing down into our signal– we must remove it with a low-pass filter
ESE250 S'13: DeHon, Kadric, Kod, Wilson-Shah Week 5 – Nyquist-Shannon theorem
ESE250 – Week 5
Nyquist-Shannon Theorem
Top Related