Sound SynthesisPart I: Introduction & Fundamentals

Nicolas Pugeaultn.pugeault@surrey.ac.ukhttp://info.ee.surrey.ac.uk/Personal/N.Pugeault/IntroductionInstruments can be made in a variety of ways: think guitar, piano, organs, etc. Use electronic devices to create sounds: synthesisers. Can either Recreate an existing timbre or something different.

IntroductionProducing a sound by sending an electrical signal to a speaker is trivial.The question is what are the relevant and desirable properties of the signal to ensure that the resulting sound is as desired (eg, similar to a real instrument). ApplicationsMusical instrumentsComputer gamesSound effects for filmsMultimedia, computer system soundsMobile phonesSpeech synthesisToysEffects

Sound Synthesis PlanSynthesis I: FundamentalsSynthesis II: AdditiveSynthesis III: Filtering and distortionSynthesis IV: Other approachesPost-processing, pitch correction (autotunes)Sound perceptionSound SynthesisPart I: Fundamentals

Nicolas Pugeaultn.pugeault@surrey.ac.ukhttp://info.ee.surrey.ac.uk/Personal/N.Pugeault/

Lecture PlanIntroduction to sound synthesisPerception of soundLoudnessPitchTimbreSound synthesis FundamentalsSummary

Sound (contd)Sound is a pattern of compression and depression of the airRecord it using microphonesPerceive it from our earsGenerate it by speaking or using speakers Energy per m2 decreases with the square of distance...

Sound is a waveformSound is a waveform, Can be reflected when hitting a non-transmissive surfaceIf the surface is flat, reflected in cohesive wayOtherwise depends on frequency and surface texture

Sound proof studio wall, for absorbing high frequencies

Attributes of soundThe simplest sound: Pure toneSinusoidal wave (440Hz)

Periode p=1/f0

Reminder: Fourier TransformIdea: All functions can be decomposed in a (possibly infinite) sum of sinusoidal functions of varying frequencies.Transforms a function from time domain to frequency domain.Eg, right, for a square wave.

First componentFirst two componentsFirst threecomponentsFirst fourcomponents

LoudnessOften measured in decibels (dB) R=20*log10(A/A0)A0 is a reference amplitude, often taken as the threshold of audibility. Logarithmic perception of loudness. A change in 6dB means a doubling of amplitude!Range of audibility: ~120dB (1 to 1million) Perception of LoudnessCorrelated with amplitude.Here: constant frequency (f0=440Hz) Varying amplitude (A = 0.2, 0.5 or 1.0)

Loudness (contd)HoweverPerception of Loudness is frequency dependent.Sound X and Y have the same amplitude, which is louder, X or Y?X (100 Hz, A=1)Y (3,500Hz, A=1)

Considering only amplitudes, sound Y should be the same loudness as sound X.However, Y is louder than X. Why?

Loudness (contd)Fletcher Munson (1933)Subjects listen to pure tonesVarious frequenciesamplitude inc. per 10dBRobinson & Dadson (1956) more accurateBasis for standard ISO-226Perceived Loudness (Phons)1 Phon = 1dB SPL @ 1kHzBritish Standard BS ISO 226 (2003) (source wikipedia)

Loudness (contd)There is a difference between sensory loudness and perceptual loudness! (Emmet, 1992)For the design of a synthesiser with large dynamical range, changing only amplitude is a poor choice since signal may clip.Solution: use spectral variation: Broad spectrum will likely result in a loud sound. Narrow spectrum will be perceived as quiet. Perception of PitchFrequency correlated with pitchHere: 3 examples of pure tones.What if sounds are more complex?Range: 20Hz-20kHzBest acuity: 200Hz-2kHz

Pitch: Fundamental & HarmonicsReal sounds are not pure more complex!The ear assumes that multiple frequency components form one sound.Harmonically related -> fuse into single pitch at Fundamental Frequency (f0 largest common divisor)Each sinusoid is called a Harmonic partial of the sound (fk = N*f0)Fundamental & Harmonics (2)

Fundamental f0First harmonic f1 = 2*f0Second harmonic f2 = 3*f0Third harmonic f3 = 4*f0Seventh harmonic f7 = 7*f0...Fundamental & Harmonics (3)The pitch is correlated with the Fundamental frequency.

Although in this example the fundamental is missing, the pitch is the same. The timbre is different.

TimbreAll those sounds have the same pitch (A4, 440Hz)Flute A4 Tuning fork A4Violin A4Singer A4They differ in timbre.

Defining TimbreDefinition (American Standard Association):That attribute of sensation in terms of which a listener can judge that two sounds having the same loudnessandpitchare dissimilar.(ASA, 1960 ; Wikipedia, 2011) Has a wastebasket quality (Dixon Ward, 1965): What is neither loudness nor pitch...Synonyms: Tone quality or colour, texture...Affected by a sounds envelope.Timbre (contd)What physical parameters relate to timbre?Static spectrum (transient)Envelope of spectrum (transient)Dynamic spectrum (time-evolving)PhaseThis list is not exhaustive. cf wastebasket quality!Timbre: Static Spectrum(220Hz)

Timbre: Envelope of Spectrum

Timbre: Envelope (contd)Difference in envelope (same note, 440Hz fundamental)Top: FluteBottom: Violin Envelope differs!Conclusion: Envelope is instrument-specific.

FluteViolinTimbre: Envelope (contd)

Arrows indicate formants.This slide indicates two speech vowels (i and u)Formants not only determine timbre but helps distinguishing vowels.(used in speech recognition)Timbre: Dynamic SpectrumWill those two sounds have the same timbre?No, same average spectrum, but different timbre!Difference: Top: original soundBottom: time reversed.Conclusion: Temporal variation of spectrum impacts timbre!

AB

Timbre: Dynamic Spectrum (contd)

Timbre: spectrogram

Time (s.)Frequency (Hz)Timbre: Dynamic Spectrum (contd)This slides shows the long term (average) spectrum for two sounds (top: original and bottom: time reversed)Spectrum is identical; timbre is totally different very misleading!Conclusion: it is important to know how the spectrum evolves in time. The timbre does not only depends on the harmonic structure but on the way spectrum varies in time.

A) NormalB) Time reversed

Time envelope (ADSR)

Time Envelope (ADSR)Attack is the time from nil to peak. Decay is the time from peak to the sustain level. Sustain is the level during the main sequence of the sounds duration, until key is released. Release is the time to decay from sustain level to zero.Time Envelope (example)

Time Envelope (example)Example of the same sound with and without attackAttack cut at 0.7s. With (blue+green): Without (green):

Timbre: Phase?Sound A

Sound B

Are A and B of different timbres?

Timbre: Phase?Timbre depends (weakly) on phase relationship between harmonics.BUT waveforms are totally different, magnitude spectra identical, and timbre are (almost) identical!Conclusion: Human hearing is not sensitive to phase differences.

Sound A: Square wave, fundamental 500Hz, 9 harmonics.

Sound B: Square wave, fundamental 500Hz, 9 harmonics, every second harmonic phase shifted by 90 degrees.

Summary 1: Loudness ControlIn order to control loudness in synthetic sounds: Modify the spectral content:more energy at high frequency louder(see right).Modify the amplitudeHigher amplitude louder

Summary 2: Pitch ControlIn order to control pitch in synthetics sounds: Modify the fundamental frequency.High fundamental frequency high pitch.

Fundamental frequencySummary 3: Timbre ControlIn order to control timbre in synthetic sounds, modify Spectral contentSpectral envelopeSpectrum in timeSpectrum evolution during transient states

PlanIntroduction to sound synthesisPerception of soundLoudnessPitchTimbreSound synthesis FundamentalsSummaryFundamental DefinitionsComputer Instrument: An algorithm that realizes (performs) a musical event. Unit Generator: A high-level building block in an instrument.

OscillatorBasic waveform generatorAbbreviations:EG Envelope GeneratorLFO Low Frequency OscillatorVCA Voltage Control AmplifierVCF Voltage Control Filter

AbbreviationsEG Envelope GeneratorLFO Low Frequency OscillatorVCA Voltage Control AmplifierVCF Voltage Control FilterImportant TermsTwo types of synthesisersmonophonicpolyphonicYou can only play one note at a time. If you play several keys together, only one note will be generated no chords!You can play several notes at the same time can play chords!Types of synthesisPlanIntroduction to sound synthesisPerception of soundLoudnessPitchTimbreSound synthesis FundamentalsSummaryAdditional ReadingC. Dodge, C., & Jerse, T. A. (1997). Computer Music: Synthesis, Composition, and Performance. Schrimer, UK.(see chapters 2 and 4)