Slide 1

Slide 2 Sound Synthesis Part IV: Subtractive & Granular synthesis, Physical modelling Slide 3 Plan Previously... Subtractive synthesis Granular Synthesis Analysis Physical modelling Summary Slide 4 Previously,... Sound is defined by: Loudness (amplitude + high freq. Comp.) Pitch (fundamental frequency) Timbre (all other freq. Comp + time envelope) Visible in a spectrogram: Slide 5 Types of synthesis Sound Synthesis Additive synthesis Distortion techniques Subtractive synthesis Granular synthesis Analysis based Physical modelling Slide 6 Simple Instrument Helmholtz model Waveform Constant frequency Envelope Envelope feeds varying amplitude to the oscillator. ASD Envelope AMP FREQ PHASE AMP ATTACK DURATION DECAY Slide 7 Additive Synthesis FREQ + Slide 8 Pros & cons Advantages Can reproduce a spectrogram bin by bin. Theoretically, can generate any sound High quality synthesis can be achieved Limitations Requires large amount of data for generation Sound analysis is only valid for a short range of pitch and loudness! Not intuitive to control Slide 9 Types of synthesis Sound Synthesis Additive synthesis Distortion techniques Subtractive synthesis Granular synthesis Analysis based Physical modelling Slide 10 Distortion (aka nonlinear) Synthesis Additive synthesis is powerful & flexible BUT requires many simple oscillators. Plethoric and non-intuitive controls on the sound spectrum. Using fewer oscillators, with more components. Complex spectra can be obtained by distorting simple oscillators. Slide 11 FM Synthesis: Instrument Design Envelopes functions F1 & F2 change the timbre of the sound (John Chowning, 1973) A) Bell-like timbre B) Wood drum sound C) Brass-like timbre fm + fc A fo F1 F2 1/d fm*I 1/d (John Chowning, 1973) Slide 12 Pros & cons Advantages Modulation index is a simple & effective way to control timbre. Computationally very efficient. Very easy to generate inharmonic spectra. Limitations Difficult to synthesise natural sounds. Slide 13 Waveshaping: Instrument Design Instrument for clarinet-like timbre From Jean-Claude Rissets Introductory Catalogue of Computer Synthesized Sounds. fm + ASD Envelope 256 F(x) * A 255 0.64 0.085 Slide 14 Pros & cons Advantages Distortion index: efficient way to control timbre Computationally very efficient Possible to compute transfer characteristic analytically. Limitations: Difficult to synthesise natural sounds. Slide 15 Types of synthesis Sound Synthesis Additive synthesis Distortion techniques Subtractive synthesis Granular synthesis Analysis based Physical modelling Slide 16 Subtractive Synthesis Idea: Start with a source with a broad spectrum (noise, pulse) and filter out the unwanted portions of the spectrum. Main sources: Noise Pulse generator SOURCE FILTER Slide 17 Sources: Pulse waveform Significant amplitude only for a short time (pulse width) Repeated periodically rich spectrum Spectrum depends on shape and width Narrow pulse high frequencies. Square, triangular, etc. But, band-limited is preferable! Slide 18 Source: Pulse Waveform (contd) Common band-limited pulse (BUZZ): Shape depends on ration f0/fs Harmonics of fundamental only up to Nyquist frequency (to avoid aliasing) Number of harmonics N = int(fs/2f0) Examples: f0=440Hz, fs=40kHz N = 45 F0=1046Hz, fs=40kHz N = 19 N FREQ AMP BUZZ Slide 19 Pulse Waveform (contd) Slide 20 Filters Chosen to carve out or reduce frequency components. Two zones: Pass Band: components are preserved Stop Band: components are reduced/removed And usually a smooth transition between them Examples: Low pass filters will preserve low frequencies High pass will preserve high frequencies (change fundamental!). Band pass select a frequency band... Slide 21 Filters: low & high pass Slide 22 Filters (contd) Definition: The cut-off frequency of a filter is where the power transmitted drops by -3dB. Band-pass filters are defined by A center frequency: fc A Bandwidth: BW OR Lower and upper cut-off frequencies: fl,fu The sharpness is called a Quality Factor: Q=f0/BW The opposite of a Band-pass is a Band-reject filter. Slide 23 Filters: Band-pass Slide 24 Timbre: Envelope [flashback] Arrows indicate formants. This slide indicates two speech vowels (i and u) Formants not only determine timbre but helps distinguishing vowels. (used in speech recognition) Slide 25 Pros & cons Advantages: Filtering allows to control formants naturally. Computationally efficient Can generate natural sounds (but requires skillful design!) Limitations Difficult to produce natural sounds! Instability and noise (when using analog implementations) Slide 26 Types of synthesis Sound Synthesis Additive synthesis Distortion techniques Subtractive synthesis Granular synthesis Analysis based Physical modelling Slide 27 Granular Synthesis Question: How can we decompose a sound into a number of micro-sounds, and then use them as fundamental building blocks in sound synthesis? Assumption: Every sound can be represented as a combination of audio grains (micro-sounds) distributed in time granular synthesis (Micro-sounds = grains = audio particles) Slide 28 How does it work? According to Dodge & Jersey (1997) Grains are the fundamental compositional elements to weave the sound Grains are defined as small bursts (typically 5-100 ms.) of sound energy encased in an envelope. Slide 29 Two categories GRANULAR SYNTHESIS SYNCHRONO US ASYNCHRONO US Grains triggered at timed interval (central clock) Sound with pitch Grains triggered at random intervals No pitch, perceived as cloud. Slide 30 A grain of Sound... A grain can be generated using a simple Helmholtz instrument. Very narrow time envelope - small d! f0 A 1/d ENVELOPE OSCILLATOR Slide 31 Synchronous Grain Synthesis In synchronous mode, grains are triggered at constant time intervals (with constant triggering frequency). Synthetic sounds can be characterized by a pitch sensation. The pitch corresponds to the triggering frequency not the frequency of the wave inside the grain! Slide 32 Synchronous a) Classical additive b) Granular synchronous Slide 33 Synchronous Grain Synthesis The spectral envelope of the waveform inside the grain determines the spectral envelope of the sound. Example: Repeating at 220Hz rate (A3) a grain enclosing a 3kHz sine wave yields a spectrum containing the harmonics of 220Hz Harmonics nearest 3kHz will receive most emphasis Increasing the rate to 261.6Hz (C4) (or any other frequency) changes the spacing between the harmonics, but the peak of the spectral envelope still appears at 3kHz. Synchronous Granular Synthesis imparts a fixed formant to a sound. Slide 34 Asynchronous Grain Synthesis In Asynchronous mode, grains are triggered with a random frequency, yielding a noise-like sound without any pitch. Sounds can be described as clouds, audio textures, audio wall papers All about timbre... Slide 35 Asynchronous a)Granular harmonic cloud (additive)b) Granular textural cloud Slide 36 Clouds are not all alike... Slide 37 Sound clouds (contd) t f t f A) cumulusB) stratus Slide 38 Properties of a sound cloud Sound clouds can be described by: Start time and duration Bandwidth of the cloud Amplitude envelope of the cloud Density of grains Grain envelope/duration Grain waveform Spatial dispersion Slide 39 Examples of sound clouds t f t f B) glissando t f B) stratus t f t f A) constant B) variableC) cumulus Slide 40 Grain waveform The Grain content can be seen like a colour it represents the timbre. Monochromatic Polychromatic Transchromatic Slide 41 Density of grains Density of grains = number of grains per second (stochastic!) Very sparse (eg. 47) Sparse (eg. 230) Dense (eg. 4700) Variable (eg. 2300) (sparse dense) Slide 42 Spatial Dispersion The way grains are spread in the cloud Deterministic Random Slide 43 Grain Envelope The way the grains envelopes and duration are distributed: Deterministic Random This affects loudness. Slide 44 Pros & cons Advantages Flexible sound processing capabilities Possible to synthesize interesting unnatural sounds (clouds) Possible to implement special effects (pitch shifting, morphing) Limitations Not always easy to control in a predictable way Can be computationally demanding Slide 45 Types of synthesis Sound Synthesis Additive synthesis Distortion techniques Subtractive synthesis Granular synthesis Analysis based Physical modelling Slide 46 Analysis-based Sound Synthesis Analysis of sound properties & psychoacoustics lead to sound synthesis techniques (eg, additive synth) What about speech? Pitch and timbre do not really cover the important properties of speech! Slide 47 Speech ?... One way to look at it: Speech is composed of: Phones (P): single sounds Diphones (D): pairs of two sounds (co- articulation) Usually, for languages |D|