Digital Sound for the Cinema - Università Iuav di Venezia · Digital Sound for the Cinema...
Transcript of Digital Sound for the Cinema - Università Iuav di Venezia · Digital Sound for the Cinema...
1
Digital Sound for the Cinema
Introduction to sound synthesisSandra Pauletto
Sandra’s background
• Lecturer in Sound Design at the University ofYork, UK
• Department of Theatre, Film and TV• Background in Physics, Music and Music
Technology• Research in sound design, auditory displays,
sonic interaction design• Born in Portogruaro, resident in the UK since
1996
2
Sound in cinema
• Production Sound– Location sound - dialogue, ambience,
atmospheres, sound effects, wild tracks– Multitrack digital recorder, files on hard
drive, automatic synchronisation viatimecode
• Post-Production Sound– ADR, Foley, digital sound design, editing
and mixing (digital processing)
Digital technologies in cinema
• Digital technologies have revolutionisedcinema sound
• Sound recordings processing and soundsynthesis can all be done digitally
• Sound software most used in the industry:– Avid ProTools +– Mixing console
3
Sound processing and sound synthesis infilm
• Realism: e.g. reverb, filtering (telephone, radio)• Continuity: e.g. EQ changing depending on camera angle• Emphasis: e.g. compression to give impression of
loudness (e.g. explosion)• Space: e.g. panning and reverb• Abstract sound effects: e.g. non-realistic effects in sci-fi
films• Synthesised sounds: e.g. alien sounds, film music• Soundtrack and recording restoration: e.g. denoise,
clicks and pops elimination
How can a computer create sounds?
Air pressure wavesAir pressure waves
Analogue electrical waves
4
Directly programming waves in the computerdigital sound synthesis
Synthesised sound can be very different from anatural or real sound
• Early 20th century:Luigi Russolo - Intonarumori
• In the mid 60s Robert Moog (http://www.moogmusic.com/)and Don Buchla (http://www.buchla.com) created firstanalogue synthesisers
• These use continuous electrical signals to create sound• 1970 MINIMOOG portable synthesiser used by
Genesis, Pink Floyd, etc. Produced until the 80s• Moog synth used in the soundtrack ofClockwork Orange by StanleyKubrick (composer Wendy Carlos)
A brief history: analogue synthesis and digitalsynthesis
5
• It can simulate analogue synthesis and also domore
• 1957 Max Mathews creates first digital synthesis:Newmann Guttman, Pitch Variations, 1957 in theBell Labs (New Jersey)
• Max Matthews created the first language forsound synthesis Music I
• Music II, III, IV, V, Common Lisp Music, CSound,etc. derived from that
Digital synthesis
6
• Only when the processors were powerful enough digitalsynthesis became a viable option
• mid 70s, the Synclavier was one of the first commercialdigital synthesiser. Used by Gary Rydstrom (Star Wars,Toy Story), Howard Shore (Lord of the Rings), AlanSilvestri (Forrest Gump, Who Framed Roger Rabbit), usedin Akira the animation film, and many more
• 1979 Fairlight CMI• 1981 Casio VL-Tone• 1983 Yamaha DX-7• From here on synthesisers are usually digital
Digital synthesis
Hardware synthsare basically computers
Software synthssoftware programs which run in any computer
Modular synths(can be hardware or
software) can connect variousmodules to create different soundsynthesis techniques
Synthesisers
7
FreePure Data, JMaxCsound, Supercollider
Not freeReaktor, MAX/MspDoepfer, Nord Modular
ModularSynthesisers
Digital samples:..., x[n - 1], x[n], x[n + 1], ...
n = number (integer) of the samplex[n] is the sample of number n
Fundamentals of digital synthesis
x(t)
t
x
8
Fundamentals of digital synthesis
Sinusoid: x[n] = A * cos(ωn + φ)A = amplituden = number of the sampleω = angular frequency = 2*pi*fφ = initial phase = angle at x[0] i.e. at n = 0
Example: 440Hz cosine waveAngular frequency ω is 2*pi*f = 2764.6 rads/secInitial Phase φ = zeroAmplitude = 1 ([-1,1] amplitude range)
The cosine wave
9
Cosine
Now we can calculate each sample of the440Hz wave:
X[1] = 1*cos(2764.6*1+0)X[2] = 1*cos(2764.6*2+0)X[3] = 1*cos(2764.6*3+0)X[4] = 1*cos(2764.6*4+0)…X[n] = 1*cos(2764.6*n+0)
Sampling rateR = sampling rate = number of samples in 1
secondUsually in cinema 48000 samples per second
Sample number and timeR*t = n, number of samples in the time tSample n occurs at time:
t = n/R seconds
10
Sound parametersPerceptualparameters
Acousticalparameters
Scale Units
Pitch
Volume
Duration
Timbre
Frequency
Wave amplitude
Time
Frequencycomponents
Log
Log
Linear
N/A
Hertzor Hzdecibelsor dBs
ms
N/A
A very simple sounda sine wave consists of one frequency
Temporal description Spectral description
A sinewave in Pure Data
11
Fourier proved:any continuous periodic wave = an infinite sum of sinewaves
Instruments’sounds:
fundamentalfrequency andmany harmonics
Harmonics are atintegersmultiples of thefundamentalfrequency
Complex sound
Harmonics have various amplitudes and they determinethe timbre of a sound
12
sine, triangular, square, sawtooth, pulse, white noise
Sine wave
Basic unit generators
Square wave
• The square wavecontains only oddharmonics with theamplitudes
• A = amplitude• n = no. of harmonic• Brass sounds
13
Sawtooth wave
• The sawtooth wavecontains all harmonicsin the ratio
• A = amplitude• N = no. of harmonic• String sounds
Triangular wave
• The triangular wavecontains only oddharmonics with theamplitudes
• A = amplitude• n = no. of harmonic
14
Pulse wave
• The pulse wavecontains all theharmonics
• Subtractive synthesis• Speech sounds
White noise
• distributedspectrum, i.e.energy existseverywherewithin a range offrequencies
• no repetitions!
• randomnumbers
15
Let’s hear them in Reaktor!
Additive Synthesis and organs
• The concept of additive synthesis is many centuriesold
• It means ADDING tones of different frequenciestogether to create new sounds
• First applied in pipe organs by means of register-stops• Pulling a register-stop would mean to send air to a set
of pipes• The air was then released into the pipe by pressing a
key in the keyboard and would produce sound• This would result in creating sounds by MIXING
frequencies
16
Additive Synthesis and organs
• First ever electrical additive synthesizer (1906,Thaddeus Cahill)• Electric motors produced the A.C. ‘tones’• Signals connected to telephone receivers with papercones attached to ‘amplify’ the signals• Use of ‘stops’ to add in motors at different frequencies
The Telharmonium
http://www.discretesynthesizers.com/archives/miessner/em1936.htm
17
• The DB33 Instrument in ProTools is an example ofadditive synthesiser• Another example is the EVB3 Instrument in Logic
The Hammond Organ
• It emulates in the digital world the Hammond B3organ which was created by Laurens Hammond inthe 1930s• The original organ was electro-mechanical• Electric motors drove a set of metal wheels (called“tone wheels”) which, as they were spinning nearelectromagnets, generated electromagnetic waves(through the process of electromagnetic induction) atcontrollable frequencies• These electromagnetic wave signals were then sentto loudspeakers and perceived as sounds
The Hammond Organ
18
The Hammond Organ
• The Hammond Organ also had features such as vibratoand reverb, and various other effects
• Digital additive synthesiser• 64 harmonics per note for 8 voices• Detuning and LFO (Low Frequency Oscillator)
Kawai K5 (late 80s)
19
Basic additive synthesis
Basic Additive Synthesis in PD
Sound effects created in additivesynthesis
Old telephone bell ringing• 2 bells: additive synthesis• 1 hammer: very short noise• 1 telephone casing: reflections inside the
casing modelled with two delays
Example from Andy Farnell
Example of detuned additive sound in PD
Additive bell sound (Jean-Claude Risset)
20
The first sound that we hear in Apocalypse Now, over a black screen, is of ahelicopter circling us in the 5.1 channel space. But itʼs not an actual helicopterthat weʼre hearing. What weʼre hearing is an electronically generatedsimulation of helicopter blades. The sound was designed by Richard Beggs, aman who did a lot of wonderful sound effects fabrication for Apocalypse Now,and he made it with a fairly early version of a Moog synthesiser. Sometimeswhen Iʼm talking to classes about this sequence, I ask: ʻWhy the synthesisedhelicopter?ʼ Lord knows we had plenty of recordings of actual helicopters. Iknow because I recorded some of them myself. Why did Francis Coppola andWalter Murch, the guru of us all, who was in charge of the sound onApocalypse Now, decide to use this synthesised helicopter? The answer, ofcourse, is that is how Willard hears it in his fevered dream; he hears thisstylised dreamy strange ghostly helicopter circling him. And that messagegets through even to the least sophisticated member of the audience. Theyrealise on some level that this strange helicopter is in Willardʼs mind. (Thom,p. 105)
Randy Thom, Screenwriting for sound, The New Soundtrack 1.2 (2011):103–112
Apocalypse Now, 1979
Dir: F.F.CoppolaSound Designer: Walter Murch
21
Additive synthesis
In real sounds:• The amplitude and
frequency of each partialcan vary in time
In synthesisers:• Functions called
Envelopes and LowFrequency Oscillators(LFOs) can be used tovary parameters in time
22
Low Frequency Oscillators or LFOs
• Low Frequency Oscillators or LFOs:– oscillators of frequency between 0-20Hz– in this frequency range an oscillator can control or
modulate various aspects of a sound• Human hearing can perceive frequencies between
20Hz to 20kHz so at frequencies higher than 20Hz anoscillator starts to become audible and it is not anLFO anymore
• LFOs can have various waveforms: sine, square,sawtooth, triangular
flute
Vibrato using an LFO
Vibrato in PD
• Is slow modulation ofthe pitch of a sound
• Low FrequencyOscillator can be usedto simulate this
depth
rate
frequency
output
amplitude
LFO
23
Vibrato spectrogram
Frequency ofharmonicsmodulated by asine wave
From synth secrets of Wendy Carlos:
One way to make an interesting sound out of afew sine waves is:
1. have them all slightly detuned around a pitch(this is a chorused sound)
2. then add vibrato which creates dynamism intime
Choral tone
24
Modulation techniquesThe term modulation refers loosely to any technique that
systematically alters the shape of a waveformExamples are:
- Amplitude modulation and ring modulation- Frequency modulation
• With modulation techniques one can create sounds thatevolve in time in a complex way
• This evolution can be clearly heard and it can also beseen by looking at the spectrum of the resulting sound
• Complex sounds are created efficiently
Amplitude modulation• Modulation of the amplitude of
one oscillator (carrier) by anotheroscillator (modulator)
• The modulation happens ataudio frequencies
i.e. between 20Hz and 20kHz
• The output sound presents 2sidebands at the followingfrequencies:
(fc – fm) , fc , (fc + fm)
CARRIER
MODULATOR
In PD see example 0
output
Fc Am
Fm
Ac
25
Ring modulation
• Ring modulation is a simple case ofamplitude modulation
• In this case the amplitude of the carrier is 0• This is equivalent to simply multiplying the
two signals together• In the spectrum of the output sound new
frequencies will emerge different from theinitial ones
Ring modulation
Ring modulationremoves the carrierfrom the spectrum
MODULATORCARRIER
<-- Same as multiplying theoutputs of a carrieroscillator with amplitude 1and a modulator oscillatorwith amplitude Am
In PD see example 1
output
FcAm
Fm
27
• Pioneered by John Chowning in the late60s
• FM can be thought as:the alteration or distortion of the frequencyof an oscillator in accordance with theamplitude of a modulating signal (Dodge & Jerse,Computer Music, 1985, p.115)
FM synthesis
http://www.youtube.com/watch?v=IsQFuesfaoI
Wall-eDir: Andrew Stanton, 2008
28
Simplest case• A sine wave (the
modulator)modulates thefrequency of asecond sine wave(the carrier)
• In FM, the frequencyof the modulator Fm
is in the audiblerange (20Hz to20kHz)
MODULATOR
CARRIER
Bouncing ball example from Farrell
Am
Fm
Fc
output
Ac
Output SpectrumfSB = fc ± n*fm
where
fSB = sidebandsfrequencies
fc = carrierfrequency
fm = modulatorfrequency
n = 1,2,3,4,…
FM creates sidebands
In PD see examples 2 and 3
31
• Dodge, C. & Jerse, T. A. (1997) Computer Music:synthesis composition and performance, Publisher:Schirmer
• Roads, C. (1996) Computer Music Tutorial, MIT Press• Farnell, A. (2009) Designing Sound, MIT Press• Lynn, P. A., and Fuerst, W. (1998) Introductory Digital
Signal Processing with Computer Applications, JohnWiley and Sons Ltd.
• Puckette, M. The Theory and Technique of ElectronicMusic, 2007, available athttp://crca.ucsd.edu/~msp/techniques.htm
Resources
• Pure Data documentation:http://crca.ucsd.edu/~msp/Pd_documentation/
• Chafe, C. (1999) A short history of digital sound synthesisby composers in the USA,ccrma.stanford.edu/~cc/lyon/historyFinal.pdf
• Smith, J. (1991) Viewpoints of the history of digitalsynthesis, CCRMA
http://www-ccrma.stanford.edu/~jos/kna/
Resources