Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for...

Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com

Advanced Granular Synthesis for Next-Generation Games

Game Developer's Conference 2007San Francisco

Leonard J. Paul

Vancouver Film School

Game Audio Instructor

info {at} VideoGameAudio.com

604-685-5808 x4035


Overview

1)How to utilise granular for speech, sound effects, music and engines

2)Prototyping granular synthesis using Pure Data and Open Sound Control

3)How to add granular synthesis to your sound driver


Using granular forspeech

sound effectsmusic andengines

1)


1) Selection order (fwd/rev or freeze)

2) Pitch shift

3) Amplitude range

4) Spatialization / panning

5) Grain duration

6) Grain density (grains/sec, # of grain voices)

7) Envelope (ASR shape or windowing function)

8) DSP effect (reverb, filtering...)

9) Feedback amount (for granular delay lines)

Granular Parameters

A S R

Vol


Game Parameters

● Granular parameters driven by game state

● (ex. RPM -> pitch)● Physics allow for continuous control

over playback of sample● (ex. ragdoll -> foley)

● Input parameters need to be carefully tuned to drive synthesis

● (ex. instrument played by novice)


Game Audio Data Diagram

Game(Physics, GFX,

AI etc..)

GameAudioCode

GranularDriverCode

DAC

Commercial

SoundTool

In-houseSoundTool

AudioData


Speech

● Time warping speech for variation● Change formants of voice● Realtime granular effects to change

character of voice


Sound Effects

● Crossfade morph via grains● Spacialization in 5.1/7.1● Contact physics for sliding/scraping● Add variation to existing sounds


Engines

● Add as a layer for variation● Pre-segmentation and playback via

RPM/load band:

Loa

d

RPM

Samples

Low RPM & low load grains


Music and Ambiences

● Stretch a sound for a granular drone● Tempo-sync grains for interesting

gating effects● Merge ambiences by selecting grains

from multiple files● Making an ambience sound more

musical and abstract


Prototyping Granular Synthesis using Pure Data and OSC

2)


Coding PD Objects

● FlexT is a cross-platform method of generating custom C++ PD objects: http://grrrr.org/ext/flext/

● Easily possible to port part of the Synthesis Toolkit (STK) to PD using Code Blocks or any other DSP code


Flext (gain~ object code)class gain: public flext_dsp

{ FLEXT_HEADER(gain,flext_dsp)

float factor;

gain(): factor(0)

{ AddInSignal("In"); // add signal inlet

AddInFloat("Gain");

AddOutSignal("Out"); // add signal outlet

FLEXT_ADDMETHOD(1,m_gain); }

virtual void m_signal(int n,t_sample *const *insigs,t_sample *const *outsigs) {

const t_sample *in = insigs[0];

t_sample *out = outsigs[0];

while(n--) *(out++) = *(in++)*factor; }

void m_gain(float db) { factor = pow(10.f,db*0.05f); }

FLEXT_CALLBACK_F(m_gain); };


OSC Layer for Prototyping

● Add an Open Sound Control (OSC) layer to your game to communicate with a rapid prototyping audio environment such as PD

● Example of using OSC with a Half-life 2 Source Mod

● Possible to compile PD to a language such as Java (pd2j2me)


OSC Code for Sending Msg#include "osc/OscOutboundPacketStream.h"

#include "ip/UdpSocket.h"

#define ADDRESS "127.0.0.1" // localhost but can be any address

#define PORT 7000

#define OUTPUT_BUFFER_SIZE 1024

int main(int argc, char* argv[])

{

UdpTransmitSocket transmitSocket( IpEndpointName( ADDRESS, PORT ) );

char buffer[OUTPUT_BUFFER_SIZE];

osc::OutboundPacketStream p( buffer, OUTPUT_BUFFER_SIZE ); // create buffer

p << osc::BeginMessage( "/test" ) << "hi world" << osc::EndMessage; // message

transmitSocket.Send( p.Data(), p.Size() ); // send message

}


Adding OSC to Half-life 2

● Download the OSCpack library from:http://www.audiomulch.com/~rossb/code/oscpack/

● Download Visual C++ 2005 Express, Half-Life 2 and create a new mod

● Add the OSC files to the mod source (some tweaking needed) and modify SoundEmitter.cpp etc.

● This is a similar process to adding OSC to your own sound driver


Adding Granular Synthesis to your Sound Driver

3)


Granular Sound Driver

● PD C++ code can be ported to the next-gen consoles as the consoles are primarily software based

● Synthesis should be creatively used in conjunction with samples for sound design

● Advantage is that real-time control over synthesis parameters is from game parameters


Granular Code ExamplesRoss Bencina – Granular Delay Line● Uses Borland C++ Builder

http://www.audiomulch.com/~rossb/

Synthesis Toolkit (STK) – Granulate● Can use flext to port to PD

http://ccrma.stanford.edu/software/stk/

SyncGrain~ for PD (from Csound)● Ported using flext

http://footils.org/cms/pms/


Related Research

● Concatenative Sound SynthesisSound "Mosaic" Analysis & ResynthesisUses grains from one sample to

approximate grains of a second samplewww.mat.ucsb.edu/~b.sturm/sand/VLDCMCaR/VLDCMCaR.html


Related Research

● Natural Grain ResynthesisMake grains that preserve transientshttp://www.cs.ubc.ca/~reynald/naturalgrains.html


Related Research

● Pure Data Compiler (pd2j2me)Compile PD code to Javahttp://www.uow.edu.au/~mh675/publications/nime_2005.pdf

public void met0Bang() { double a; a = counter.bang(); a = a * 3; a = a / 4; }


Example

● Example of gameplay utilizing granular synthesis


Questions?

Email: info-at-VideoGameAudio.com

Web: www.VideoGameAudio.com

Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for...

Documents

Transcript of Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for...