Canonical Shaders for Optimal Performance Canonical Shaders for
DSP: SHADERS AND LIGHTING FOR AUDIO
description
Transcript of DSP: SHADERS AND LIGHTING FOR AUDIO
![Page 1: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/1.jpg)
![Page 2: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/2.jpg)
DSP: SHADERS AND LIGHTING FOR AUDIO
Guy Whitmore
Director of Audio
Microsoft Game Studios
![Page 3: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/3.jpg)
An Apt Analogy
If raw SFX are akin to 3D visual models, then runtime audio DSP (Digital Signal
Processing) is akin to lighting, pixel shaders, and particle effects.
![Page 4: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/4.jpg)
Agenda
» Current state of DSP in games» Potential for DSP use in games» DSP middleware in traditional
linear audio tools» Creating a DSP middleware market
for games
![Page 5: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/5.jpg)
Why Run-time DSP?
Why not simply bake it in?» Blending and cohesion » Variation» Dimension and depth» Flexibility» Mix iteration and fine control» Shape the mix: Sweeten or fatten
![Page 6: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/6.jpg)
Keepin’ It Real
» In Linear media, DSP is typically kept 'real' until the final mix stage
» In games, final mix is the output of the console as it's being played! The work of the sound designer
continues within the game itself.
![Page 7: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/7.jpg)
Common Uses of DSP in Games
» Environmental Reverb Hardware based vs. software
» Basic EQ and Low Pass Filters LPF over distance or underwater
» Pitch shifting and dopler Variation Car Engines
![Page 8: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/8.jpg)
Further Uses for DSP
» Radio chat DSP» Early reflections for environmental
reverb» USB microphone input» Pitch detection/correction» Time stretch/compression» Aural exciter on swords » Live chat DSP to morph voice
![Page 9: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/9.jpg)
Game Parameters Directly Tied to DSP
» Time of day, change of seasons» Weapon magic or quality over time» Distortion on highest car RPMs» LPF on unloaded auto RPMs
(deceleration)
![Page 10: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/10.jpg)
Run-time Mixing and DSP
» Mix sets with different DSP settings Change the mood from one scene to
another
» EQ and compression intended for mixing via groups Channel strips for character VO
» Mastering chain DSP Mastering EQ, compression, limiting
![Page 11: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/11.jpg)
Where Does DSP for Games Currently Come From?
» As part of Console SDKs Reverbs, compression, EQ
» As part of an Audio Engine license» Coded in-house
Centralize your custom DSP
Result: Limited range and variety of DSP
![Page 12: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/12.jpg)
DSP in Traditional Linear Audio Production
Digital Audio Workstations (DAWs)» Come with many DSP plug-ins» Plug-in archetechture enables 3rd party
DSP makers to participate in a DAW's ecosystem.
Result: » Large and varied 3rd party selection of DSP» High end brand name to free-ware
![Page 13: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/13.jpg)
Why so Many DSP Makers? Isn’t a reverb a reverb?
» Choice» Aesthetics and taste» Variety» Differentiation
Currently most 360 titles use the XDK reverb
» Appropriateness to the title» CPU and tech requirments vary
![Page 14: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/14.jpg)
DSP Licensing For Games…so far» Console makers/card makers
Xbox 360 – Princeton Digital Reverb Xbox (and DirectMusic) – Waves Reverb
» Audio Engine companies? Wwise - McDSP
» Developer licenses Bungie - Waves
» Publisher licenses MGS - Wave Arts Ltd.
![Page 15: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/15.jpg)
MGS and Wave ArtsA pattern for future deals
In short, it’s a…
» multi-license
» multi-algorithm Master Verb, Trackplug, Final Plug, Multidynamics
» multi-platform Xbox 360 and PC
» Multi-format deal XAPO, generic, Wwise and FMOD plug-in compatibility coming
» Structured to give Wave Arts the upfront capital to port the plug-ins.
![Page 16: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/16.jpg)
Wave Arts
Now that the upfront work is done, other publishers and developers can directly license DSP from Wave Arts.
Possibilities:» More algorithms » Port to PS3? …to Wii?
![Page 17: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/17.jpg)
The DreamWouldn’t it be cool if…
» The extensive variety of DSP brands used our linear DAWs were available to us in our game environments?
» We could easily 'plug-and-play' new DSP in our audio engines (no coders) for seamless auditioning?
» Licencing 3rd party DSP was the norm, relatively easy, and affordable?
![Page 18: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/18.jpg)
A 3rd Party DSP Market for Game Audio
Benefits:» Access to a wide range of quality
algorithms» Independent of game platforms and audio
engines » Easy to audition and edit in game and
offline» Sound designers are familiar with many of
the brands and algorithms
![Page 19: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/19.jpg)
Technical ChallengesThe Easy Part (relatively speaking)
» Porting DSP to various game platforms
» Optimizing DSP to console archetechture and for efficiency.
» Availability of dev kits for DSP makers
» Various plug-in formats An industry standard plug-in format is a worthy
goal and should be pursued, but it’s not mandatory for a successful middleware ecosystem.
![Page 20: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/20.jpg)
Financial ChallengesWhat will the market bare?
» How should DSP licenses be charged?
» What's a fair price per SKU? Affordable by developers/publishers While making the effort worthwhile for a DSP
maker Differentiation between DVD and downloadable
games?
» Audio/game budgets for DSP licenses
![Page 21: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/21.jpg)
Chicken Egg ConundrumHow to kick-start a middleware ecosystem
Kindling for the fire» Build a critical mass of licensors
3-5 deals similar to Wave Arts
![Page 22: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/22.jpg)
An Industry ChallengeTo major game publishers and audio leaders:
Strike new DSP porting/licensing deals with existing DSP makers
» Modeled similarly to the MGS – Wave Arts deal i.e.
Multi-license and multi-algorithm DSP company able to license algorithms to
other developers after contract term
» Work with DSP companies that… Make interesting quality algorithms Show willingness to forge into a new market Create CPU efficient algorithms Will support their products into the future
![Page 23: DSP: SHADERS AND LIGHTING FOR AUDIO](https://reader036.fdocuments.net/reader036/viewer/2022081501/56813c47550346895da5c362/html5/thumbnails/23.jpg)
??Questions??
??????????????
???????
??????????