Media #WWDC16
Transcript of Media #WWDC16
![Page 1: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/1.jpg)
© 2016 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
A guide to audio best practices and APIs
Media #WWDC16
Session 507
Delivering an Exceptional Audio Experience
Saleem Mohammed Audio CraftsmanDoug Wyatt Audio Plumber
![Page 2: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/2.jpg)
OverviewIntroduction
CoreAudio and Drivers
Application
![Page 3: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/3.jpg)
OverviewIntroduction
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
![Page 4: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/4.jpg)
OverviewIntroduction
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AudioToolbox
AudioUnits AUGraph
![Page 5: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/5.jpg)
OverviewIntroduction
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AudioToolbox
AudioUnits AUGraph
CoreMIDI
![Page 6: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/6.jpg)
OverviewIntroduction
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AudioToolbox
AudioUnits AUGraph
CoreMIDI
OpenAL
![Page 7: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/7.jpg)
OverviewIntroduction
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AudioToolbox
AudioUnits AUGraph
AUAudioUnit
AVAudioEngineCoreMIDI
OpenAL
![Page 8: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/8.jpg)
AgendaIntroduction
![Page 9: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/9.jpg)
AgendaIntroduction
Essential SetupSimple Playback and Recording ScenariosAdvanced Playback and Recording Scenarios Multichannel Audio
![Page 10: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/10.jpg)
AgendaIntroduction
Essential SetupSimple Playback and Recording ScenariosAdvanced Playback and Recording Scenarios Multichannel AudioReal-time AudioInstruments, Effects, and GeneratorsMIDI
![Page 11: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/11.jpg)
Configuration for iOS, tvOS, and watchOSEssential Setup
![Page 12: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/12.jpg)
Audio Is a Managed Service
Device
Operating System
Drivers
![Page 13: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/13.jpg)
Audio Is a Managed Service
Inputs
Device
Operating System
Drivers
![Page 14: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/14.jpg)
Audio Is a Managed Service
Inputs Outputs
Device
Operating System
Drivers
![Page 15: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/15.jpg)
Audio Is a Managed Service
Another AppSome App
Inputs Outputs
Device
Operating System
Drivers
![Page 16: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/16.jpg)
Audio Is a Managed Service
Another AppSome App
Inputs Outputs
Device
Operating System
Drivers
Your App
AVAudioSession
![Page 17: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/17.jpg)
Express app’s high-level audio needsAVAudioSession
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
AVAudioSessionCategory
ModeOptions
Your App
![Page 18: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/18.jpg)
Advanced configurationAVAudioSession
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
AVAudioSessionInput SelectionChannel Count
Sample Rate
Your App
![Page 19: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/19.jpg)
Interact with other audio appsAVAudioSession
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
ActivateAVAudioSession
CategorySession Properties
Your App
![Page 20: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/20.jpg)
Essential stepsAVAudioSession
Sign up for notificationsSet category, mode, and optionsManage activationHandle notifications
![Page 21: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/21.jpg)
Essential stepsAVAudioSession
Sign up for notificationsSet category, mode, and optionsManage activationHandle notifications
![Page 22: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/22.jpg)
Sign up for Notifications
AVAudioSessionInterruptionNotificationAVAudioSessionRouteChangeNotificationAVAudioSessionMediaServicesWereResetNotification
![Page 23: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/23.jpg)
Essential stepsAVAudioSession
Sign up for notificationsSet category, mode, and optionsManage activationHandle notifications
![Page 24: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/24.jpg)
Productivity appSet Category, Mode, and Options
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient)
}
catch {
print(error)
}
![Page 25: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/25.jpg)
Productivity appSet Category, Mode, and Options
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient)
}
catch {
print(error)
}
![Page 26: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/26.jpg)
Productivity appSet Category, Mode, and Options
Will obey ringer switchWill not play audio in the backgroundWill always mix with others
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient)
}
catch {
print(error)
}
![Page 27: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/27.jpg)
Podcast appSet Category, Mode, and Options
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
try AVAudioSession.sharedInstance().setMode(AVAudioSessionModeSpokenAudio)
}
catch {
print(error)
}
![Page 28: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/28.jpg)
Podcast appSet Category, Mode, and Options
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
try AVAudioSession.sharedInstance().setMode(AVAudioSessionModeSpokenAudio)
}
catch {
print(error)
}
![Page 29: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/29.jpg)
Podcast appSet Category, Mode, and Options
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
try AVAudioSession.sharedInstance().setMode(AVAudioSessionModeSpokenAudio)
}
catch {
print(error)
}
![Page 30: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/30.jpg)
Podcast appSet Category, Mode, and Options
Will interrupt other applications*Background audio key
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
try AVAudioSession.sharedInstance().setMode(AVAudioSessionModeSpokenAudio)
}
catch {
print(error)
}
![Page 31: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/31.jpg)
Navigation appSet Category, Mode, and Options
do {
let myAudioSession = AVAudioSession.sharedInstance()
try myAudioSession.setCategory(AVAudioSessionCategoryPlayback,
mode: AVAudioSessionModeDefault,
options:[.interruptSpokenAudioAndMixWithOthers, .duckOthers])
}
catch {
print(error)
}
![Page 32: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/32.jpg)
Navigation appSet Category, Mode, and Options
do {
let myAudioSession = AVAudioSession.sharedInstance()
try myAudioSession.setCategory(AVAudioSessionCategoryPlayback,
mode: AVAudioSessionModeDefault,
options:[.interruptSpokenAudioAndMixWithOthers, .duckOthers])
}
catch {
print(error)
}
![Page 33: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/33.jpg)
Navigation appSet Category, Mode, and Options
do {
let myAudioSession = AVAudioSession.sharedInstance()
try myAudioSession.setCategory(AVAudioSessionCategoryPlayback,
mode: AVAudioSessionModeDefault,
options:[.interruptSpokenAudioAndMixWithOthers, .duckOthers])
}
catch {
print(error)
}
![Page 34: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/34.jpg)
Navigation appSet Category, Mode, and Options
do {
let myAudioSession = AVAudioSession.sharedInstance()
try myAudioSession.setCategory(AVAudioSessionCategoryPlayback,
mode: AVAudioSessionModeDefault,
options:[.interruptSpokenAudioAndMixWithOthers, .duckOthers])
}
catch {
print(error)
}
![Page 35: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/35.jpg)
Navigation appSet Category, Mode, and Options
*Background audio key
do {
let myAudioSession = AVAudioSession.sharedInstance()
try myAudioSession.setCategory(AVAudioSessionCategoryPlayback,
mode: AVAudioSessionModeDefault,
options:[.interruptSpokenAudioAndMixWithOthers, .duckOthers])
}
catch {
print(error)
}
![Page 36: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/36.jpg)
Essential stepsAVAudioSession
Sign up for notificationsSet category, mode, and optionsManage activationHandle notifications
![Page 37: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/37.jpg)
Manage Session Activation
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
ActivateAVAudioSession
Category:PlayAndRecord
Session Properties
Your App
![Page 38: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/38.jpg)
Manage Session Activation
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
ActivateAVAudioSession
Category:PlayAndRecord
Session Properties
Your App
![Page 39: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/39.jpg)
Manage Session Activation
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
ActivateAVAudioSession
Category:PlayAndRecord
Session Properties
Your App
![Page 40: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/40.jpg)
Manage Session Activation
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
ActivateAVAudioSession
Category:PlayAndRecord
Session Properties
Your App
![Page 41: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/41.jpg)
Manage Session Activation
AVAudioSession
Another App
AVAudioSession
Some App
Inputs Outputs
Device
Operating System
Drivers
ActivateAVAudioSession
Category:PlayAndRecord
Session Properties
Your App
![Page 42: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/42.jpg)
Essential stepsAVAudioSession
Sign up for notificationsSet category, mode, and optionsManage activationHandle notifications
![Page 43: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/43.jpg)
InterruptionsHandle Notifications
No Playback UI
func handleInterruption(notification: NSNotification) {
let interruptionType = notification.userInfo![AVAudioSessionInterruptionTypeKey]
as! AVAudioSessionInterruptionType
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, players have been paused; update any internal state
}
else { //end interruption
// activate session, start playing, update internal state
}
}
![Page 44: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/44.jpg)
InterruptionsHandle Notifications
No Playback UI
func handleInterruption(notification: NSNotification) {
let interruptionType = notification.userInfo![AVAudioSessionInterruptionTypeKey]
as! AVAudioSessionInterruptionType
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, players have been paused; update any internal state
}
else { //end interruption
// activate session, start playing, update internal state
}
}
![Page 45: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/45.jpg)
InterruptionsHandle Notifications
No Playback UI
func handleInterruption(notification: NSNotification) {
let interruptionType = notification.userInfo![AVAudioSessionInterruptionTypeKey]
as! AVAudioSessionInterruptionType
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, players have been paused; update any internal state
}
else { //end interruption
// activate session, start playing, update internal state
}
}
![Page 46: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/46.jpg)
InterruptionsHandle Notifications
No Playback UI
func handleInterruption(notification: NSNotification) {
let interruptionType = notification.userInfo![AVAudioSessionInterruptionTypeKey]
as! AVAudioSessionInterruptionType
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, players have been paused; update any internal state
}
else { //end interruption
// activate session, start playing, update internal state
}
}
![Page 47: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/47.jpg)
InterruptionsHandle Notifications
Playback UI
...
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, update internal state as well as UI!
}
else { //end interruption
if let interruptionOption = notification.userInfo![AVAudioSessionInterruptionOptionKey]
as? AVAudioSessionInterruptionOptions {
if interruptionOption == .shouldResume {
// activate session, start playing, update UI & internal state
}
}
...
![Page 48: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/48.jpg)
InterruptionsHandle Notifications
Playback UI
...
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, update internal state as well as UI!
}
else { //end interruption
if let interruptionOption = notification.userInfo![AVAudioSessionInterruptionOptionKey]
as? AVAudioSessionInterruptionOptions {
if interruptionOption == .shouldResume {
// activate session, start playing, update UI & internal state
}
}
...
![Page 49: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/49.jpg)
InterruptionsHandle Notifications
Playback UI
...
if interruptionType == AVAudioSessionInterruptionType.began {
//session inactive, update internal state as well as UI!
}
else { //end interruption
if let interruptionOption = notification.userInfo![AVAudioSessionInterruptionOptionKey]
as? AVAudioSessionInterruptionOptions {
if interruptionOption == .shouldResume {
// activate session, start playing, update UI & internal state
}
}
...
![Page 50: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/50.jpg)
InterruptionsHandle Notifications
Not every Begin is followed by an End• e.g., media players that interrupt each other
![Page 51: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/51.jpg)
Route changesHandle Notifications
Media apps should stop playing if route change reason is .OldDeviceUnavailableIf reason is .NewDeviceAvailable OR .OldDeviceUnavailableRe-evaluate session properties
let routeChangeReason = notification.userInfo![AVAudioSessionRouteChangeReasonKey]
as! AVAudioSessionRouteChangeReason
if routeChangeReason == .oldDeviceUnavailable {
//media players stop playback; ex. headsets unplugged while listening to music
}
if routeChangeReason == .oldDeviceUnavailable || routeChangeReason == .newDeviceAvailable {
//advanced use cases; re-evaluate session properties; ex. Sample rate
}
![Page 52: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/52.jpg)
Route changesHandle Notifications
Media apps should stop playing if route change reason is .OldDeviceUnavailableIf reason is .NewDeviceAvailable OR .OldDeviceUnavailableRe-evaluate session properties
let routeChangeReason = notification.userInfo![AVAudioSessionRouteChangeReasonKey]
as! AVAudioSessionRouteChangeReason
if routeChangeReason == .oldDeviceUnavailable {
//media players stop playback; ex. headsets unplugged while listening to music
}
if routeChangeReason == .oldDeviceUnavailable || routeChangeReason == .newDeviceAvailable {
//advanced use cases; re-evaluate session properties; ex. Sample rate
}
![Page 53: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/53.jpg)
Route changesHandle Notifications
Media apps should stop playing if route change reason is .OldDeviceUnavailableIf reason is .NewDeviceAvailable OR .OldDeviceUnavailableRe-evaluate session properties
let routeChangeReason = notification.userInfo![AVAudioSessionRouteChangeReasonKey]
as! AVAudioSessionRouteChangeReason
if routeChangeReason == .oldDeviceUnavailable {
//media players stop playback; ex. headsets unplugged while listening to music
}
if routeChangeReason == .oldDeviceUnavailable || routeChangeReason == .newDeviceAvailable {
//advanced use cases; re-evaluate session properties; ex. Sample rate
}
![Page 54: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/54.jpg)
Media services were resetHandle Notifications
![Page 55: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/55.jpg)
Media services were resetHandle Notifications
Rare, but it does happen
![Page 56: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/56.jpg)
Media services were resetHandle Notifications
Rare, but it does happenAVAudioSession sharedInstance pointer still valid
![Page 57: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/57.jpg)
Media services were resetHandle Notifications
Rare, but it does happenAVAudioSession sharedInstance pointer still validNeed to reset category, mode, options, etc
![Page 58: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/58.jpg)
Media services were resetHandle Notifications
Rare, but it does happenAVAudioSession sharedInstance pointer still validNeed to reset category, mode, options, etcNeed to destroy and recreate objects• AVAudioEngine, queues, remote I/Os, players, etc
![Page 59: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/59.jpg)
Media services were resetHandle Notifications
Rare, but it does happenAVAudioSession sharedInstance pointer still validNeed to reset category, mode, options, etcNeed to destroy and recreate objects• AVAudioEngine, queues, remote I/Os, players, etc
Test with Settings -> Developer -> Reset Media Services
![Page 60: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/60.jpg)
Essential steps reviewAVAudioSession
Sign up for notificationsSet category, mode, and optionsManage activationHandle notifications
![Page 61: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/61.jpg)
NEWAirPlay and A2DP in PlayAndRecord
Stereo audio over Bluetooth and AirPlay
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord,
mode: AVAudioSessionModeDefault,
options: [.allowAirPlay, .allowBluetoothA2DP])
![Page 62: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/62.jpg)
NEWAirPlay and A2DP in PlayAndRecord
Stereo audio over Bluetooth and AirPlay
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord,
mode: AVAudioSessionModeDefault,
options: [.allowAirPlay, .allowBluetoothA2DP])
![Page 63: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/63.jpg)
NEWAirPlay and A2DP in PlayAndRecord
Stereo audio over Bluetooth and AirPlay
Now you can use microphone while playing to Bluetooth/AirPlay
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord,
mode: AVAudioSessionModeDefault,
options: [.allowAirPlay, .allowBluetoothA2DP])
![Page 64: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/64.jpg)
NEWAirPlay and A2DP in PlayAndRecord
Stereo audio over Bluetooth and AirPlay
Now you can use microphone while playing to Bluetooth/AirPlayLet the user pick the route from Control Center or a MPVolumeView
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord,
mode: AVAudioSessionModeDefault,
options: [.allowAirPlay, .allowBluetoothA2DP])
![Page 65: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/65.jpg)
New Property for VoIP Apps NEW
let audioSession = AVAudioSession.sharedInstance()
if let currentPort:AVAudioSessionPortDescription = audioSession.currentRoute.inputs.first {
let disableSoftwareVoiceProcessing = currentPort.hasHardwareVoiceCallProcessing
}
![Page 66: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/66.jpg)
New Property for VoIP Apps NEW
let audioSession = AVAudioSession.sharedInstance()
if let currentPort:AVAudioSessionPortDescription = audioSession.currentRoute.inputs.first {
let disableSoftwareVoiceProcessing = currentPort.hasHardwareVoiceCallProcessing
}
![Page 67: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/67.jpg)
New Property for VoIP Apps
Not needed if using Apple’s Voice Processing IO kAudioUnitSubType_VoiceProcessingIO
NEW
let audioSession = AVAudioSession.sharedInstance()
if let currentPort:AVAudioSessionPortDescription = audioSession.currentRoute.inputs.first {
let disableSoftwareVoiceProcessing = currentPort.hasHardwareVoiceCallProcessing
}
![Page 68: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/68.jpg)
New Property for VoIP Apps
Not needed if using Apple’s Voice Processing IO kAudioUnitSubType_VoiceProcessingIO
NEW
let audioSession = AVAudioSession.sharedInstance()
if let currentPort:AVAudioSessionPortDescription = audioSession.currentRoute.inputs.first {
let disableSoftwareVoiceProcessing = currentPort.hasHardwareVoiceCallProcessing
}
Enhancing VoIP Apps with CallKit Mission Thursday 5:00PM
![Page 69: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/69.jpg)
Further Reference
Audio session programming guide • https://developer.apple.com/library/ios/documentation/Audio/Conceptual/
AudioSessionProgrammingGuide
Audio Session and Multiroute Audio in iOS WWDC 2012
What’s New in Core Audio WWDC 2014
What’s New in Core Audio WWDC 2015
![Page 70: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/70.jpg)
Simple Playback and Recording
![Page 71: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/71.jpg)
Simple playback and recordingAVFoundation Framework
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AudioToolbox
AudioUnits AUGraph
AUAudioUnit
CoreMIDI
OpenAL
CoreAudio and Drivers
Application
![Page 72: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/72.jpg)
Simplest way to play an audio fileAVAudioPlayer
![Page 73: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/73.jpg)
Simplest way to play an audio fileAVAudioPlayer
Plays file types supported by AudioFile API• e.g., wav, caf, m4a, aac, mp3, aif
![Page 74: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/74.jpg)
Simplest way to play an audio fileAVAudioPlayer
Plays file types supported by AudioFile API• e.g., wav, caf, m4a, aac, mp3, aif
Provides basic playback operations
![Page 75: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/75.jpg)
Simplest way to play an audio fileAVAudioPlayer
Plays file types supported by AudioFile API• e.g., wav, caf, m4a, aac, mp3, aif
Provides basic playback operationsSupports volume, metering, looping, playback rate, panning
![Page 76: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/76.jpg)
Simplest way to play an audio fileAVAudioPlayer
Plays file types supported by AudioFile API• e.g., wav, caf, m4a, aac, mp3, aif
Provides basic playback operationsSupports volume, metering, looping, playback rate, panningChannel assignment (iOS/tvOS)
![Page 77: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/77.jpg)
Simplest way to play an audio fileAVAudioPlayer
Plays file types supported by AudioFile API• e.g., wav, caf, m4a, aac, mp3, aif
Provides basic playback operationsSupports volume, metering, looping, playback rate, panningChannel assignment (iOS/tvOS)Multiple AVAudioPlayer objects for multiple sounds; synchronized playback
![Page 78: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/78.jpg)
Simplest way to play an audio fileAVAudioPlayer
Plays file types supported by AudioFile API• e.g., wav, caf, m4a, aac, mp3, aif
Provides basic playback operationsSupports volume, metering, looping, playback rate, panningChannel assignment (iOS/tvOS)Multiple AVAudioPlayer objects for multiple sounds; synchronized playback
NEW
public func setVolume(_ volume: Float, fadeDuration duration: TimeInterval)
/* fade to a new volume over a duration */
![Page 79: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/79.jpg)
// AVAudioPlayer Example - Productivity App
class ViewController: UIViewController {
var successSoundPlayer:AVAudioPlayer!
let successSoundURL = Bundle.main().urlForResource("success", withExtension: "caf")
override func viewDidLoad() {
do { // setup AVAudioSession if necessary (Ambient); setup other members
successSoundPlayer = try AVAudioPlayer.init(contentsOf: successSoundURL!)
successSoundPlayer.prepareToPlay()
}
catch {
// handle error
}
}
@IBAction func saveDocument() {
// do some other work; if successful, play success sound!
successSoundPlayer.play() //check return value
}
}
![Page 80: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/80.jpg)
// AVAudioPlayer Example - Productivity App
class ViewController: UIViewController {
var successSoundPlayer:AVAudioPlayer!
let successSoundURL = Bundle.main().urlForResource("success", withExtension: "caf")
override func viewDidLoad() {
do { // setup AVAudioSession if necessary (Ambient); setup other members
successSoundPlayer = try AVAudioPlayer.init(contentsOf: successSoundURL!)
successSoundPlayer.prepareToPlay()
}
catch {
// handle error
}
}
@IBAction func saveDocument() {
// do some other work; if successful, play success sound!
successSoundPlayer.play() //check return value
}
}
![Page 81: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/81.jpg)
// AVAudioPlayer Example - Productivity App
class ViewController: UIViewController {
var successSoundPlayer:AVAudioPlayer!
let successSoundURL = Bundle.main().urlForResource("success", withExtension: "caf")
override func viewDidLoad() {
do { // setup AVAudioSession if necessary (Ambient); setup other members
successSoundPlayer = try AVAudioPlayer.init(contentsOf: successSoundURL!)
successSoundPlayer.prepareToPlay()
}
catch {
// handle error
}
}
@IBAction func saveDocument() {
// do some other work; if successful, play success sound!
successSoundPlayer.play() //check return value
}
}
![Page 82: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/82.jpg)
// AVAudioPlayer Example - Productivity App
class ViewController: UIViewController {
var successSoundPlayer:AVAudioPlayer!
let successSoundURL = Bundle.main().urlForResource("success", withExtension: "caf")
override func viewDidLoad() {
do { // setup AVAudioSession if necessary (Ambient); setup other members
successSoundPlayer = try AVAudioPlayer.init(contentsOf: successSoundURL!)
successSoundPlayer.prepareToPlay()
}
catch {
// handle error
}
}
@IBAction func saveDocument() {
// do some other work; if successful, play success sound!
successSoundPlayer.play() //check return value
}
}
![Page 83: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/83.jpg)
Simplest way to record an audio fileAVAudioRecorder
Record for a specific durationRecords until stoppedMeteringSupports a variety of encoding formats• AAC, HE-AAC, HE-AACv2, ALAC, LPCM
![Page 84: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/84.jpg)
Settings dictionaryAVAudioRecorder
FormatSample rateNumber of channelsFor LPCM• Bit depth, endian-ness
For encoded formats • Quality, bit rate
![Page 85: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/85.jpg)
// AVAudioRecorder Example
do { // setup AVAudioSession (Record/PlayAndRecord); user permission; input selection
let formatSettings = [AVSampleRateKey : 44100.0,
AVNumberOfChannelsKey : 1,
AVFormatIDKey : Int(kAudioFormatMPEG4AAC),
AVEncoderBitRateKey : 192000,
AVEncoderAudioQualityKey : AVAudioQuality.high.rawValue]
recorder = try AVAudioRecorder.init(url: recordSoundURL, settings: formatSettings)
recorder.prepareToRecord()
}
catch { /* handle error */ }
... @IBAction func toggleRecorder() { if recorder.isRecording { recorder.stop() } else { recorder.record() // provide feedback using meters for example } }
![Page 86: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/86.jpg)
// AVAudioRecorder Example
do { // setup AVAudioSession (Record/PlayAndRecord); user permission; input selection
let formatSettings = [AVSampleRateKey : 44100.0,
AVNumberOfChannelsKey : 1,
AVFormatIDKey : Int(kAudioFormatMPEG4AAC),
AVEncoderBitRateKey : 192000,
AVEncoderAudioQualityKey : AVAudioQuality.high.rawValue]
recorder = try AVAudioRecorder.init(url: recordSoundURL, settings: formatSettings)
recorder.prepareToRecord()
}
catch { /* handle error */ }
... @IBAction func toggleRecorder() { if recorder.isRecording { recorder.stop() } else { recorder.record() // provide feedback using meters for example } }
![Page 87: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/87.jpg)
// AVAudioRecorder Example
do { // setup AVAudioSession (Record/PlayAndRecord); user permission; input selection
let formatSettings = [AVSampleRateKey : 44100.0,
AVNumberOfChannelsKey : 1,
AVFormatIDKey : Int(kAudioFormatMPEG4AAC),
AVEncoderBitRateKey : 192000,
AVEncoderAudioQualityKey : AVAudioQuality.high.rawValue]
recorder = try AVAudioRecorder.init(url: recordSoundURL, settings: formatSettings)
recorder.prepareToRecord()
}
catch { /* handle error */ }
... @IBAction func toggleRecorder() { if recorder.isRecording { recorder.stop() } else { recorder.record() // provide feedback using meters for example } }
![Page 88: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/88.jpg)
// AVAudioRecorder Example
do { // setup AVAudioSession (Record/PlayAndRecord); user permission; input selection
let formatSettings = [AVSampleRateKey : 44100.0,
AVNumberOfChannelsKey : 1,
AVFormatIDKey : Int(kAudioFormatMPEG4AAC),
AVEncoderBitRateKey : 192000,
AVEncoderAudioQualityKey : AVAudioQuality.high.rawValue]
recorder = try AVAudioRecorder.init(url: recordSoundURL, settings: formatSettings)
recorder.prepareToRecord()
}
catch { /* handle error */ }
... @IBAction func toggleRecorder() { if recorder.isRecording { recorder.stop() } else { recorder.record() // provide feedback using meters for example } }
![Page 89: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/89.jpg)
Playback of local and stream audioAVPlayer
Works with file and streaming contentStandard player controls available AVPlayerView / AVPlayerViewControllerWorks with both audio and video media
Advances in AVFoundation Playback Mission Wednesday 9:00AM
![Page 90: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/90.jpg)
Advanced Playback and Recording
![Page 91: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/91.jpg)
Advanced Use Cases
![Page 92: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/92.jpg)
Advanced Use Cases
Playback and recording—files, buffers
![Page 93: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/93.jpg)
Advanced Use Cases
Playback and recording—files, buffersAudio processing—effects, mixing
![Page 94: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/94.jpg)
Advanced Use Cases
Playback and recording—files, buffersAudio processing—effects, mixing3D audio
![Page 95: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/95.jpg)
Advanced Use Cases
Playback and recording—files, buffersAudio processing—effects, mixing3D audio Examples • Karaoke app• DJ app• Game
![Page 96: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/96.jpg)
Advanced playback and recordingAVFoundation Framework
CoreAudio and Drivers
Application
AVFoundation
AudioToolbox
AudioUnits AUGraph
AUAudioUnit
AVAudioEngineCoreMIDI
OpenAL
![Page 97: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/97.jpg)
AVAudioEngine
![Page 98: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/98.jpg)
AVAudioEngine
Powerful, feature-rich Objective-C/Swift API set
![Page 99: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/99.jpg)
AVAudioEngine
Powerful, feature-rich Objective-C/Swift API setSimplifies real-time audio
![Page 100: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/100.jpg)
AVAudioEngine
Powerful, feature-rich Objective-C/Swift API setSimplifies real-time audioManages a graph of nodes
![Page 101: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/101.jpg)
AVAudioEngine
Powerful, feature-rich Objective-C/Swift API setSimplifies real-time audioManages a graph of nodes Features• Play and record audio• Connect audio processing chains, perform mixing• Capture audio at any point in the processing chain• 3D Spatialization
![Page 102: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/102.jpg)
AVAudioNodeAVAudioEngine
![Page 103: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/103.jpg)
AVAudioNodeAVAudioEngine
Source Nodes
Provide data for rendering
AVAudioPlayerNode AVAudioInputNode AVAudioUnitSampler
![Page 104: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/104.jpg)
AVAudioNodeAVAudioEngine
Source Nodes
Provide data for rendering
AVAudioPlayerNode AVAudioInputNode AVAudioUnitSampler
Processing Nodes
Process audio data
AVAudioUnitEffect AVAudioMixerNode AVAudioEnvironmentNode
![Page 105: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/105.jpg)
AVAudioNodeAVAudioEngine
Source Nodes
Provide data for rendering
AVAudioPlayerNode AVAudioInputNode AVAudioUnitSampler
Processing Nodes
Process audio data
AVAudioUnitEffect AVAudioMixerNode AVAudioEnvironmentNode
Destination Node
Terminating node connected to output hardware
AVAudioOutputNode
![Page 106: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/106.jpg)
KaraokeSample Engine Setup
NodeTapBlock(Analyze)
InputNodeEffectNode
(EQ)
PlayerNode(Backing Track)
PlayerNode(Sound Effects)
MixerNode OutputNode
![Page 107: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/107.jpg)
KaraokeSample Engine Setup
NodeTapBlock(Analyze)
InputNodeEffectNode
(EQ)
PlayerNode(Backing Track)
PlayerNode(Sound Effects)
MixerNode OutputNode
![Page 108: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/108.jpg)
KaraokeSample Engine Setup
NodeTapBlock(Analyze)
InputNodeEffectNode
(EQ)
PlayerNode(Backing Track)
PlayerNode(Sound Effects)
MixerNode OutputNode
![Page 109: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/109.jpg)
KaraokeSample Engine Setup
NodeTapBlock(Analyze)
InputNodeEffectNode
(EQ)
PlayerNode(Backing Track)
PlayerNode(Sound Effects)
MixerNode OutputNode
![Page 110: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/110.jpg)
KaraokeSample Engine Setup
NodeTapBlock(Analyze)
InputNodeEffectNode
(EQ)
PlayerNode(Backing Track)
PlayerNode(Sound Effects)
MixerNode OutputNode
![Page 111: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/111.jpg)
Game
3D Space
Sample Engine Setup
MixerNodeOutputNode
InputNode
EnvironmentNode(3D)
PlayerNode
PlayerNode(Backing Track)
![Page 112: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/112.jpg)
Game
3D Space
Sample Engine Setup
MixerNodeOutputNode
InputNode
EnvironmentNode(3D)
PlayerNode
PlayerNode(Backing Track)
![Page 113: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/113.jpg)
Game
3D Space
Sample Engine Setup
MixerNodeOutputNode
InputNode
EnvironmentNode(3D)
PlayerNode
PlayerNode(Backing Track)
![Page 114: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/114.jpg)
Game
3D Space
Sample Engine Setup
MixerNodeOutputNode
InputNode
EnvironmentNode(3D)
PlayerNode
PlayerNode(Backing Track)
positionocclusion..
volume
listenerPositionreverbParameters
![Page 115: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/115.jpg)
Game
3D Space
Sample Engine Setup
MixerNodeOutputNode
InputNode
EnvironmentNode(3D)
PlayerNode
PlayerNode(Backing Track)
positionocclusion…
volume
listenerPositionreverbParameters
![Page 116: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/116.jpg)
Core Classes
![Page 117: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/117.jpg)
Core classesAVAudioFormat
![Page 118: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/118.jpg)
Core classesAVAudioFormat
Describes data format in an audio file or stream• Standard format—non-interleaved Float32• Common formats—Int16/32, Float32/64• Compressed formats—settings dictionary
![Page 119: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/119.jpg)
Core classesAVAudioFormat
Describes data format in an audio file or stream• Standard format—non-interleaved Float32• Common formats—Int16/32, Float32/64• Compressed formats—settings dictionary
Contains AVAudioChannelLayout
![Page 120: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/120.jpg)
Core classesAVAudioFormat
Describes data format in an audio file or stream• Standard format—non-interleaved Float32• Common formats—Int16/32, Float32/64• Compressed formats—settings dictionary
Contains AVAudioChannelLayoutModern interface to• AudioStreamBasicDescription• AudioChannelLayout
![Page 121: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/121.jpg)
Core classesAVAudioBuffer
![Page 122: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/122.jpg)
Core classesAVAudioBuffer
AVAudioPCMBufferAVAudioCompressedBuffer
![Page 123: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/123.jpg)
Core classesAVAudioBuffer
AVAudioPCMBufferAVAudioCompressedBufferModern interface to• AudioBufferList• AudioStreamPacketDescription
![Page 124: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/124.jpg)
Core classesAVAudioFile
![Page 125: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/125.jpg)
Core classesAVAudioFile
Read and write files of any supported formatTakes/provides data in the form of AVAudioPCMBufferTransparently decodes while reading, encodes while writing
![Page 126: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/126.jpg)
Core classesAVAudioFile
Read and write files of any supported formatTakes/provides data in the form of AVAudioPCMBufferTransparently decodes while reading, encodes while writingSupersedes• AudioFile• ExtAudioFile
![Page 127: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/127.jpg)
Core classesAVAudioConverter
![Page 128: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/128.jpg)
Core classesAVAudioConverter
Audio format conversion
![Page 129: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/129.jpg)
Core classesAVAudioConverter
Audio format conversion• PCM to PCM
- Integer/float, bit depth, endian swap, interleave/de-interleave, sample rate conversion
![Page 130: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/130.jpg)
Core classesAVAudioConverter
Audio format conversion• PCM to PCM
- Integer/float, bit depth, endian swap, interleave/de-interleave, sample rate conversion• PCM to/from compressed
- Encoding, decoding
![Page 131: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/131.jpg)
Core classesAVAudioConverter
Audio format conversion• PCM to PCM
- Integer/float, bit depth, endian swap, interleave/de-interleave, sample rate conversion• PCM to/from compressed
- Encoding, decoding
Supersedes• AudioConverter
![Page 132: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/132.jpg)
Core classesAVAudioConverter
Audio format conversion• PCM to PCM
- Integer/float, bit depth, endian swap, interleave/de-interleave, sample rate conversion• PCM to/from compressed
- Encoding, decoding
Supersedes• AudioConverter
NEW
public let AVSampleRateConverterAlgorithm_MinimumPhase: String
![Page 133: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/133.jpg)
Core ClassesAVAudioFormat AVAudioChannelLayout
AVAudioBuffer
AVAudioPCMBuffer
AVAudioCompressedBuffer
AVAudioConverterAVAudioFile
![Page 134: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/134.jpg)
AVAudioFormat AVAudioChannelLayout
AVAudioBuffer
AVAudioPCMBuffer
AVAudioCompressedBuffer
Core Classes
Has
Use
Use
Has
Has
Use
AVAudioConverterAVAudioFile
![Page 135: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/135.jpg)
Core Classes with AVAudioEngine
AVAudioFile
AVAudioFormat
AVAudioPCMBuffer
AVAudioNode
AVAudioConverter
AVAudioCompressedBuffer
AVAudioPlayerNodeAVAudioNodeTapBlock
![Page 136: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/136.jpg)
Core Classes with AVAudioEngine
AVAudioFile
AVAudioFormat
AVAudioPCMBuffer
AVAudioNode
AVAudioConverter
AVAudioCompressedBuffer
AVAudioPlayerNodeAVAudioNodeTapBlock
has
![Page 137: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/137.jpg)
Core Classes with AVAudioEngine
AVAudioFile
AVAudioFormat
AVAudioPCMBuffer
AVAudioNode
AVAudioConverter
AVAudioCompressedBuffer
AVAudioPlayerNodeAVAudioNodeTapBlockuses
uses
is a
has
![Page 138: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/138.jpg)
Core Classes with AVAudioEngine
AVAudioFile
AVAudioFormat
AVAudioPCMBuffer
AVAudioNode
AVAudioConverter
AVAudioCompressedBuffer
AVAudioNodeTapBlockuses
uses
provides
is a
has
AVAudioPlayerNode
![Page 139: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/139.jpg)
Core Classes with AVAudioEngine
AVAudioFile
AVAudioFormat
AVAudioPCMBuffer
AVAudioNode
AVAudioConverter
AVAudioCompressedBuffer
AVAudioNodeTapBlockuses
provides
takes
provides
uses
is a
has
AVAudioPlayerNode
![Page 140: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/140.jpg)
Core Classes with AVAudioEngine
AVAudioFile
AVAudioFormat
AVAudioPCMBuffer
AVAudioNode
AVAudioConverter
AVAudioCompressedBuffer
AVAudioPlayerNodeAVAudioNodeTapBlock
takes
provides
uses
uses
provides
is a
has
![Page 141: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/141.jpg)
Playback and mixingwatchOS
AVAudioSessionCore classes• AVAudioFormat/Buffer/File/Converter
AVAudioEngine• AVAudioPlayerNode• AVAudioMixerNode• AVAudioOutputNode
NEW
![Page 142: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/142.jpg)
DemoAVAudioEngine on watchOS
![Page 143: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/143.jpg)
// AVAudioEngine - watchOS Gaming Example
// Class setup
class GameController: WKInterfaceController {
// other members ex. SceneKit scene
// engine and nodes
var audioEngine = AVAudioEngine()
let explosionPlayer = AVAudioPlayerNode()
let launchPlayer = AVAudioPlayerNode()
// URLS to our audio assets
let explosionAudioURL = URL.init(fileURLWithPath: "/path/to/explosion.caf")
let launchAudioURL = URL.init(fileURLWithPath: "/path/to/launch.caf")
// buffers for playback
var explosionBuffer:AVAudioPCMBuffer?
var launchBuffer:AVAudioPCMBuffer?
}
![Page 144: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/144.jpg)
// AVAudioEngine - watchOS Gaming Example
// Class setup
class GameController: WKInterfaceController {
// other members ex. SceneKit scene
// engine and nodes
var audioEngine = AVAudioEngine()
let explosionPlayer = AVAudioPlayerNode()
let launchPlayer = AVAudioPlayerNode()
// URLS to our audio assets
let explosionAudioURL = URL.init(fileURLWithPath: "/path/to/explosion.caf")
let launchAudioURL = URL.init(fileURLWithPath: "/path/to/launch.caf")
// buffers for playback
var explosionBuffer:AVAudioPCMBuffer?
var launchBuffer:AVAudioPCMBuffer?
}
![Page 145: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/145.jpg)
// AVAudioEngine - watchOS Gaming Example
// Class setup
class GameController: WKInterfaceController {
// other members ex. SceneKit scene
// engine and nodes
var audioEngine = AVAudioEngine()
let explosionPlayer = AVAudioPlayerNode()
let launchPlayer = AVAudioPlayerNode()
// URLS to our audio assets
let explosionAudioURL = URL.init(fileURLWithPath: "/path/to/explosion.caf")
let launchAudioURL = URL.init(fileURLWithPath: "/path/to/launch.caf")
// buffers for playback
var explosionBuffer:AVAudioPCMBuffer?
var launchBuffer:AVAudioPCMBuffer?
}
![Page 146: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/146.jpg)
// AVAudioEngine - watchOS Gaming Example
// Class setup
class GameController: WKInterfaceController {
// other members ex. SceneKit scene
// engine and nodes
var audioEngine = AVAudioEngine()
let explosionPlayer = AVAudioPlayerNode()
let launchPlayer = AVAudioPlayerNode()
// URLS to our audio assets
let explosionAudioURL = URL.init(fileURLWithPath: "/path/to/explosion.caf")
let launchAudioURL = URL.init(fileURLWithPath: "/path/to/launch.caf")
// buffers for playback
var explosionBuffer:AVAudioPCMBuffer?
var launchBuffer:AVAudioPCMBuffer?
}
![Page 147: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/147.jpg)
audioEngine.attach(explosionPlayer)
audioEngine.attach(launchPlayer)
do {
// for each of my url assets
let explosionAudioFile = try AVAudioFile.init(forReading: explosionAudioURL)
explosionBuffer = AVAudioPCMBuffer.init(pcmFormat:explosionAudioFile.processingFormat,
frameCapacity: AVAudioFrameCount(explosionAudioFile.length))
try explosionAudioFile.read(into: explosionBuffer!)
// make connections
audioEngine.connect(explosionPlayer, to: audioEngine.mainMixerNode,
format: explosionAudioFile.processingFormat)
audioEngine.connect(launchPlayer, to: audioEngine.mainMixerNode,
format:launchAudioFile.processingFormat)
}
catch { /* handle error */ }
![Page 148: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/148.jpg)
audioEngine.attach(explosionPlayer)
audioEngine.attach(launchPlayer)
do {
// for each of my url assets
let explosionAudioFile = try AVAudioFile.init(forReading: explosionAudioURL)
explosionBuffer = AVAudioPCMBuffer.init(pcmFormat:explosionAudioFile.processingFormat,
frameCapacity: AVAudioFrameCount(explosionAudioFile.length))
try explosionAudioFile.read(into: explosionBuffer!)
// make connections
audioEngine.connect(explosionPlayer, to: audioEngine.mainMixerNode,
format: explosionAudioFile.processingFormat)
audioEngine.connect(launchPlayer, to: audioEngine.mainMixerNode,
format:launchAudioFile.processingFormat)
}
catch { /* handle error */ }
![Page 149: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/149.jpg)
audioEngine.attach(explosionPlayer)
audioEngine.attach(launchPlayer)
do {
// for each of my url assets
let explosionAudioFile = try AVAudioFile.init(forReading: explosionAudioURL)
explosionBuffer = AVAudioPCMBuffer.init(pcmFormat:explosionAudioFile.processingFormat,
frameCapacity: AVAudioFrameCount(explosionAudioFile.length))
try explosionAudioFile.read(into: explosionBuffer!)
// make connections
audioEngine.connect(explosionPlayer, to: audioEngine.mainMixerNode,
format: explosionAudioFile.processingFormat)
audioEngine.connect(launchPlayer, to: audioEngine.mainMixerNode,
format:launchAudioFile.processingFormat)
}
catch { /* handle error */ }
![Page 150: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/150.jpg)
audioEngine.attach(explosionPlayer)
audioEngine.attach(launchPlayer)
do {
// for each of my url assets
let explosionAudioFile = try AVAudioFile.init(forReading: explosionAudioURL)
explosionBuffer = AVAudioPCMBuffer.init(pcmFormat:explosionAudioFile.processingFormat,
frameCapacity: AVAudioFrameCount(explosionAudioFile.length))
try explosionAudioFile.read(into: explosionBuffer!)
// make connections
audioEngine.connect(explosionPlayer, to: audioEngine.mainMixerNode,
format: explosionAudioFile.processingFormat)
audioEngine.connect(launchPlayer, to: audioEngine.mainMixerNode,
format:launchAudioFile.processingFormat)
}
catch { /* handle error */ }
![Page 151: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/151.jpg)
do {
//start engine and players
try audioEngine.start()
explosionPlayer.play()
launchPlayer.play()
}
catch { /* handle error */ }
// create an asteroid and launch
launchPlayer.scheduleBuffer(launchBuffer!, completionHandler: nil)
// wait to launch again
// asteroid is destroyed
explosionPlayer.scheduleBuffer(explosionBuffer!, completionHandler: nil)
// clean up scene and destroy the node
![Page 152: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/152.jpg)
do {
//start engine and players
try audioEngine.start()
explosionPlayer.play()
launchPlayer.play()
}
catch { /* handle error */ }
// create an asteroid and launch
launchPlayer.scheduleBuffer(launchBuffer!, completionHandler: nil)
// wait to launch again
// asteroid is destroyed
explosionPlayer.scheduleBuffer(explosionBuffer!, completionHandler: nil)
// clean up scene and destroy the node
![Page 153: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/153.jpg)
do {
//start engine and players
try audioEngine.start()
explosionPlayer.play()
launchPlayer.play()
}
catch { /* handle error */ }
// create an asteroid and launch
launchPlayer.scheduleBuffer(launchBuffer!, completionHandler: nil)
// wait to launch again
// asteroid is destroyed
explosionPlayer.scheduleBuffer(explosionBuffer!, completionHandler: nil)
// clean up scene and destroy the node
![Page 154: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/154.jpg)
do {
//start engine and players
try audioEngine.start()
explosionPlayer.play()
launchPlayer.play()
}
catch { /* handle error */ }
// create an asteroid and launch
launchPlayer.scheduleBuffer(launchBuffer!, completionHandler: nil)
// wait to launch again
// asteroid is destroyed
explosionPlayer.scheduleBuffer(explosionBuffer!, completionHandler: nil)
// clean up scene and destroy the node
![Page 155: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/155.jpg)
Multichannel Audio
![Page 156: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/156.jpg)
Multichannel Audio on tvOS
HDMI Receiver
Surround Sound System5.1/7.1
1408 x 792
![Page 157: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/157.jpg)
ReviewAVAudioSession Channel Count
do {
//set category, mode & options etc...
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setActive(true)
let desiredNumberOfChannels = 6 // 5.1 surround rendering
if audioSession.maximumOutputNumberOfChannels >= desiredNumberOfChannels {
try audioSession.setPreferredOutputNumberOfChannels(desiredNumberOfChannels)
}
let actualNumberOfChannels = audioSession.outputNumberOfChannels
/* ... */
}
catch { /*handle error*/ }
![Page 158: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/158.jpg)
ReviewAVAudioSession Channel Count
do {
//set category, mode & options etc...
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setActive(true)
let desiredNumberOfChannels = 6 // 5.1 surround rendering
if audioSession.maximumOutputNumberOfChannels >= desiredNumberOfChannels {
try audioSession.setPreferredOutputNumberOfChannels(desiredNumberOfChannels)
}
let actualNumberOfChannels = audioSession.outputNumberOfChannels
/* ... */
}
catch { /*handle error*/ }
![Page 159: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/159.jpg)
ReviewAVAudioSession Channel Count
do {
//set category, mode & options etc...
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setActive(true)
let desiredNumberOfChannels = 6 // 5.1 surround rendering
if audioSession.maximumOutputNumberOfChannels >= desiredNumberOfChannels {
try audioSession.setPreferredOutputNumberOfChannels(desiredNumberOfChannels)
}
let actualNumberOfChannels = audioSession.outputNumberOfChannels
/* ... */
}
catch { /*handle error*/ }
![Page 160: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/160.jpg)
ReviewAVAudioSession Channel Count
do {
//set category, mode & options etc...
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setActive(true)
let desiredNumberOfChannels = 6 // 5.1 surround rendering
if audioSession.maximumOutputNumberOfChannels >= desiredNumberOfChannels {
try audioSession.setPreferredOutputNumberOfChannels(desiredNumberOfChannels)
}
let actualNumberOfChannels = audioSession.outputNumberOfChannels
/* ... */
}
catch { /*handle error*/ }
![Page 161: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/161.jpg)
AVAudioSession Channel Labels
Port Descriptiontype: HDMI
![Page 162: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/162.jpg)
AVAudioSession Channel Labels
ChannelDescription ChannelDescription ChannelDescription …
Port Descriptiontype: HDMI
![Page 163: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/163.jpg)
AVAudioSession Channel Labels
ChannelDescription
channelNumber
channelLabel
ChannelDescription ChannelDescription …1
kAudioChannelLabel_Left
Port Descriptiontype: HDMI
![Page 164: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/164.jpg)
AVAudioEngine Setup
Multichannel contentMono content -> Spatialize (games)
![Page 165: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/165.jpg)
Multichannel contentAVAudioEngine Setup
PlayerNode OutputNodeMixerNodeMultichannelHardware
MultichannelContent
![Page 166: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/166.jpg)
Multichannel contentAVAudioEngine Setup
Set Connection Format Get Hardware Format
PlayerNode OutputNodeMultichannelHardware
MultichannelContent
Get Content Format Set Connection Format
Channel Mapping
MixerNode
![Page 167: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/167.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
PlayerNode OutputNodeMultichannelHardware
MonoContent
PlayerNodeMonoContent
PlayerNodeMonoContent
EnvironmentNode
![Page 168: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/168.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
Get Hardware Format
PlayerNode OutputNodeMultichannelHardware
MonoContent
Spatial Mapping
PlayerNodeMonoContent
PlayerNodeMonoContent
EnvironmentNode
![Page 169: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/169.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
PlayerNode OutputNodeMultichannelHardware
MonoContent
PlayerNodeMonoContent
PlayerNodeMonoContent
Map Get Hardware Format
Spatial Mapping
EnvironmentNode
![Page 170: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/170.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
Set Compatible Format
PlayerNode OutputNodeMultichannelHardware
MonoContent
PlayerNodeMonoContent
PlayerNodeMonoContent
Get Hardware Format
Spatial Mapping
EnvironmentNode
![Page 171: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/171.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
PlayerNode OutputNodeMultichannelHardware
MonoContent
Get Content Format Set Connection Format
PlayerNodeMonoContent
PlayerNodeMonoContent
Set Compatible Format Get Hardware Format
Spatial Mapping
EnvironmentNode
![Page 172: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/172.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
PlayerNode OutputNodeMultichannelHardware
MonoContent
PlayerNodeMonoContent
PlayerNodeMonoContent
SoundField
SoundField
SoundField
MonoContent
Get Content Format Set Connection Format Set Compatible Format Get Hardware Format
Spatial Mapping
EnvironmentNode
![Page 173: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/173.jpg)
Spatialize content with multiple mono sourcesAVAudioEngine Setup
PlayerNode OutputNodeMultichannelHardware
MonoContent
PlayerNodeMonoContent
PlayerNodeMonoContent
Volume
Position
Occlusion
MonoContent
Get Content Format Set Connection Format Set Compatible Format Get Hardware Format
Spatial Mapping
EnvironmentNode
![Page 174: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/174.jpg)
SummaryAVAudioEngine
![Page 175: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/175.jpg)
SummaryAVAudioEngine
Powerful, feature-rich
![Page 176: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/176.jpg)
SummaryAVAudioEngine
Powerful, feature-richSimplifies real-time audio
![Page 177: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/177.jpg)
SummaryAVAudioEngine
Powerful, feature-richSimplifies real-time audioMultichannel, 3D audio
![Page 178: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/178.jpg)
SummaryAVAudioEngine
Powerful, feature-richSimplifies real-time audioMultichannel, 3D audioSupersedes • AUGraph• OpenAL
![Page 179: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/179.jpg)
References
What’s New in Core Audio WWDC 2014
AVAudioEngine in Practice WWDC 2014
What’s New in Core Audio WWDC 2015
![Page 180: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/180.jpg)
Real-Time Audio
Doug Wyatt Audio Plumber
![Page 181: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/181.jpg)
What Is Real-Time Audio?
Low latencyNeeded for• Music: Software synthesizers, effects• Telephony (VoIP)• Other interactive engines
![Page 182: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/182.jpg)
What Is Real-Time Audio?
Deadline: You have N milliseconds to produce N milliseconds of audioTypically ~3–20 ms
![Page 183: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/183.jpg)
Real-Time Constraints
![Page 184: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/184.jpg)
Real-Time Constraints
Must not block by• Allocating memory• Taking a mutex or waiting on a semaphore• Reading from a file• Logging• Calling libdispatch (notably async)• Calling ObjC and Swift runtimes• Doing anything else that involves memory allocation or a mutex
![Page 185: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/185.jpg)
Real-Time Audio
![Page 186: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/186.jpg)
Real-Time Audio
Audio Units• Modular, reusable signal generation/processing blocks
![Page 187: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/187.jpg)
Real-Time Audio
Audio Units• Modular, reusable signal generation/processing blocks
You can host them• System built-in units• Units chosen by user
![Page 188: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/188.jpg)
Real-Time Audio
Audio Units• Modular, reusable signal generation/processing blocks
You can host them• System built-in units• Units chosen by user
Build your own• As plug-ins (extensions)• Registered privately to your app
![Page 189: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/189.jpg)
Audio Units: Components
![Page 190: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/190.jpg)
Audio Units: Components
AudioToolbox maintains a system registry
![Page 191: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/191.jpg)
Audio Units: Components
AudioToolbox maintains a system registryComponent key: Type/subtype/manufacturer (4-character codes)
![Page 192: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/192.jpg)
Audio Units: Components
AudioToolbox maintains a system registryComponent key: Type/subtype/manufacturer (4-character codes)Audio Component types• Audio Units: Input/output, generators, instruments, effects, converters• Codecs: Encoders, decoders
![Page 193: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/193.jpg)
ImplementationsAudio Units: Components
![Page 194: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/194.jpg)
ImplementationsAudio Units: Components
Audio Unit application extensions (macOS, iOS)
![Page 195: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/195.jpg)
ImplementationsAudio Units: Components
Audio Unit application extensions (macOS, iOS)Component bundles (macOS only)
![Page 196: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/196.jpg)
ImplementationsAudio Units: Components
Audio Unit application extensions (macOS, iOS)Component bundles (macOS only)Inter-app audio nodes (iOS only)
![Page 197: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/197.jpg)
ImplementationsAudio Units: Components
Audio Unit application extensions (macOS, iOS)Component bundles (macOS only)Inter-app audio nodes (iOS only)Registered at runtime
![Page 198: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/198.jpg)
ImplementationsAudio Units: Components
Audio Unit application extensions (macOS, iOS)Component bundles (macOS only)Inter-app audio nodes (iOS only)Registered at runtimeApple built-in
![Page 199: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/199.jpg)
Audio Input/Output Units
![Page 200: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/200.jpg)
Audio Input/Output Units
Most commonly used
![Page 201: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/201.jpg)
Audio Input/Output Units
Most commonly usedPreferred higher-level interface to the system’s basic I/O path
![Page 202: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/202.jpg)
Audio Input/Output Units
Most commonly usedPreferred higher-level interface to the system’s basic I/O pathAUAudioUnit (AudioToolbox/AUAudioUnit.h)• Modern interface to version 2 Audio Units: AUHAL, AURemoteIO
![Page 203: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/203.jpg)
DemoUsing AUAudioUnit
![Page 204: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/204.jpg)
Effects, Instruments, and Generators
![Page 205: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/205.jpg)
HostingEffects, Instruments, and Generators
Also AUAudioUnitChain render blocksParametersAU-provided views
![Page 206: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/206.jpg)
Writing your ownEffects, Instruments, and Generators
![Page 207: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/207.jpg)
Writing your ownEffects, Instruments, and Generators
Within your own app• Subclass AUAudioUnit• +[AUAudioUnit registerSubclass: … ]
![Page 208: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/208.jpg)
Writing your ownEffects, Instruments, and Generators
Within your own app• Subclass AUAudioUnit• +[AUAudioUnit registerSubclass: … ]
To write a plug-in for distribution• Also AUAudioUnit subclass• Packaged as Audio Unit Extension
![Page 209: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/209.jpg)
DemoAudio Unit Extensions
Torrey Holbrook Walker The Demonstrator
![Page 210: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/210.jpg)
For more informationAUAudioUnit
Audio Unit Extensions WWDC 2015
![Page 211: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/211.jpg)
MIDI
![Page 212: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/212.jpg)
Music Instrument Digital InterfaceMIDI
![Page 213: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/213.jpg)
Music Instrument Digital InterfaceMIDI
Task Preferred APIs
![Page 214: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/214.jpg)
Music Instrument Digital InterfaceMIDI
Task
Play standard MIDI file(or your own sequences)
Preferred APIs
AVAudioSequencer
![Page 215: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/215.jpg)
Music Instrument Digital InterfaceMIDI
Task
Play standard MIDI file(or your own sequences)
Control software synth
Preferred APIs
AVAudioSequencer
AVAudioUnitMIDIInstrument AUAudioUnit
![Page 216: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/216.jpg)
Music Instrument Digital InterfaceMIDI
Task
Play standard MIDI file(or your own sequences)
Control software synth
Communicate with MIDI hardware(e.g., USB, Bluetooth), MIDI network
Preferred APIs
AVAudioSequencer
AVAudioUnitMIDIInstrument AUAudioUnit
CoreMIDI
![Page 217: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/217.jpg)
Music Instrument Digital InterfaceMIDI
Task
Play standard MIDI file(or your own sequences)
Control software synth
Communicate with MIDI hardware(e.g., USB, Bluetooth), MIDI network
Inter-process real-time MIDI
Preferred APIs
AVAudioSequencer
AVAudioUnitMIDIInstrument AUAudioUnit
CoreMIDI
CoreMIDI
![Page 218: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/218.jpg)
Recap
CoreAudio and Drivers
Application
![Page 219: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/219.jpg)
Recap
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AVAudioEngine
![Page 220: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/220.jpg)
Recap
CoreAudio and Drivers
Application
AudioToolbox
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AVAudioEngine
AUAudioUnit
![Page 221: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/221.jpg)
Recap
CoreAudio and Drivers
Application
AVFoundationAVAudioPlayer
AVAudioRecorder
AVPlayer
AVAudioSession
AudioToolbox
AUAudioUnit
AVAudioEngineCoreMIDI
![Page 222: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/222.jpg)
More Information
https://developer.apple.com/wwdc16/507
![Page 223: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/223.jpg)
Related Sessions
Enhancing VoIP Apps with CallKit Mission Thursday 5:00PM
Advances in AVFoundation Playback Mission Wednesday 9:00AM
Audio Session and Multiroute Audio in iOS WWDC 2012
What’s New in Core Audio WWDC 2014
AVAudioEngine in Practice WWDC 2014
What’s New in Core Audio WWDC 2015
Audio Unit Extensions WWDC 2015
![Page 224: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/224.jpg)
Labs
Audio Lab Graphics, Games, and Media Lab D Friday 3:00PM
![Page 225: Media #WWDC16](https://reader036.fdocuments.net/reader036/viewer/2022082210/58a2d5281a28ab6d6a8b6fb1/html5/thumbnails/225.jpg)