MANAGED IMMERSIVE AUDIO COMMUNICATION

Alfonso TorrejonPart time MSD

Aim

• Explore the audio role, examine advantages and limitations, needs and value associated to immersive 360° panoramic video technologies that could be applied to educational, training and job related performance evaluation

• evaluation of different immersive audio environments with a sample of users

Basic of immersive video

Outwardly Pointing Cameras (Bells Labs)

Live use: Microsoft: RoundTable Polycomm: CX5000

VOD: Immersive Media

Panoramic recording with vertical lens

Basic of immersive video

The Vision: http://www.superlectures.com/icassp2011/lecture.php?lang=en&id=207

Basic of immersive video

http://www.youtube.com/watch?v=HlxIWmA4pMk&feature=haxa_popt00us13#t=0m36s

•Binaural audio recording

•Wave field audio

Immersive audio

Immersive audio

• Audio work available by DT labsSoundScape Renderer

Immersive audioExperiments:

• Binaural audio single source: it will require to DIY dummy heads for recording audio (initially) and then to stream alive (through the FMS server) •Wave field techniques and crosstalk for speakers.•Illusory self-motion•Immersive video & VR for added visual stimuli to the project•Multiple combined binaural audio sources to simulate real environments•Multiple-talkers filtering techniques•SoundScape, training and development

Expected milestones:

• 02/12: binaural dummy heads fabricated and tested.

• 02/12: Flash Media Server loaded, setup and customized for experiments.

• 03/12: Applications for network audio streaming for multiple-sources binaural streams.

• 03/12: 360° panoramic video stream deployment onto the FMS server

• 04/12: SoundScape deployment and programming.

Immersive audio

Managed Immersive audio

Goals:• Ability to add audio hot spots dynamically generated onto real-time videos, either panoramic or not, and controlled by the viewer.

•Exploring binaural techniques for multiple sound sources and receivers while in motion and in complex environment.

•Experiment with interferences, unexpected audio and multiple talkers.

•Experimenting with ultrasound for directional audio combining with binaural techniques.

Managed Immersive audio

Managed Immersive audio

Q & A