Chapter 12 – Auditory Localization and Organization

Localization of sounds

Localizing Visual stimuli in vertical and horizontal dimensions – Too easy!! Visual stimuli at different places activate different locations on the retina. Each different place on the retina is connected to a different area of the cortex. So it’s kind of easy to see how the brain can locate different visual stimuli – by noting where the activity occurs on the retina, LGN, area V1.

Localizing in the 3rd dimension – depth – is more difficult. It requires “neural deduction” based on monocular cues and comparison of images in the two eyes – binocular disparity.

The issues involving perceiving sound are like those involved in using binocular disparity to perceive depth or distance . . .

A sound originating from the left activates ALL the receptors on both the left cochlea and right cochlea.Similarly, a sound originating from the right activates ALL the receptors on both the left cochlea and right cochlea.

So, perceiving location requires a comparison of the input to the two ears.

Some terms . . .

Auditory Locatization and Organization - 1 3/21/5

Azimuth – location on a circle around the head

Elevation – location above or below the head.

0◦

90◦ right

180◦

270◦

Or 90◦ left

Azimuth – Location in a horizontal circle around the listener.

Elevation – Location in a vertical circle around the listener.

0◦ is immediately in front of us.

0◦

45◦

Cues for localization in the azimuth

Sound localization in the azimuth involves comparison of the sounds in the two ears.

The primary cues are binaural cues: Specific differences in the sounds between the two ears.

1. Interaural Time Differences (ITDs): G9 p 291

Also called time of arrival differences

A sound originating on the left will reach the left ear slightly sooner than it reaches the right ear.Vice versa for sounds originating on the right.

Maximum difference in time of arrival is associated with a sound on immediately to the left of the left ear or the right of the right ear. . . .

The maximum difference associated with a sound from one side is .000600 seconds, 600 millionths of a second, or 600 microseconds.

Time of arrive differences are by themselves enough to allow precise localization of sounds. Careful experiments in which the only cue for location was time of arrival have demonstrated that persons can accurately locate sounds based on this cue only.

The threshold for ITD is about 100 µsec – sounds which differ in ITD by less than 100 µsec are perceived to be directly in front of us 50% of the time.

Interaural time differences (ITDs) are most important for localization of low frequency sounds.

Auditory Locatization and Organization - 2 3/21/5

600 µsec600 µsec = .000600 sec. = 600 millionths of a second

2. Interaural Level Differences (ILDs) G9 p 291

Differences in Level (intensity) between the ears

Sounds which originate on the left will tend to be more intense at the left ear.Sounds which originate on the right will tend to be more intense at the right ear.

Our perception corresponds to reality . . .

All other things equal, a sound localized to the left will be more intense in the left ear. A sound on the right will be more intense in the right ear.

But: How different the intensities will be in the two ears depends on the frequency of the sound.

A sound originating on the left has two ways of getting to the right ear (assuming no reflective surfaces) . . .

a. bending around the head. b. going through the head.Low frequency sounds High frequency sounds must go throughbend easily around the head the head, reducing intensity.

The result is the following:

Frequency Maximum Intensity difference between ears

300 Hz 1 db (barely noticeable) Low frequency sound easily bend around the head.

4000 Hz 5 – 10 db (noticeable)

10,000 Hz 10-20 db (quite noticeable) High frequency sounds are blocked by the head.

Auditory Locatization and Organization - 3 3/21/5

Ear in “shadow”

Graphically . . .

So interaural level differences provide good information on location of high frequency sounds.

But interaural intensity differences provide little information on location of low frequency sounds

This is one of the reasons that some persons with high quality sound systems have only one speaker designed for very low frequency sounds. It’s because we are less able to identify the location of such sounds, so there is no need for two in a stereo system.

Auditory Locatization and Organization - 4 3/21/5

Perceiving Elevation G9 p 293

The Cone of Confusion

ITDs and ILDs give great information about localization in the azimuth.

Alas, they don’t provide much information about elevation.

The collection of locations for which ITD and ILD are equal is called the cone of confusion.

The locations on the cone differ in elevation.

A and B have exactly the same ITD and ILD.

This means that ITD and ILD are of little use in determining elevation.

Spectrum cues used to judge elevation and whether sound is in front of us or in back

The shape of the pinna differentially attenuates different frequency components of the sound stimulus.

This shaping of the sound stimulus creates information for the location of the sound called the spectral shape cue – a cue for location based on the shape of the spectrum. This cue is particularly important for perception of the elevation of the sound stimulus.

The shaping of the sound spectrum by the pinna is specific to each individual ear.

This means that using the shape of the sound spectrum to perceive elevation must be learned.

It also implies that if the shape of an adult’s pinna were changed, that adult would have difficulty locating the elevation of sounds they could previously locate very easily.

Note that the sound from above is generally more intense n the ear canal.

Note also that the sound from below lacks intensity in the 6-8 KHz frequency range relative to the sound from above.

Auditory Locatization and Organization - 5 3/21/5

Spectra of same sound from two elevations

Influence of shape of the pinna in judging elevation

Figure 12.9 G9 p 295 shows the effects of wearing an artificial pinna on ability to localize sounds.

The crossings of the blue grid are where sounds were presented.The red dots are where the sounds were reported to be located.Note the close correspondence for this control condition.

On the first day that observers wore an implant, their reports were horribly inaccurate in the vertical dimension, although pretty nearly correct in the horizontal dimension.

After 5 days of wearing the implant, observers’ reported locations in the vertical dimension became more accurate.

After 19 days wearing the implant, observers’ reports of both vertical and horizontal location were pretty accurate.

Immediately after removing the implant, observers’ reports were still quite accurate, suggesting that they had not “lost” an ability, but simply gained a new ability.

This study suggests that the timbre (the experienced frequency spectrum) of the sound is a cue for that sound’s elevation

Auditory Locatization and Organization - 6 3/21/5

Using timbre to determine whether a sound is in front of us or in back.

The pinna attenuates the high frequency components of sounds located in back of the head.

This means that we can tell when a familiar sound is behind us because it sounds slightly muffled compared to the way it sounds when it’s in front of us.

A study demonstrating the effect of the ear flap . . .

Batteau (1967) as reported in L&S, p. 424.

Batteau created artificial pinnae out of plaster and put them on an artificial head. They then put microphones in the hole in the artificial pinna where the ear drum would be.

Condition 1: Recorded sounds played in different locations around the head with the plaster pinnas. Recall that there was a microphone in each plaster ear canal.

Condition 2: Recorded the same sounds played in different locations around a head in which microphones had been embedded without plaster pinnae. That is, the artificial head had no ears, just ear canals with microphones at the end of the canals.

Played back the recordings of those sounds to human observers.

Results:

Condition 1: Participants were able to identify locations of the sounds in front of and behind the artificial head.

Condition 2: Participants were not able to localize the sounds correctly.

Conclusion: The plaster, artificial pinnae muffled the sounds played behind the head in a way that was similar to how sounds are muffled by the real pinna.

Effect of differences between the two pinnae on location in azimuth

We’d be much better at locating sounds in the azimuth if our ears had different shapes.

Perceiving distance (Not in G9, from K&S p 296)

Familiarity – comparison of perceived sound with memories of similar sounds at different distances

Spectral analysis – High frequency sounds decrease in intensity over distance more than low frequency sounds. So we hear the bass guitar across the lake, but not the singer. We hear the rumble of thunder but not the high frequency components of a crack of lightning.

Ratio of direct to reflected sound – Nearer sounds have a different ratio of direct to reflected than far sounds.

Auditory Locatization and Organization - 7 3/21/5

Possible neural circuits for ITD and ILD – G9 p 295

Study of barn owls has provided evidence for existence of neurons that respond only to sounds presented in a particular location.

The superior olivary complex contains the medial superior olive, a collection of neurons that have been identified as responding to ITD.

Neurons have been found which respond only when one ear receives sound slightly before the other – to time of arrival differences – that is, ITDs.

From the web site of David Heeger, NYU.

Auditory Locatization and Organization - 8 3/21/5

A circuit for neurons responding to Interaural Time Differences. – G9 p 297

Issue: Design a neural circuit so that a neuron responds most actively when a sound arrives at the right ear before it arrives at the left ear.

A simple way to create time of arrival signaling is to have a longer path from neurons originating in one ear to the postsynaptic neuron than the path from the other ear. This is analagous to the motion detectors we studied a couple of chapters ago.

Sounds in front of usIf a sound strikes both L and R at the same time, L’s action potential will arrive at the end of its axon, followed by R’s. That’s because the action potential has further to travel in R than it does in L. Thus, the total release of neurotransmitter substance at MSO will be long-term, but low intensity, resulting in low probability of MSO neuron responding.

Sounds to the right of usSound strikes the right ear first, but the path from the ear to the MSO from the right ear is longer.So R’s action potential will reach the end of its axon at about the same time as L’s, causing a massive release of neurotransmitter, enough to cause the MSO neuron to fire.

Sounds to the left of usBut if a sound strikes L first, then strikes R shortly afterward, L’s action potential will reach the end of its axon first and much much later (in neural time) R’s action potential will reach the end of its axon, resulting in insufficient neurotransmitter to cause the MSO neuron to respond.

Note that in such circuits, the long path is from the ear closest to the sound. (In case you need to construct such a neural circuit.)

Play ..\..\..\MDBT\P312\Ch14AuditoryPhenomena\ITD Neuron.pptx (It’s in Class Videos)

Auditory Locatization and Organization - 9 3/21/5

Stimulus

Ears

Long axon from right cochlear nucleus

Short axon from left cochlear nucleus

Sound on the right

MSO

MSONeuron

RL

Back of head

The Jeffress Neural Coincidence Model – combining many of the above circuits.

Figure 12.12, G9 p 297

Each neuron – 1, 2, 3, etc except 5 – has a different length path to each ear.

Neuron 5 has an equal length path from each ear, so it will respond best to sounds from the front.

.

Neuron 3 has a long path from the right ear and a short path from the left ear, so 3 will respond best to sound that reach the right ear first.

Imagine 1000s of neurons like 1,2,3,4,5,6,7,8,9 above, each with a different length path to each ear. Each of those neurons will respond best to a sound at a particular location on the azimuth.Each curve is the firing rate of a neuron to different ITDs.

Auditory Locatization and Organization - 10 3/21/5

Mammalian Neurons

It appears that tuning curves of neurons in mammals are not as sharp as those of owls.

The gerbil curve is about twice as wide as the owl curve in Figure 12.14 p 298.

It’s been suggested that in mammals, there are fewer more broadly tuned neurons, ones that fire over a wide range of locations to the right and others that fire over a wide range of locations to the left.

The blue curve is of a neuron that responds above its base rate to sounds from the right and below its base rate to sounds from the left.

The left curve has the opposite response characteristics.

Note – each of these curves is like the response curve of an opponent processes neuron in the LGN.

In this model, the location of a sound is signaled by the pattern of responses of these two types of neurons . . .

Sound 1 is on the left- resulting in a large response of the L neuron, but a small response of the R neuron.

Sound 3 is on the right – resulting in a small response of the L neuron but a large response of the R neuron.

Auditory Locatization and Organization - 11 3/21/5

Forgettabout what’s below this box.

A Neural Circuit responding to Interaural Level Differences

(This one is almost too easy.)

Below is a simple-minded circuit to process intensity differences.

Sounds in front of usNeurotransmitter from L and R counteract each other, resulting in base rate activity of LSO. So a base rate of activity would signal that a sound is in front of us.

Sounds to the right Since the sound is more intense at R, R releases more excitatory neurotransmitter substance the L, making ILD more active. So through its heightened activity, ILD signals “sounds to the right”.

Sounds to the leftSince the sound is more intense at L, L releases more inhibitory neurotransmitter substance than R, reducing LSO activity. So through its lowered rate of activity ILD signals “sounds to the left”.

Obviously, the neuron ILD by itself could not completely identify the location of a sound, but its activity, in conjunction with that of other neurons responding to other aspects of the auditory scene would help signal sound location in the azimuth.

.

Auditory Locatization and Organization - 12 3/21/5

RLL

ILD

The auditory pathway to the auditory cortex –

Note that each primary auditory projection area (A1) is buried in a sulcus.

An important point to get from this is that there is a LOT of processing of the auditory signal in each of these structures.

A natural question is: What exactly are these structures doing?

As Yantis, p 325 says, “The exact functions served by the different subcortical structures and the different response patterns of their neurons are the subject of considerable ongoing research and debate, but it’s clear that these structures and the neural signal processing that takes place within them along the auditory pathways play a critical role in encoding, with exquisitely high fidelity, the rapidly changing stimuli that typically make up the auditory environment.”

Auditory Locatization and Organization - 13 3/21/5

Tonotopic maps are found in almost every cortical area.

An image of the initial processing of sound that emphasizes the tonotopic maps in the various areas . . .

Reference is journal.frontiersin.org

Red represents low frequency soundsBlue represents high frequency sounds

AN: Auditory NerveCN: Cochlear Nucleus

LSO: Lateral Superior OliveMSO: Medial Superior OliveMNTB: Medial Nucleus of the Trapezoidal Body – not mentioned in Y1

The point is that every area through which the auditory signal passes is organized tonotopically – by frequency.

Auditory Locatization and Organization - 14 3/21/5

Low Freq

High Freq

The auditory Cortex – Y1 p 327

The auditory cortex is located on the upper surface of the temporal lobe tucked into the lateral sulcus – the sulcus separating the temporal lobe from the frontal and parietal lobes.

A primary discovery associated with the auditory cortex is the tonotopic map – the neurons are organized by the frequency of sound to which they’re most sensitive.

Note that the maps reverse as you move through the auditory cortex – high to low; low to high; high to low

The region is separated into three subregions – the core, the belt, and the parabelt.

Neurons in the core respond best to pure tones.

Neurons in the belt and parabelt respond to more complex sound stimuli.

This is analogous to the relationships of visual neuron responses in V1 vs. V4, MT, and other cortical areas. The complexity of the stimuli to which neurons respond becomes greater, the farther you go from the initial projection area.

Auditory Locatization and Organization - 15 3/21/5

What and where pathways

Auditory cortex neurons that carry information about sound localization send their axons to the parietal lobe.

Auditory cortex neurons that carry information that might help in identifying what is in the external environment send their axons to the frontal lobe.

The auditory pathways meet the visual “what” and “where” pathways.

The “where” pathways meet in the parietal lobe (in the monkey).

The “what” pathways meet in the prefrontal cortex (again, in the monkey).

This, of course, means that there are neurons that receive both visual and auditory information – in the blue areas in the above figure.

Images of brain activity of a human performing either identification of sounds or location of sounds.

From Goldstein’s 8th Ed.

Auditory Locatization and Organization - 16 3/21/5

Architectural Acoustics G9 p 302

Reverberation – Delay and strength of echos

Definition: The time it takes for the reflections of the sound to become 60 decibels less than the original sound.

Play VL 12.14

Best is about 2 seconds for concert halls and 1.5 seconds for opera houses.

Best is 0.4-0.6 seconds for a classroom, about 1.0-1.5 seconds for an auditorium.

Intimacy time: The time between arrival of the sound directly from the source and the arrival of the first reflection. Like reverberation, but focusing on the time of the first reflection, not the total amount of reflected sound.

Best is about 20 milliseconds. (.020 sec).

Bass ratio: The ratio of low frequencies to middle frequencies of reflected sounds.

Best is a high ratio.

Spaciousness factor: The fraction of all the sound received by a listener that is reflected, as opposed to direct.

Best is high.

Signal to Noise Ratio: The decibel difference between the amplitude of the signal (voice in a classroom) vs. the amplitude of sounds without the signal – the sound of the heating system, surrounding cars, people in the hallway.

Need about 10-15 S/N ratio – but higher is better.

Auditory Locatization and Organization - 17 3/21/5

Auditory Scene Analysis – Y1 p 341

Identifying the objects in the auditory scene.

The different pieces of the auditory experience.

Example – Musical scenes . . .

American Woman by the Guess Who – (Play from 1:11 on . . .)..\..\..\MDBT\P312\Ch14AuditoryPhenomena\American Woman Whole Version by the Guess Who.mp4Taken from YouTube.

Turtle Blues – 2:56 – breaking glass..\..\..\MDBT\P312\Ch14AuditoryPhenomena\Turtle Blues from Cheap Thrills by Janis Joplin.mp4

The problem . . .

what we hear has multiple sources, each broadcasting its own waveform,

but

the sound waveform arrives as a single waveform, a mixture of all of the original waveforms.

How do we break up the composite waveform into its pieces, identifying the different sources so that we hear different things simultaneously?

Auditory Locatization and Organization - 18 3/21/5

This page skipped in Summer 2016

Factors affecting how we identify auditory “objects”.

These are analagous to the Gestalt Laws of Grouping

1. Location – aka spatial segregation.

Sounds from the same location will be more likely to be perceived as coming from the same source.

2. Onset Time – aka temporal segregation.

Sounds which begin at the same time are grouped together.

3. Pitch and timbre similarities – spectral segregation.

Play VL 12.3

Play VL 12.6

Harmonic coherence – Most sounds are fundamental + harmonicsSo having a sound which is not one of the harmonics makes it seem

separate.

Sigview – Load 200,400,500,800 vs 200,400,600,800 workspaces. Play each.

4. Auditory Continuity.

VL 12.9 / Web 12.10

5. Experience.

The two melodies – Three Blind Mice and Mary had a Little Lamb – are more easily identified by those familiar with the two melodies.

Web 12.11 / VL 12.11

Auditory Locatization and Organization - 19 3/21/5

This page skipped in Summer 2016

Hearing and Vision.

Sound influences what we see

Play VL 12.15 / Web 12.12

First, play with sound turned down.Then, play with sound turned on.

I’m not thrilled with the other two Web VLs.

Areas of the brain activated by both vision and sound. G9 p 312, Figure 12.34

Neurons in the monkey parietal lobe were recorded during auditory and visual stimulation.

The red areas are areas in which stimulation – auditory in (a), visual in (b) – resulted in high activity above the base rate.

The rightmost panel (c) shows that the two areas overlap considerably.

Auditory Locatization and Organization - 20 3/21/5

This page skipped in Summer 2016

Echolocation by persons who are blind.

Thaler et al (2011) study

The researchers recorded from the ear canals of experienced echolocators.So they recorded both the clicking sounds made by the echolocators and the echos of the clicks as those echos reached their ears.

They then played the recordings of the clicks and their echos back to the echolocators and to normally sighted volunteers and recorded cortical activity associated with what was heard.

They found . . .

1. Activity in the auditory cortexes of both.

2. Activity in the visual cortex of the echolocators, but not the sighted volunteer.

Visual cortex activity in echolocator, as he/she listened to the clicks and their echos.

Absence of visual cortex activity in normally sighted volunteer.

Auditory Locatization and Organization - 21 3/21/5