Vienna 4/28/2006 1

Click vs. click-click vs. blink-click: Factors influencing human sound localization in the horizontal plane

Norbert Kopčo

TU Košice Dept. of Cybernetics and AIBoston University Hearing Research Center

Dartmouth College Center for Cognitive Neuroscience

Vienna 4/28/2006 2

Intro: Sound localization

3-dimmensional: azimuth, elevation, distance

depends on:• stimulus type: spectrum, temporal aspects• environment: anechoic, reverberant• source movement: static, dynamic • presence of other stimuli (auditory or visual)• a priori knowledge / expectations about the scene

Vienna 4/28/2006 3

Effect of additional stimuli

The extra sound can act as a:

Masker – localization worse

Adaptor – localization biased (Attraction/Repulsion)

Real sound (of which the target is a reflection) – localization worse/suppressed

Perceptual stream of which the target is or is not a part

Cue – localization better (doesn’t have to be auditory)

Anchor – change localization strategy

Vienna 4/28/2006 4

Effect of additional sounds

Temporal relations studied previously:

Extra sound precedes target by:• 10 secs to mins Adaptation/Repulsion• 50 msecs to 1 sec Adaptation/ reflections• 4 – 40 msecs Precedence• Concurrent sounds Adaptation/Repulsion• Inverse order Backward masking

Vienna 4/28/2006 5

Auditory Pathway and Spatial HearingCochlea – peripheral filtering and

neural coding

Olivary complex – processing of binaural information

Thalamus (Inferior Colliculus) – integration, modulation detection

Auditory Cortex – auditory object formation, figure/ground separation, ASA

Posterior Parietal Cortex – supramodal spatial representation & attentional modulation

Vienna 4/28/2006 6

Current goal

Begin to understand auditory localization in a more complex scene:

- when target is preceded by another identical sound/s from a known location that the listener should ignore (Exp 1)

- when target is preceded by visual or auditory cue that allows the listener to direct spatial attention (Exp 2)

- when a concurrent visual stimulus induces a shift in auditory perception / ventriloquism (Exp 3)

Vienna 4/28/2006 7

Experiment 1Perceptual and central effects in sound localization

with a preceding distractor

(aka Click vs. Click-click vs. Click-click-click-click...)

Collaborators

Barbara Shinn-Cunningham, Virginia Best Hearing Research Center

Boston University

Vienna 4/28/2006 8

Exp 1 - Preceding distractor: Intro

Several preceding studies indicated that preceding stimulus influences localization at SOAs of several hundreds milliseconds (Kopco et al., 2001, Perrott and Pacheco, 1989)

Goal:- Characterize this influence (bias and std.dev. in responses)- Determine its cause. Candidates:

- short-term adaptation in brainstem representations- reverberation suppression and acoustics- strategy- perceptual organization- attention: focused away from distractor location

Vienna 4/28/2006 9

Exp 1 - Preceding distractor: Hypotheses

Peripheral factors will have short-term effects

Central factors will influence results at longer separations

Effect of reverberation can be separated by comparing performance in anechoic and echoic rooms

Effect of perceptual organization can be addressed by modifying the stimuli

Vienna 4/28/2006 10

Exp 1 - Preceding distractor: Methods

Anechoic room or a classroom

Blocks of trials with fixed distractor location

Trials with SOAs of 25,50,100,200 or 400 ms interleaved w/ no distractor trials

Seven subjects

Vienna 4/28/2006 11

Exp 1 - Preceding distractor: Results

Complex pattern of biases and standard deviation effects observed

Four main effects in terms of bias discussed

Bias 1: Lateral bias for frontal targets and lateral distractor in room

Vienna 4/28/2006 12

Exp 1 - Preceding distractor: Results – Bias 1

Largest effect

Strongest at

short SOAs

No comparable

effect of frontal

distractor

ROOM

Vienna 4/28/2006 13

Exp 1 - Preceding distractor: Results – Bias 1

Effect eliminated in anechoic room has to do with reverberation.

Acoustic or neural interaction?

ANECHOICROOM

Vienna 4/28/2006 14

Exp 1 – Bias 1: Perceptual organization

Effect not due to acoustics because correct representation is available

ROOM: click-click-click-click … click

ROOM: Click-click

Vienna 4/28/2006 15

Exp 1 – Bias 1: Standard deviation

The largest increase in standard deviation corresponds with the largest bias

Neural suppression along with reflections

BUT: Why only lateral distractor?

Vienna 4/28/2006 16

Exp 1 - Preceding distractor: Results – Bias 2

Targets in the middle of the range are attracted by the distractor, independent of:

- Environment- Distractor location- Only at short

SOAs

Interactions in low-level spatial maps (brainstem)

Vienna 4/28/2006 17

Exp 1 - Preceding distractor: Results – Bias 3

Lateral targets are repulsed by lateral distractors

- Independent of SOA- Independent of

environment

Probably central effect: e.g., change in response strategy, using distractor as an anchor w/ known location

Not in front because of higher resolution.

Vienna 4/28/2006 18

Exp 1 - Preceding distractor: Context

There is bias also in the no-distractor responses

The bias is always away from the non-present distractor

Because the runs were interleaved, this bias had to build up anew during each run

Vienna 4/28/2006 19

Exp 1 - Preceding distractor: Context

Difference in no-distractor responses in the frontal and lateral distractor context

- Is independent of azimuth- Grows over time- Slightly stronger for the 8-click

train contextContextual plasticity on time

scale of minutesSimilar to effects of long-term

exposureEither due to bottom-up factors

(distribution of stimuli) or top-down factors (focusing away from distractor)

Con

text

ual b

ias

Vienna 4/28/2006 20

Exp 1 - Preceding distractor: Summary

A preceding distractor coming from a known location

- Induces a complex pattern of biases - Over a range of time scales- Probably caused at different stages

in the spatial auditory processing pathway

Vienna 4/28/2006 21

Experiment 2Modality-dependant attentional control in human

sound localization

(aka Click vs. Beep-click vs. Blink-click)

In collaboration w/ students

Beáta Tomoriová, Rudolf Andoga, Martin BernátPerception and Cognition LabTechnical University, Košice

Vienna 4/28/2006 22

Exp 2 – Uni-/Cross-modal attention: Intro

Several studies explored the question whether directing automatic or strategic attention by an auditory cue can improve sound localization (Spence & Driver, 1994; Sach, 2000; Kopco & Shinn-Cunningham, 2003)

Results: improvements in RTs (Spence&Driver), but small (Sach) or no (Kopco) improvements in performance

Possible reason: the SOAs too short to orient attention

Goal:- determine whether attentional effects occur at longer SOAs- compare the effect of a visual and auditory cue

Vienna 4/28/2006 23

Exp 2 – Uni-/Cross-modal attention: Hypotheses

No effect of automatic attention (previous studies)

Strategic attention will affect performance at long SOAs

Effect modality-independent because spatial cuing very coarse (only left vs. right)

Vienna 4/28/2006 24

Exp 2 – Uni-/Cross-modal attention: Methods

Virtual auditory environment

Target – broadband click

Cue indicates side of target:- visual (arrow on a computer

screen)- auditory (monaural tone)- SOA: 400, 800, 1600 ms- Informative: 100%, 80%,

50% validity- analysis: mean and s.d. in

responses

90

45

1530

60

90

45

1530

60

Vienna 4/28/2006 25

Exp 2 – Uni-/Cross-modal attn: Results - bias

Mean effect of auditory cue (averaged across target azimuth):- Invalid cues cause medial bias, fairly independent of SOA- Valid cues cause similar medial bias

Vienna 4/28/2006 26

Exp 2 – Uni-/Cross-modal attn: Results - bias

When cue modality is visual:

- Invalid cues cause medial bias, similar to the auditory cues

- Valid cues cause lateral bias that grows with SOA

Modality through

which expectation of

the target location is

controlled influences

the perceived location

Vienna 4/28/2006 27

Exp 2 – Uni-/Cross-modal attn: Results – s.d.

Effect in terms of standard deviation:

No effect of auditory cue

Visual cue never improves performance, but invalid cue at 1600 ms increases s.d.

Summary:

Cuing doesn’t improve performance

Expectation of side of stimulus induces bias in a modality dependent way

Might have something to do with the coordinate systems in which visual and auditory space are represented

Vienna 4/28/2006 28

Experiment 3Behavioral examination of the auditory spatial

coordinate system using the ventriloquism effect

Collaboration

Jennifer GrohCenter for Cognitive Neuroscience, Dept of Psychological

and Brian Sciences, Dartmouth College

Barbara Shinn-Cunningham, I-Fan LinBoston University

Vienna 4/28/2006 29

Exp 3 – Coordinate system of auditory space: Intro

Mullette-Gillman et al. (2005):

Does the visual and auditory spatial coding have the same reference frame in the monkey parietal cortex?

Is the frame head-centered (as in auditory periphery) or eye-centered (as in visual periphery)?

This is an issue only for primates and animals that can move their eyes (not barn owls)

Result: some neurons in PPC A-only, some V-only, some AV, some head-centered, some eye-centered

Vienna 4/28/2006 30

Exp 3 – Coordinate system of auditory space: Intro

Here, use the ventriloquism effect to address a similar question behaviorally in monkeys and in humans:

Is the coordinate system at which the auditory behavioral responses are determined head- or eye-centered?

Method:

1. Induce a local shift in the auditory spatial map for a fixed eye position.

2. Move eyes to a new position.

3. If the region of the shift doesn’t change head-centric

4. Otherwise, eye-centric coordinate system

Vienna 4/28/2006 31

Exp 3 – Coordinate system of auditory space: Method

Study performed in humans and in monkeys

Monkey data here

Vienna 4/28/2006 32

Exp 3 – Coordinate system of auditory space: Results (preliminary)

Difference between positive (rightward) and negative (leftward) shifts induced in the central region with left fixation point and generalization testedwith right fixation point

Result:

Induced shift generalizes

on the right side

No shift in bias due to change

in fixation point Head-centric coordinate system

-30 -24 -18 -12 -6 0 6 12 18 24 30-6

-4

-2

0

2

4

6Diff (M+SE) between responses w/ RIGHT and LEFT shift. Trials:0-2000. 4 repeats.

FIX -8,-16

FIX 8,-16

Vienna 4/28/2006 33

Overall summary

Three experiments explored various aspects of horizontal sound localization

Understanding is limited even in the simple auditory scenes studied

Need follow-ups to clarify results

Vienna 4/28/2006 34

Acknowledgements

US National Institutes of Health (PIs: Shinn-Cunningham and Groh)

US National Academy of Sciences (Shinn-Cunningham, Kopčo)

Slovak Scientific Grant Agency (Kopčo)

Vienna 4/28/2006 35

Distance perception in reverberant environments- is consistent experience necessary for accurate distance perception?

- also, studies looking at other parameters (mono- vs. binaural, anechoic vs. reverberant, real vs. simulated environments)

“Room learning” and calibration to its acoustic properties- is localization accuracy and “room learning” affected by changes in listener

position in a room?- do speech perception mechanisms calibrate to different acoustic

environments?

Spatial release from masking- effect of signal and masker location on detectability/intelligibility of pure tones,

broadband non-speech stimuli, and speech in anechoic and reverberant environments

Overview of recent studies of binaural and spatial hearing