Keeping the visual world stable: The importance of spatiotemporal continuity in vision

Árni Kristjánsson

1 University of Iceland, 2 University College London3 UC Berkeley

The visual system clearly likes consistency

o Learning in the deployment of transient attention (Kristjánsson et al. 2001; Kristjánsson & Nakayama, 2003)

o Priming of features and target position in visual search tasks (Maljkovic & Nakayama, 1994; 1996; Kristjánsson et al., 2002)

o Probability cueing of location (Geng & Behrmann, 2005; Miller, 1988)

o Statistical learning (Fiser & Aslin, 2001; Turk-Browne et al., 2005)

o Contextual cueing (Chun & Jiang, 1998)

oPriming determines attentoinal choice (Brascamp, Blake & Kristjansson, 2011)

No “top-down” control

Maljkovic & Nakayama, 1994

Sensitivity increases as the same search is repeated Sigurdardottir, Kristjánsson & Driver Visual Cognition, 2008 Error rates perhaps not a very precise measure of this...

Followed by a random-dot pattern mask

Presented for 200 ms

Is a target present or not?

Dependent measure: Accuracy

Measured sensitivity (d’) and response bias (c).

Changes in response criteria cannot account for the priming pattern

As number of repetition of target orientation increases, sensitivity to target as measured by d’ increases while the response criterion remains relatively constant

d-pr

ime

Crite

rion

(c)

One function of priming?

Imagine yourself at a party with someone that you have a crush on, or are even in love with.

You seem to be constantly aware of where that person is and your gaze is repeatedly drawn towards the dashing red dress/shirt that he or she is wearing, or their shining black hair in such fine contrast to their paler face, despite your best efforts to not look too eager.

This person is an example of a stimulus that is the focus of your attention and matters very much to you.

Recent research has unveiled how our attention and gaze seem to be automatically drawn towards those features that we have recently attended to and are important to us, such as the red dress or dark hairof our object of affection.

6

What are the neural correlates of priming?Kristjansson et al., JoCN, 2005

We tested performance of patients suffering from hemispatial neglect following damage to parietal cortex

Hemispatial neglect: Patients tend to ignore stimuli in the visual hemifield opposite to the affected hemisphere of the brain. Cross out all

lines

Line bisectionCopying task

Experimental paradigm:

+

Find the oddly coloured diamond and respond whether the cut-off is at the top or at the bottom

Stimuli presented for 200 milliseconds (dependent measure: accuracy). On 20% of trials no target was presented and the observers had the option of responding that no target was present

Stimuli present until response (dependent measure: response time)

Two paradigms tested:

Response timemeasure:

Priming more orless intact in thepatients with theattentional disorder

But their response times in the affected hemifield are much longer...

Priming cannot be solely attributed to the attentionalmechanisms disrupted in hemispatial neglect

Accuracy Measure (performance in the left hemisphere):

Position priming:

Interesting dissociation between position and colour primingfrom missed targets (brief presentation):

When left target missed, no subsequent position priming (i.e. no improvement in performance)

When search array is presented briefly, neglect patients often miss a target that appears in left hemifield (“extinction”)

Graph showspercent correcton current trial as a function of what took place on last trial for performance in left hemifield

Last trial

Colour priming

On the other hand there is considerable improvement in performance as the colour of a missed target is repeated

Position priming seems to require the awareness of the target, whereas colour priming can take place even when the target was missed

Graph showspercent correcton current trial as a function of what took place on last trial in left hemifield

Functional Magnetic Resonance Imaging (fMRI)

Neural correlates of priming in visual search (Kristjansson et al. Cerebral Cortex, 2006)

+

Previous research in many priming paradigms has shown decreases in the BOLD signal with the repetition of stimuli -repetition suppression (e.g. Kourtzi et al. 2003)

Same task as before (search array presented for 200 ms)

Regions showing significant repitition suppression as target colour is repeated

Regions showing significant repitition suppression as

target position is repeated

Results show significant repetition suppression for a number of diverse regions in the brainAreas common for priming of colour and position:

Intraparietal sulcusFrontal eye fieldsAnterior cingulate

...among others, areas associated with the operation of visual attention

This may reflect that the primed feature or object receives prioritized processing through the

operation of visual attention - feedback?

Also; repetition suppression in early visual cortex as target colour - or position, are repeated.

Anterior fusiform

Areas showing significant suppression for colour repetition only:

Area in fusiform cortex often associated with colour processing (“V4”)

Areas showing significant suppression for position repetition only:

Right Supramarginal gyrus and Right Inferior frontal gyrus

May reflect the regions connected with the “exogenous” attentional network proposed by Corbetta & Shulman (2003).

AREAS COMMON TO POSITION & COLOR REP.

Finally, repetition of both position and color together (and hence repetition of the global “Gestalt”of search display) was associated with distinct decreases in the left FG, in otherwords only found when both color and position are repeated

ACC & IPS

Visual crowding places a fundamental limit on object recognition in cluttered scenes.

VISUAL CROWDING

David Whitney , Dennis M. Levi

Trends in Cognitive Sciences Volume 15, Issue 4 2011 160 - 168

Critical distance for crowding

Current aims• Views of natural scenes unfold over time, and objects of interest that

were present a moment ago tend to remain present. • Most studies of crowding suffer from the limitation that they typically

involve static scenes. - The role of object continuity in crowding is therefore

unaddressed.

• We investigated intertrial effects upon crowding in visual scenes investigating effects of whether objects remain constant on consecutive visual search trials.

Experiment 1

Experiment 1Priming effect upon crowding

Experiment 1 -Percent correct as a function

of flanker distance

PERC

ETN

CO

RREC

TCritical distances as a function of repetition

Experiment 2 – The role of distractor repetition

Similar design, except three colors swapping roles from distractor to target

Experiment 2 - critical distance decreaseswith priming

Leftwards shifts in psychometric functions with repetition of target and distractors

6 subjects - average

Experiment 3 – flankers more eccentric than search items

Flankers MORE effective when more eccentric than less eccentric than targets (Bouma, 1970; Banks et al., 1977; Petrov, et al., 2007; Farzin, et al., 2009

- News to me…

Experiment 3: a repeat of experiment 2, except that that the flankers were more eccentric than search items

Experiment 3 – flankers more eccentric than search items

• Crowding is considerably diminished when objects remain constant on consecutive visual search trials.

• Both constant target and distractor identity decrease the critical distance for crowding from flankers.

-Their effects independent and additive

• More generally, our results show how object continuity through between-trial priming releases objects otherwise unidentifiable from crowding.

• Crowding, although a significant bottleneck on object recognition, can be strongly mitigated by statistically likely temporal continuity of objects.

Crowding depends not only on what is momentarily present, but also on what was previously attended.

Conclusions Part 1

The emotional robot: Facilitated reflexive attentional orienting under emotional arousal.

 .

Árni KristjánssonSchool of Health Sciences, University of Iceland

Berglind ÓladóttirDepartment of Psychology, University of

IcelandSteven B Most

University of Delaware

Fear

• Emergency reaction:- Sympathetic nervous system produces adrenaline

• fight or flight response

• Automatic reflex

James-Lange theory

• We become scared because we run; sad because we cry

• The physiological response is key

Schachter & Singer

• Two factor theory of emotion: “Cognitive arousal theory”

• "people search the immediate environment for emotionally relevant cues to label and interpret unexplained physiological arousal."

• The physiological response is very important but so is interpretation of the environment.

Agreement that the physiological response is very strong and automatic

– what are the effects upon cognition?

Attentional Rubbernecking (Most et al. 2005)

• The effects of presenting an emotion-inducing picture on performance on a picture identification task.

• Target: Picture oriented 90° from upright(present on 50% of trials)

Attentional Rubbernecking (Most et al.. 2005)

Results:Performance when the lag between pictures was 2 was much worse following negative pictures than scrambled or neutral

“Attenional rubbernecking” – attention “sticks” to the emotion inducing picture.

“The naked truth”: Positive, arousing distractors impair rapid target perception

• Rubbernecking not unique to negative images

• Erotic pictures: 20 men tested

• Bradley, Codispoti, Sabatinelli, & Lang, (2001): Men rate erotic images more highly and show a stronger physiological response.

Current goals

• Can priming of visual attention and perception help in overcoming these effects of emotion inducing pictures?

• Do emotion inducing pictures cause automaticity of behavior? Will the primed/prepotent response dominate behavior?

• Emotion -> Reflexive behaviorPriming ≈ Reflexive use of prepotent response?

Experiment 1

20 participants

Respond whether notch at top or bottom of target diamond.

Measure: Inverse efficiency (RT/proportion correct)- Incorporates speed and accuracy in the same measure.

Experiment 1 - results

Inverse efficiency is larger following neutral and inverted pictures than emotion-inducing ones

Emotion-inducing pictures clearly have an effect on attentional orienting

But priming effects are stronger following the emotion inducing pictures (Significant interaction: F(4,76) = 6.48, p = .001)

Experiment 2

Effects upon sensitivity?

Search stimuli briefly presented; followed by a mask.

Measure: d’ for 2AFC:d’FC = 2Z(pc), so d’FC = (√2)*d‘

Experiment 2 - results

Overall, d’FC is lower for emotional pictures than neutral or inverted

d’FC increases with color repetition

Priming effects from repetition of target and distractor color are stronger for emotional pictures (significant interaction: F(6,54)

= 4.47, p = .001)

Experiment 3

In expt 2: Some signs of a reversal: Performance better following emotional pictures when the primed/prepotent response is the correct one

Used “largest” priming effects we know (Wang, Kristjánsson & Nakayama, 2005)

Only neutral and negative pictures

Otherwise similar to expt. 1

Experiment 3 - results

Results similar as before

Priming reduces difference between neutral and emotional pictures

Significant interaction (F(4,44) = 10.76, p < .001)

Experiment 3 – Results according to size of effect of emotion (median split: small or large emotion effect)

Difference between those who show small or large effect of scary pictures

Small effect Large effect

Significant differences:

Conclusions

• Following emotion-inducing stimuli, the target from previous trials is preferentially attended (more so than following neutral pictures)

• Emotional pictures induce reflexive attending to primed stimulus and facilitate prepotent response

• Visual search is; overall; impaired following the presentation of emotional pictures, but if the task involves performing the primed, prepotent response, this can be overcome

- in extreme cases actually leading to improved performance

Conclusions (1)

Short-term priming in vision allows you to reorient quickly to previously viewed items that are important for immediate behaviour (such as keeping track of your child in a playground)

Primign has a dominating influence upon attentional function

Conclusions (2)

Evidence from fMRI: Neurological structures often associated with attentional mechanisms are involved in the priming effects for both colour and position (e.g. IPS & FEF)

May connect with visual areas and modulate theirresponses

Intact priming seen for neglect patients when priming stimulus is detected