1

TEMPORAL DYNAMICS OF DECISION-MAKING DURING MOTION PERCEPTION

IN THE VISUAL CORTEX

(2008) Vision Research, 48, 1345-1373

Praveen K. Pilly Stephen Grossberg

2

Decision-Making?

Cognitive decision-making

Perceptual decision-making

3

Motivation

4

Main Questions

How does the brain make perceptual decisions?

How do we decide the direction of a moving object embedded in clutter?

How does the brain perform a direction discrimination task in a context-appropriate manner?

5

Motion Direction Discrimination Experiments

VALUABLE PARADIGM

Train monkeys to

discriminate the direction of a random dot motion stimulus

report the judgment via a choice saccade

Record behavior and area LIP neuronal responses

Shadlen & Newsome, 2001 Roitman & Shadlen, 2002

6

Random Dot Motion Stimulus

Interleaving of 3 uncorrelated random dot sequences

Coherence level: the fraction of dots moving non-randomly

60 Hz frame rate

Signal dots move from

frame n to frame n+3,

frame n+3 to frame n+6,

and so on

7

3.2% MORE AMBIGUITY

Two-Alternative Forced Choice Task

Right or Left?

8

51.2%LESS

AMBIGUITY

Two-Alternative Forced Choice Task

Right or Left?

9Two Experimental Contexts

REACTION TIME (RT) FIXED DURATION (FD)

Unlimited viewing duration before saccade in the

judged direction

Fixed viewing duration before saccade in the

judged direction

Shadlen and Newsome, 2001 Roitman and Shadlen, 2002

Roitman and Shadlen, 2002

10

Data from the Experiments

Accuracy of decisions in both FD and RT tasks as a function of

coherence

Reaction time of decisions in the RT task as a function of coherence

for correct and error trials

Area LIP neuronal responses during correct and error trials in both FD

and RT tasks for various coherences

Correlation between the temporal dynamics of LIP responses and

saccadic behavior (accuracy, reaction time of decisions)

Differences between sensory MT/MST and decision LIP responses

11

Existing Proposals / Models

‘BAYESIAN INFERENCE’ IN THE BRAIN

Beck et al., 2008; Gold & Shadlen, 2001, 2007; Jazayeri & Movshon, 2006; Ma et al., 2006; Pouget et al., 2003; Rao, 2004

NEURAL MODELS

Ditterich, 2006a, 2006b; Mazurek et al., 2003; Wang, 2002

Abstract; Non-neural; Propose explicit Bayesian decoders in brain areas

Do not clarify important computations that need to occur between the motion stimulus and saccadic response

Have a number of issues that need to addressed

Rev. Thomas Bayes

Treatise on Man(Rene Descartes)

12MOtion DEcision (MODE) Model

MOTION BCS: Grossberg et al., 2001 Berzhanskaya et al., 2007

Contextual gating of response

Choice of saccadic response

Winning direction chosen and fed back to MT

Pool signals over multiple orientations, opposite contrast- polarities, both eyes, multiple depths, and a larger spatial range

FT signals are strengthened, ambiguous signals weakened

Evidence accumulation amplifies feature tracking (FT)

signals

Local directional signals

Random dot motion input

Non-directional signals

13

Motion Processing from Retina to Area MST

Geometric aperture problem

BARBERPOLE ILLUSION

Feature tracking signals

Percept

Ambiguous signals

How do sparse feature tracking signals capture so many ambiguous signals to determine the global motion direction?

14

Local Directional Signals

Fried et al., 2002, 2005

Null direction inhibition model

Barlow & Levick, 1965

Grossberg et al., 2001

15

Short-Range Motion Signals

Local directional processes can be fooled by

low coherence

multiple dots

interleaving of uncorrelated dot sequences

16

Do Random Dot Motion Stimuli pose an Aperture Problem?

INFORMATIONAL APERTURE PROBLEM

17

MT-MST Circuit: Motion Capture

Inter-directional competition across space in area MST

Directionally-asymmetricfeedback inhibition from area MST to area MT across space

18MT and MST Responses during Stimulus Viewing

MODEL SIMULATIONS

MT MST

Britten et al., 1993

MTpref

null

19

Informational Aperture Problem

Directional short-range filters (V1)

51.2% coherence

Rightwardmotion

20

Informational Aperture Problem Resolution

Area MST

51.2% coherence

Rightwardmotion

Effectiveness of the motion capture process is limited by coherence level

and also viewing duration

21LIP Recurrent Competitive Field (RCF)

Grossberg, 1973+

Self-normalizes total activitylike computing real-time probabilities

Recurrent on-center off-surround shunting network

RCFs have also been used to model reach decisions in dorsal premotor cortex

Cisek, 2006

Noise-saturation problem

22Stochastic LIP RCF

23

RT Task SimulationsSample Correct Trials

RT Task

24LIP Responses during RT Task Correct Trials

SimulationsRoitman & Shadlen, 2002

More coherence in preferred direction causes:Faster cell activation

More coherence in opposite direction causes:Faster cell inhibition

Coherence stops playing a role in the final stages of LIP firing for preferred choices

25FD Task SimulationsSample Correct Trials

FD Task

The “gain of the LIP response is greater in the RT version of the task” when compared to the FD task

Roitman & Shadlen, 2002

26LIP Responses during FD Task Correct Trials

More coherence in preferred direction causes:Faster cell activationHigher maximal cell activation

More coherence in opposite direction causes:Faster cell inhibitionLower minimal cell activation

SimulationsRoitman & Shadlen, 2002

27Accuracy of Decisions

More coherence in the motion causes more accurate decisions

SimulationsMazurek et al., 2003

Roitman & Shadlen, 2002

RT task accuracy is slightly better than FD task accuracy at lower coherences (< 25%)

50 50

28

Effect of Viewing Duration on Accuracy in FD Task

Gold & Shadlen, 2003 Simulations

29LIP Responses in the RT Task during

Correct and Error Trials

Roitman & Shadlen, 2002 Simulations

LIP encodes the perceptual decision regardless of the direction and strength of the dots, unlike sensory MT/MST neurons

30

LIP Response Dynamics correlate with Reaction Time

Roitman & Shadlen, 2002 Simulations

6.4%

31Speed of Decisions (RT Task)

Correct (-) and Error (- -) Trials

More coherence in the motion causes faster reaction time

RTs on error trials are greater than those on correct trials

SimulationsRoitman & Shadlen, 2002

32

Slower Error Trial RTs?

At low coherences, the LIP cell dynamics are controlled more by cellular noise processes

As time passes, the likelihood of a wrong LIP cell being chosen increases

Slower RT indirectly explains slower rate of change in LIP responses on error trials

Brownian motion process

33Is Motion Direction Discrimination an example of

Bayesian Decision-Making?

“ logarithm of the likelihood ratio (logLR) provides a natural currency for trading off sensory information, prior probability and expected value to form a perceptual decision ”

Gold & Shadlen, 2001

S1: direction d

S2: opposite direction D

I: spatio-temporal input

logLR is proposed to be equivalent to opponent motion read-out

How does this explain decision-making properties in response to a variety of perceptual stimuli and task conditions?

)()/(

)()/(

22

11

SpSIp

SpSIpLR

)()/(ln)()/(lnln 2211 SpSIpSpSIpLR

34

Bayesian Inference is a Popular Hypothesis

This approach does provides an intuitive framework

Does it disclose brain mechanisms underlying perception and decision-making?

Probabilistic nature of decision-making in response to uncertainty

Neuronal variabilityBayesian inference in the brain?

Gold & Shadlen, 2001, 2007 Knill & Pouget, 2004Pouget et al., 2003; etc.

35

Brain without Bayes

“… We question the popular wisdom that the brain operates as an information-processing device that performs probabilistic inference …”

Shadlen et al., 2008

“… a categorical decision is readout by a Bayesian decoder ... Our work suggests that explicit representation of probability densities by neurons might not be necessary …” 

Furman & Wang, 2008

Grossberg & Pilly, 2008

“… This generality is part of its [Bayes’ rule] broad appeal, but is also its weakness in not proving enough constraints to discover models of any particular science …”

36

Wang, 2002; Wong & Wang, 2006; Mazurek et al., 2003; Ditterich, 2006a, 2006b

Comparison to other Neural Models

Our model goes beyond alternative models:

Uses the real-time perceptual stimuli used in the experiments

Does not make many of the specialized assumptions of previous models

Clarifies the different roles of sensory MT/MST and decision LIP cells

Simulates the effect of viewing duration on the psychometric function

Incorporates the difference in LIP responsiveness to the two task conditions

Considers the visual contribution to LIP response due to choice target

Simulates the entire time course of LIP responses during both tasks on both correct and error trials

Highlights the important role of BG in contextually gating the saccadic response

…

37MODE Model Predictions

Gradual resolution of the informational aperture problem in area MT Pack & Born, 2001

Explanation for the lack of coherence-independent initial transient pause in LIP activity in the FD task, unlike the RT task

Lower LIP activity, before motion onset, in multiple-choice tasks

Volitional top-down mechanism to make ‘forced choices’

Churchland et al., 2007

Stimulus manipulations such as:higher dot densitymore interleaved sequencesbriefer signal dots

should:decrease accuracy increase reaction times have influences on MT, MST, and LIP responses similar to those that

occur due to lowering motion coherence