02 Signal Detection Theory - Elder Laboratoryelderlab.yorku.ca/~elder/teaching/psych6256/lectures/02...

J. Elder PSYC 6256 Principles of Neural Coding

2. SIGNAL DETECTION THEORY

Probability & Bayesian Inference


2

Signal Detection Theory

  Provides a method for characterizing human performance in detecting, discriminating and estimating signals.

  For noisy signals, provides a method for identifying the optimal detector (the ideal observer) and for expressing human performance relative to this.

  Origins in radar detection theory   Developed through the 1950s and on by Peterson,

Birdsall, Fox, Tanner, Green & Swets



3

Example 1

  The observer sits in a dark room   On every trial, a dim light will be flashed with 50%

probability.   The observer indicates whether she believes the

light was flashed or not.   This is a yes-no detection task.



4

Noise

  In this example, the information useful for the task is the light energy of the stimulus.

  By the time the stimulus information is received by decision centres in the brain, it will be corrupted by many sources of noise:

  photon noise

  isomerization noise

  neural noise

  Many of these noise sources are Poisson in nature: the dispersion increases with the mean.



5

Equal-Variance Gaussian Case

  It is often possible to approximate this noise as Gaussian-distributed, with the same variance for both stimulus conditions.

  Then the noise is independent of the signal state.



6

Discriminability d’

p x |S = sH( ) = 12πσ

exp −x − µH( )2

2σ 2

⎛

⎝⎜⎜

⎞

⎠⎟⎟

p x |S = sL( ) = 12πσ

exp −x − µL( )2

2σ 2

⎛

⎝⎜⎜

⎞

⎠⎟⎟

µH − µL

σ

d ' = signal separation

signal dispersion=µH − µL

σ



7

Criterion Threshold   The internal response is often approximated as a continuous

variable, called the decision variable.

  But to yield an actual decision, this has to be converted to a binary variable (yes/no).

  A reasonable way to do this is to define a criterion threshold z:

x ≥ z → ' yes'x < z → 'no'

x

z

x



8

Effect of Shifting the Criterion



9

How did we calculate these numbers?

p x |S = sH( ) = 12πσ

exp −x − µH( )2

2σ 2

⎛

⎝⎜⎜

⎞

⎠⎟⎟

p x |S = sL( ) = 12πσ

exp −x − µL( )2

2σ 2

⎛

⎝⎜⎜

⎞

⎠⎟⎟

µH − µL

σ

d ' = zFA − zHIT



10

What is the right criterion to use?   Suppose the observer wants to maximize the expected number

of times they are right.

  Then the optimal decision rule is to always select the state s with higher probability for the observed internal response x:

  This is the maximum likelihood detector.   For the equal-variance case, this means that the criterion is the

average of the two signal levels:

p x | sH( )p x | sL( ) ≥1→ ' yes '

p x | sH( )p x | sL( ) <1→ 'no '

z =

12

µL + µH( ) z

The ‘likelihood ratio test’



11

Optimal Performance

  The performance of the maximum likelihood observer for this yes/no task is given by

p(correct) = p(HIT) = p(CORRECT REJECT) = erfc − ′d

2 2

⎛⎝⎜

⎞⎠⎟



12

Bias

  For this optimal decision rule, the different types of errors are balanced: p(FA) = p(MISS)

  For observers that use a different criterion, the different types of errors will be unbalanced.

  Such observers have lower p(correct) and are said to be biased.

z



13

ROC Curves

  Suppose the experiment is repeated many times under different instructions.

  The first time, the observer is instructed to be extremely stringent in their criterion, only reporting ‘yes’ when they are 100% sure the light was flashed.

  On subsequent repetitions, the observer is instructed to gradually relax their criterion.



14

ROC Curves   As the criterion threshold is swept from right to left, p(HIT)

increases, but p(FA) also increases.

  The resulting plot of p(HIT) vs p(FA) is called a receiver-operating characteristic (ROC).

′d = 0

Increasing ′d



15

ROC Curves

  Note that d’ remains fixed as the criterion is varied!   Thus d’ is criterion-invariant, and is thus a pure

reflection of the signal-to-noise ratio.



16

Example 2: Motion Direction Discrimination

  Random dot kinematogram   Signal dots are either all moving up or all moving down   Noise dots are moving in random directions

Britten et al (1992)



17

100% Coherence



18

30% Coherence



19

5% Coherence



20

0% Coherence



21

The Medial Temporal Area (V5)

www.thebrain.mcgill.ca



22

Experimental Details

  Signal direction always in preferred or anti-preferred direction for cell.

  What kind of task is this?   Note that now there is external noise as well as

internal noise.   To calculate neural discrimination performance,

assumed neuron paired with identical neuron, tuned to opposite direction of motion.

Behaviour Neuron Anti-Preferred Direction

Preferred Direction

False Alarm Rate

Hit

Rat

e



25

Priors

  Note that if the probabilities of the two signal states are not equal, the maximum likelihood observer will be suboptimal.

  In this case we must make use of the posterior ratio.

p sH | x( )p sL | x( ) ≥1→ ' yes '

p sH | x( )p sL | x( ) <1→ 'no '

Maximum a posteriori (MAP) rule



26

MAP Inference

  Using Bayes’ rule, we obtain:

  Thus we simply scale the likelihoods by the priors.

p sH | x( )p sL | x( ) =

p x | sH( )p sH( )p x | sL( )p sL( )



27

Loss and Risk

  Maximizing p(correct) is not always the best thing to do.

  How would you adjust your criterion if you were  A venture capitalist trying to detect the next Google?  A pilot looking for obstacles on a runway?



28

Loss Function   In general, different types of correct decision or action will

yield different payoffs, and different types of errors will yield different costs.

  These differences can be accounted for through a loss function:

Let a(x) represent the action of the observer, given internal response x.

Then L s,a(x)( ) represents the cost of taking action a, given world state s.



29

The Ideal Observer

  The Ideal Observer uses the decision rule that minimizes the Expected Loss, aka the Risk R(a|x):

R(a | x) = L s,a(x)( )p(s,x)

s∑ = L s,a(x)( )p(x | s)

s∑ p(s)



30

Example 3: Slant Estimation

02 Signal Detection Theory - Elder Laboratoryelderlab.yorku.ca/~elder/teaching/psych6256/lectures/02...

Documents

Transcript of 02 Signal Detection Theory - Elder Laboratoryelderlab.yorku.ca/~elder/teaching/psych6256/lectures/02...