Psychophysics of Human visual Motion Detection

2. Applications

George Mather

Questions for today

• Why do matrix displays appear tilted?

• Why does sport become more difficult at sunset?

• Why do people drive too fast in fog?

• Why do tennis players challenge line calls?

• When you look at the matrix displays used in railway stations, buses, trains, text appears tilted while it is moving.

• Rows are updated sequentially, but why the slight tilt?

Why do matrix displays appear tilted?

Motion and position: spatiotemporal interpolation

• Morgan (1980) presented two lines in apparent motion, in exact spatial alignment. One line was drawn slightly later than the other.

• The temporal offset was seen as an apparent spatial offset.

• Others reported similar effects (Burr, 1979; Fahle & Poggio, 1981).

• Explanation: The visual system interpolates between the discrete samples to construct a continuous trajectory for each line (not individual stations).

• The apparent spatial offset ( S) between the lines is the separation between their interpolated trajectories.

• How is interpolation achieved?

• A clue: Good spatiotemporal interpolation only occurs when the time interval between stations (ie. sampling interval) is very short (velocity is held constant).

• Left: Data from Morgan (1980) plotting the quality of interpolation as a function of motion discontinuity in stroboscopic displays (maximum score is 50).

• Right: Data from Fahle & Poggio (1981) plotting vernier threshold as a function of motion discontinuity. Red lines connect points of constant velocity but different ISI.

0

5

10

15

20

25

30

35

40

0 20 40 60 80 100 120 140

Mo

tio

n C

on

tin

uit

y In

dex

ISI (msec)

Effect of inter-stimulus interval1.0 deg.

20

0m

se

c.

• Qualitatively, the size of spatiotemporal receptive fields can explain the ISI effect in interpolation.

• What does the computational model of motion detection predict?

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 25 50 75 100 125 150 175

Mo

tio

n E

ner

gy

ISI (msec)

• The direction signal of the motion sensor declines as discontinuity increases.

• Beyond ISIs of ~75 msec there is no consistent motion response to support interpolation; explains the psychophysics.

Why is sport difficult near sunset?

Effects of mean luminance

• Psychophysical results indicate that motion sensors integrate over a longer temporal window at lower luminance.

• Two possible consequences:

• 1) A shift in apparent position, known as a spatial lag (notice the peaks in the functions).

• 2) Reduction in apparent velocity (Hammett et al., 2005).

Seeing movement in the dark

• Gegenfurtner et al. (1999) compared speed perception in cone-mediated vision against speed perception in rod-mediated vision.

• Rod-detected motion appeared to move at about 75% of the speed of a reference motion detected only using cones.

• What does this mean for sport?

Seeing movement in the dark

• Objects appear to lag, and to move more slowly in the dark, so timing when striking or catching balls will be incorrect.

• Maybe you do not play sport in the dark.

• What does it mean for driving?

Seeing movement in the dark

• Areas of the visual field outside the headlights may appear to be moving too slowly, so the driver may speed up.

• Notice that apparent speed is slower at lower contrast.

• What does this mean for driving?

Seeing movement in fog

• Snowden et al. (1998) placed participants in a driving simulator, and asked them to judge the simulated speed of the display in clear, misty and foggy conditions.

• Apparent speed is slower in fog.

Seeing movement in fog

• Then Snowden et al. (1998) asked participants to ‘drive’ at a set speed in clear, misty and foggy conditions.

• Participants drove at faster speed in fog.

• Accidents caused by excessive speed in fog may be due to misperception of speed at low contrast.

• Why do luminance and contrast affect perceived speed?

30 mph

50 mph

70 mph

Explaining speed effects

• One explanation is that low luminance and contrast alters the brain’s code for velocity in favour of lower velocities (the model by Hammett et al., 2005).

• Another explanation argues in favour of a ‘Bayesian’ model of perceived speed (Stocker and Simoncelli, 2006).

Bayesian inference in perception

• Bayesian inference is needed to resolve the ambiguities inherent in visual images.

• Incoming signals are combined with built-in assumptions.

• It also provides a way to deal with unreliable signals

Functional diagram forBayesian inference in sensory processing. A decision about a feature or object in the world is based on optimal combination of sensory information (likelihoods) with knowledge (priors), using Bayes’ rule.

Decision

SensoryProcessing

Posterior Probability

KnowledgeBayes’Rule

Stimulus

Likelihood

Prior

Bayesian models

Bayesian model of speed

• A prior of zero speed and a sensory estimate of speed (a) are combined to give a percept (b & c).

• At low contrast (d) the sensory estimate is unreliable so the estimate favours the slow prior (e & f).

• Hammett et al. (2007) argue against a Bayesian account because high speeds are over-estimated at low luminance.

Why do tennis players challenge line calls?

• In 1980 John McEnroe declared: “You cannot be serious!”

Modern technology

• Hawk-Eye

What is Hawk-Eye?

• The court is covered by multiple video cameras.• Image processing software identifies the centre of the ball in each camera image.• Information from different cameras is combined to compute the 3-D position of the

ball.• The process is repeated in each TV frame to recover the trajectory of the ball• The trajectory is used to compute the bounce contact area to within 30 mm.

Line call challenges

• Line calls are still made by officials.

• Players can challenge line calls.• When a call is challenged, the

umpire calls for a review of Hawk Eye data.

• Only two unsuccessful challenges are allowed by each player in each set.

• Challenges are recorded on a form.

Tennis psychophysics

• Psychophysics investigates the relation between perceptual experience and physical stimulation.

• Line call challenge records can be treated as psychophysical data.

• What is the relation between line calls, challenges, and actual ball position?

• Do challenges generally reflect gamesmanship, lapses in concentration, or genuine perceptual uncertainty?

• Can laboratory-based models account for the data?

• Paper by Mather (2008)…

The data set

• Fifteen ATP tournaments:– Bejing (2006), Cincinatti (2006), Indian Wells

(2007), Indianapolis (2006), Los Angeles (2006), Madrid (2006), Miami (2006), Miami (2007), Moscow (2006), New Haven (2006), Paris (2006), Rotterdam (2007), Shanghai (2006), Toronto (2006), Washington (2006)

• 246 professional players• 1473 challenges

General features of the data

• 94% of challenges occurred for ball bounces within 100mm of the edge of a court line (tennis balls are 67mm in diameter, court lines are 50mm wide).

• The remaining 6% of challenges were spread widely up to a distance of 449mm.

• So the overwhelming majority of challenges seem to involve bounces that are difficult to call.

Bounces within 100mm of a court line

• 1380 challenges.

• 51.3% were “out” calls, 48.7% were “in” calls.

• 60.7% of challenged calls were correct, reliably above chance.

• But 39.3% of the challenged calls were incorrect.

• How do challenges and errors relate to ball position?

Challenges and errors as a function of bounce position

• Challenges and errors peak sharply at the line, suggesting that performance is limited by uncertainty.

-0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

-100 -50 0 50 100

Challenges

Errors

Proportion ofChallengesor Errors

Ball Position (mm)

A simple model

• What happens in boxes 2 and 3?

• Output varies despite constant input, in terms of position.

1. Ball bouncesat position x

2. Player computesball-bounce position

relative to line

3. Line judge computesball-bounce position

relative to line, calls INor OUT

4. Signs ofthe positions

agree?

5. Playerchallenges

6. Player

correct?

Y

N

7. Challengesuccessful; line

judge error

8. Challengefails; player error

Y

N

9.

Noisy signals

• The visual system must assign a position to the ball bounce and to the line.

• A core assumption in sensory science is that signals are subject to internal ‘noise’; random fluctuations in output.

• In the model, we can assume that this random process perturbs position assignments.-1 0 1 2 3 4 5 6

Probability Density

Sensory Signal

Ball bounce position assignment

• The ball bounces 20mm outside the court.

• The solid line shows uncertainty in the assignment of the line’s position.

• The broken line shows uncertainty in the assignment of the bounce position.

• On some occasions (blue) the assignments lead to a correct decision.

• On other occasions (red) they lead to an incorrect decision.

-100 -50 0 50 100

Probability

Position (mm)

Line Ball

How well can the model predict the data?

• A Monte Carlo simulation of the model was implemented.

• Only two free parameters: the space constants of the player’s and the line judge’s uncertainty distributions.

• 25,000 trials at each of 20 ball positions, to compute challenge and error rates.

• Best-fitting values were selected by RMS error minimisation.

Predicted functions

• Data points are actual data (collapsed across ‘in’ and ‘out’ positions). • Curves are based on the best-fitting space constants. • Coefficients of determination are 0.963 and 0.938.

0 20 40 60 80 100

Errors

Ball Position (mm)

0

0.05

0.1

0.15

0.2

0 20 40 60 80 100

Challenges

Proportion ofChallengesor Errors

Ball Position (mm)

Model results

• Best-fitting space constants were 30mm for the player and 22mm for the line judge

• For best-fitting space constants, simulated challenges occurred in 20.8% of all trials for ball bounces up to 100mm from the lines.

• In 39.6% of simulated challenges the line judge’s call was incorrect (actual data: 39.3%).

• A simple psychophysical model can successfully predict performance in a complex and demanding real-world perceptual task.

• Model predicts that 8.2% of balls bouncing within 100mm of a court line will be called incorrectly by the line judge, due to perceptual limitations.

Apparent position of moving objects

• Laboratory research shows that motion affects apparent position.• Objects appear displaced in the direction of travel – flash-lag, motion-

induced-position-shift. Eg. Pavan & Mather (2008).• Is this effect present in the line call error data? • If so, balls should be called ‘out’ when they are in.• Plot % of ‘out’ calls as a function of ball position.

‘Out’ calls as a function of position

• The best-fitting curve shows a slight displacement in the predicted direction.• 50% point is at -14.4mm.• Very small effect - One-third of the diameter of a tennis ball.

• Challenge sheets record the line involved in each call.• So the analysis can be applied to sub-sets of the data:

– Base/service lines (628 challenges)– Side lines (567 challenges)– Centre lines (185 challenges)

• Do you expect differences?

Base/service

Side/Centre

Model Fit

• Best fits for centre and side lines are identical.• Base/service challenges and errors occur further from the line.• Why?

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0 20 40 60 80 100

Predicted Base/Service ErrorsBase/Service ErrorsPredicted Side ErrorsSide ErrorsPredicted Centre ErrorsCentre Errors

Errors

Ball Position (mm)

-0.05

0

0.05

0.1

0.15

0.2

0 20 40 60 80 100

Predicted Base/Service ChallengesBase/Service ChallengesPredicted Side ChallengesSide ChallengesPredicted Centre ChallengesCentre Challenges

Challenges

Ball Position (mm)

Base/service

Side/Centre

Model Fit Results

• Line judges perform better than players• Differences in 50% points are not statistically reliable. No convincing

evidence for position errors induced by motion.• Both judges and players find bounces near the base and service lines more

difficult to judge. May be due to:– Viewing angle– Greater uncertainty in bounce point along the trajectory of the ball.

Demonstration