Role of phase information and eye pursuit in the detection of moving objects in noise

D. L. Wilson and R. Manjeshwar Vol. 16, No. 3 /March 1999/J. Opt. Soc. Am. A 669

Role of phase information and eye pursuit in thedetection of moving objects in noise

David L. Wilson

Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio 44106 and Department ofRadiology, University Hospitals of Cleveland and Case Western Reserve University, 2074 Abington Road,

Cleveland, Ohio 44106

Ravindra Manjeshwar

Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio 44106

Received May 5, 1998; accepted October 1, 1998; revised manuscript received October 26, 1998

As part of an ongoing study that uses objective image quality measures to optimize medical imaging x-rayfluoroscopy, we investigated two basic features of the detection of moving cylinders that mimic arteries, cath-eters, and guide wires. First, we compared detection with and without a phase cue consisting of a nearbyalternating light and dark square. Depending on object size and velocity, phase cuing improved detectionfrom 1% to 15% and gave an average of 6%, an effect much smaller than the 38% predicted from a Monte Carlosimulation of the ideal observer. Evidently, humans were limited in their ability to incorporate knowledge ofthe phase cue. Second, we evaluated the effect of eye pursuit of a fixation point that moved with the target.In general, motion at the highest velocity degraded (74%) and enhanced (68%) detection of small and largeobjects, respectively. With eye pursuit, both effects were substantially reduced in a manner consistent with areduced retinal velocity. Our data compared favorably with a human observer model that included a spa-tiotemporal contrast sensitivity response and smooth-pursuit eye movements with a gain of 0.8. Thesemechanisms of perception are thought to be present in coronary artery x-ray fluoroscopy imaging, where phaseinformation is available from the moving heart and where motion markers are available from x-ray opaquemarkers incorporated in thin catheters and guide wires. © 1999 Optical Society of America[S0740-3232(99)00103-9]

OCIS codes: 110.4280, 110.3000, 330.4150, 330.2210, 330.4060, 330.1880, 330.1800, 330.5510.

1. INTRODUCTIONHigh-frame-rate, noisy x-ray fluoroscopy image sequencesare used to guide minimally invasive medical interven-tions. While the radiation dose to patients having themost intricate procedures is a health concern, reducingthe dose increases image noise, and the dose is sometimesincreased to permit better visualization of an interven-tion. Hence there are conflicting needs to reduce thedose while maintaining or improving image quality.1 Weuse human perception studies to understand and opti-mize x-ray fluoroscopy image quality. In particular, theability of humans to detect low-contrast objects providesan objective, quantitative measure of image quality. Us-ing such studies, we have investigated low-acquisition-rate pulsed fluoroscopy,1–4 temporal filtering,5 spatialfiltering,6 motion effects,7,8 and the effect of x-ray systemmotion blur.8

In the current report we investigate some basic issuesin the detection of moving objects in noisy image se-quences. In experiments, we give subjects progressivelymore assistance in this task. First, subjects fixate at apoint on the screen and attempt to detect the target,knowing only its velocity and size. Second, subjects fix-ate at a point on the screen, but we now give them thephase of the motion, using a nearby square that alter-nates between light and dark in synchrony with themovement of the target. Phase information greatly im-

0740-3232/99/030669-10$15.00 ©

proves detection by an idealized, computationalobserver,9 and we wish to determine whether humans canuse this additional information in their detection strat-egy. (Note that phase cuing is present in cardiac x-rayfluoroscopy imaging, where one can perceive the generalmotion of the heart.) Third, subjects fixate on a markerthat moves with the target. In this final case subjectsknow the size, velocity, and phase, and they can also pur-sue the target with their eyes to reduce the effective ve-locity of the target on the retina. Several authors reportthat eye pursuit improves acuity when one is viewing amoving object and increases the ability to detect motionblur in images.10–12 We extend these experiments tostudy the effect on object detection.

With regard to phase cuing, there are additional, rel-evant reports in the literature. Burgess andGhandeharian13 reported that knowledge of the phase ofa sinusoidal spatial target in a single image frame ofstatic white noise improves detection. As reviewed morefully in Subsection 5.A, Eckstein et al. extended this ex-periment to the temporal domain.14 They reported thattemporal phase cues significantly improve detection of astationary, temporally varying target in spatiotemporalwhite noise. There are other experiments that use noise-free backgrounds that demonstrate that humans use tem-poral cues to aid detection of temporally varyingstimuli.15 To our knowledge, our experiments are the

1999 Optical Society of America

670 J. Opt. Soc. Am. A/Vol. 16, No. 3 /March 1999 D. L. Wilson and R. Manjeshwar

first to investigate the effect of phase cuing on detection ofa compact moving target in a noisy image sequence.

In the two experiments presented here, we compare de-tection under paired conditions: detection with andwithout phase cuing, or detection with and without a mo-tion marker for eye pursuit. In both experiments inde-pendent variables include target velocity and size. Weuse sequences of images containing band-limited, spa-tiotemporal white noise and targets consisting of cylin-ders that mimic blood vessels, catheters, and guide wiresin x-ray fluoroscopy. We use the adaptive reference/testforced-choice paradigm developed in our laboratory.2,3

Before proceeding with a description of the experiments,we develop a human observer model that includessmooth-pursuit eye movements.

2. THEORYA. Eye Pursuit MechanismsThe small foveal region of the retina is responsible forhigh acuity vision. When an object of interest moves, theeyes pursue the object in an attempt to view it with thefovea. Although humans display approximately five ma-jor types of eye movements, saccades and smooth pursuitare most important for the pursuit of moving objects.16

Smooth-pursuit eye movements attempt to keep the im-age of a moving object on the fovea by matching eye ve-locity to target velocity. The ratio of eye velocity tostimulus velocity is the gain, gsp , of the ocular-motor sys-tem. Gains typically range from 0.4 to 0.95, and smooth-pursuit eye movements are most effective at velocities,80 deg/s.17–19 Because the gain is less than unity,there is necessarily an accumulated error in smooth pur-suit over time. When the spatial error exceeds a thresh-old, smooth pursuit is interrupted, and a saccade movesthe fovea to the target. Saccades are fast, conjugatemovements of both eyes that quickly change the fixationpoint to reduce the average position error. As the veloc-ity of the target increases, the frequency of saccades in-creases and stabilizes at approximately four saccades persecond. A saccadic movement results in a refractory pe-riod of '50–100 ms.

In our experiments we use relatively low velocities(<6.04 deg/s), where it is easy to perform smooth-pursuiteye movements. Smooth-pursuit eye movements effec-tively reduce the velocity of a moving object on the retina,and we include this in the model described next.

B. Human Observer ModelThe basic model for detecting moving objects in noisy im-age sequences has been described in detail previously.7,8

It is a nonprewhitening matched filter modified to includea visual system response. It is similar in concept to somemodels for target detection in single images20,21 and is anextension of our previous model for detection of stationaryobjects in white and temporally correlated noisy imagesequences.4,5

The observer model is developed in the Fourier domain(Fig. 1). The moving cylinder has a spectrumS( fx , fy , ft), where fx and fy are spatial-frequency com-ponents and ft is the temporal frequency. Assuming aninfinitely long cylinder along the y axis allows a two-

dimensional simplification in x and t.7 We use a two-dimensional (2D), visual-system spatiotemporal contrastsensitivity function, Vst( fx , ft). The model signal-to-noise ratio, or SNR, is

SNR

5

AyEEuS~ fx , ft!Vst~ fx , ft!u2d fxd ft

F E EuS~ fx , ft!u2uVst~ fx , ft!u4Pn~ fx , ft!d fxd ftG1/2 ,

(1)where Ay is a constant resulting from integration along fyand Pn( fx , ft) is the power spectrum of the image noise.

Assuming that the cylinder moves continuously acrossthe retina at a constant velocity, vr , the signal is s(x, t)5 s(x 2 vr t), and the power spectral density is given be-low, where d ( ) is the impulse function7:

uS~ fx , ft!u2 5 uS~ fx!u2d ~vrfx 1 ft!. (2)

For white image noise, the noise power spectral density isa constant, Pn( fx , ft) 5 se

2. After integration along ft ,the discrete implementation equation is as given below,where integrals are replaced by summations and whereV( fx , vr) [ Vst( fx , 2vrfx) 5 Vst( fx , ft):

SNR 5

Ay( uS~ fx!V~ fx , vr!u2Dfx

F( uS~ fx!V~ fx , vr!u2uV~ fx , vr!u2se2DfxG1/2 .

(3)

In the above expression the filter template includes thenoise-free target signal, S( fx), which is computed nu-merically from the spatial target.

Note that the velocity driving the model is the retinalvelocity, vr , and not the physical velocity of the movingcylinder. Only under the condition of idealized gaze fixa-tion is the retinal velocity equal to the physical velocity.With eye pursuit, the velocity of the stimulus is modifiedby the smooth-pursuit system gain, gsp , to give the reti-nal velocity, vr , of the moving target:

vr 5 H vtarget gaze fixation

~1 2 gsp! 3 vtarget eye pursuit. (4)

Smooth-pursuit gain depends on factors like the velocityof the target and the predictability of motion.22 For uni-

Fig. 1. Human observer model that describes target detection innoisy image sequences. The input to the model is a sequence ofnoisy images. The model consists of a human spatiotemporal vi-sual response, a spatiotemporal template filter, an optional inter-nal noise source, and a threshold detector. In the case of eyepursuit, the retinal velocity is reduced by the smooth-pursuitgain.


form ramp motion with velocities less than 50 deg/s, gainranges from 0.75 to 0.95.19 We use a gain of 0.8 in ourmodel.

Human spatiotemporal contrast sensitivity is mea-sured from flicker experiments by means of eitherstationary23,24 or traveling25 sine-wave stimuli. Kelly re-ported a contrast sensitivity function from traveling sine-wave functions with a stabilized visual field over 7.5 degof visual angle.25 This contrast sensitivity function char-acterizes both the foveal and the peripheral responses ofthe human visual system. We modify it by inserting theretinal velocity, vr :

V~ fx , vr! 5 @6.1 1 7.3ulog~vr /3!u3#

3 4p2vrfx2 exp@24p fx~vr 1 2 !/45.9#. (5)

The dimension of V is a unitless contrast, and fx is givenin terms of the visual angle, giving a unit of cycle per de-gree. For convenience, vr is given in terms of the visualangle in degrees per second. Kelly determined that theunstabilized response for stationary sine waves was closeto the stabilized response at vr 5 0.15 deg/s.25 In calcula-tions, for vr . 0.15 deg/s, we use Eq. (5), and for 0 < vr< 0.15 deg/s we use Eq. (5) with vr 5 0.15 deg/s.

3. METHODSA. Reference/Test Adaptive Forced ChoiceThe basic experiment consisted of comparing target detec-tion under test and reference conditions, i.e., with andwithout phase cuing, or with and without a moving fixa-tion marker for eye pursuit. We used a reference/test,adaptive forced-choice technique specially designed forsuch comparisons.2,3,26 Briefly, we used a four-alternative forced-choice (4AFC) paradigm whereby amoving cylinder was placed in one of four panels and thesubject correctly or incorrectly chose the position that she,or he, thought the object resided. Viewing time was un-limited, and the display was terminated by the subject.Subjects were urged to give accurate responses and wererewarded with a computer beep when they successfullyselected the target location. We alternated the referenceand test presentations to reduce the effect of undesirablevariables such as subject attention level on thecomparison.3 To perform adaptation, all previous re-sponses in the session were used to estimate a signal con-trast for the next trial that would achieve a target prob-ability correct of 80%. Signal contrasts wereindependently adapted for the interleaved reference andtest presentations. All the image data were computergenerated between presentations. An experiment con-sisted of 100 pairs of reference and test presentations.As we describe next, reported values were signal contrastsensitivities (1/C) and response times.

For analysis of detection data, we used a standard sig-nal detection model that assumed that human observersuse a continuous decision variable with equal Gaussianprobability density functions for the cases of ‘‘objectpresent’’ and ‘‘no object present.’’ 2,3,27,28 The detectabil-ity index, d8, is the distance between the means of thetwo overlapping distributions normalized by the standarddeviation.27 In the adaptive technique we used a model

for d8 that needed to be applicable only over a small op-erating range. We assumed that d8 5 uC, where C wasthe contrast and u was a constant of proportionality. Weused a contrast definition without normalization:

C 5 gb 2 gt , (6)

where gb was the mean background gray level and gt wasthe minimum target gray level. In the adaptive processwe used a maximum-likelihood technique to estimate thevalue of u from all the measured responses. We then cal-culated the next contrast from C 5 d80%8 /u, where d80%85 1.893, the value for 80% probability correct in a 4AFCexperiment. We report results in terms of 1/C, the signalcontrast sensitivity. The details of adaptation, param-eter estimation, and calculation of standard errors are de-scribed elsewhere.3 Since contrast sensitivities were ob-tained in a paired fashion with the reference/testparadigm, we rigorously applied statistics such as thepaired t-test.

B. Phantoms and DisplayThe 4AFC display is shown in Fig. 2. The noise-freepanel at the top contained a reference copy of the cylinderand the markers. The cylindrical target was placed anequal number of times, in random order, in each of thefour noisy panels. Sixty-four image frames mimickingx-ray fluoroscopy noise were displayed in a repetitiveloop. The cylinder moved left and right periodically atconstant speeds that mimicked the average, but not themaximum, speed observed in coronary angiography. Asa function of time, the displacement of the target followeda triangular waveform. The total excursion was fixed,and the period was adjusted as a function of target veloc-ity. Markers were located reasonably far from the sta-tionary target so as not to provide a strong location cue(see Subsection 5.A). All four panels contained high-contrast markers, but only one contained the target cyl-inder.

In the first experiment detection was measured withand without phase cuing. The phase cue consisted ofchanging the gray scale of all eight markers in the displayfrom light to dark in synchrony with the target motion.For the condition without phase cuing, the markers re-mained dark. In both conditions subjects were asked tofixate midway between the two markers. We did notrecord eye movements to confirm fixation, but subjectswere instructed and trained to fixate between the mark-ers.

In the second experiment we compared detection withand without a moving fixation marker for eye pursuit.For the eye pursuit condition, the high-contrast markersmoved with the cylinder at a constant distance from thecylinder, and subjects were asked to fixate between thehigh-contrast markers. At the velocities used, subjectsshould easily perform smooth-pursuit eye movements.For the condition without eye pursuit, a stationary phasecue was given, and subjects were asked to fixate midwaybetween the markers.

The targets for detection were projected cylinders thatsimulated arteries, catheters, and guide wires in x-rayimaging. The details of construction of the phantom


were described previously.7,8 Blur due to the x-ray sys-tem line-spread function was introduced by convolution ofthe image of projected cylinders with a 1D kernel.7,29

Blurred cylinders were used both in experiments and inmodel calculations. In figures, the contrast values arethe noise-free peak differences between the projected cyl-inder and background prior to blurring.

Noise in the computer-generated images simulatedx-ray fluoroscopy Poisson noise corresponding to an imageintensifier exposure of 20 mR/s. The noise had a gray-scale standard deviation of 29.2 on a mean background of128.

We used display software running on a PowerMac 9500200-MHz personal computer (Apple Computer Inc., Cu-pertino, Calif.). The display had a carefully linearizedluminance response. The mean and maximum lumi-nance values were 37.3 and 75.7 cd/m2 at gray levels of128 and 255, respectively. The display resolution was640 3 480 pixels at 66 frames/s. Each image frame wasrepeated once to mimic 33 x-ray image acquisitions/s, avalue close to the rate of 30 acquisitions/s, which is avail-able on many commercial x-ray systems.

C. Experimental ParametersWe now summarize some important experimental param-eters. The display pixel size on the screen was 0.4 mm.Subjects sat 1.0 m from the screen, giving a visual angleof 0.023 deg/pixel. Before simulated x-ray system blur-ring, cylinder diameters were 1 and 21 pixels. We usedthree velocities of 0, 132, and 264 pixels/s, giving 0, 3.02,and 6.04 deg/s, respectively, and frame-to-frame displace-ments of 0, 4 and 8 pixels, respectively. The total excur-sion was 128 pixels. The period of the target motion was

Fig. 2. 4AFC display consisting of 440 3 480 pixels divided intofive panels of 88 3 480 pixels. The target cylinder is placed ran-domly in one of the four noisy panels. The top panel is a noise-free stationary reference. The high-contrast, black markers are3 3 3 pixel squares. The two markers in each panel are 75 pix-els apart. In an experiment the black markers are easily seen,whereas the cylinder contrast is adapted near the threshold fordetection. In the case of motion the target cylinder moves be-tween the arrows.

1.94 and 0.97 s at 132 and 264 pixels/s, respectively.Each presentation consisted of a repeating cine loop of 64different frames. Experiments were conducted in a dark-ened room. Three graduate students participated as ob-servers: one author (RM), and two others naı̈ve to thehypotheses. All observers had normal or corrected-to-normal vision.

An experimental session lasted approximately 1 h andconsisted of 200 detection trials and more than 50,000unique noisy image frames. For each experiment therewere two training sessions and 400 trials prior to acqui-sition of data for analysis. All the subjects had signifi-cant previous experience in similar experiments.

D. Monte Carlo SimulationsWe used Monte Carlo simulations similar to ideal ob-server calculations reported by Burgess30 to investigatethe effect of phase known versus phase unknown. We as-sumed a 1-pixel-diameter cylinder aligned along the yaxis and blurred by the line-spread function. To simplifythe representation of the moving cylinder to two dimen-sions we considered a 2D triangular signal in x and t.

This simplified representation greatly reduced thenumber of calculations and memory requirements, and itcan be corrected to give the true 3D case. For the 3Dproblem the matched filter output SNR2 is proportional toNf NlE/s2, where E 5 *S2(x)dx is the signal energyalong x, Nf is the number of frames, Nl is the length of thecylinder in pixels, and s is the standard deviation of thenoise. This expression shows that the theoretical SNR2

for the 3D case is simply obtained by multiplication of theSNR2 of the 2D case by Nl .

For each Monte Carlo trial the computational observerwas presented with a set of four alternatives consisting ofone 2D array containing signal plus noise and three 2Darrays containing noise only. The observer computed adecision metric, using one of the strategies below, and cor-rectly or incorrectly chose the 2D array having the maxi-mum chance of containing the signal. At each signal en-ergy level the program ran for 5000 trials, computed thepercent correct, and calculated a value of d8. Ten signalintensities ranging from 1 to 30 gray levels were used.The standard deviation of the noise was 29.2 gray levels.

When the phase was known the computational ob-server performed a signal-known-exactly, matched filteroperation, and the scalar output was the decision vari-able. In the case of unknown phase the observer per-formed matched filter operations, using 64 possiblephases of the signal as templates.9,30 For each of the fourpossible panels a likelihood, Li , was calculated with

Li 5 H(j51

K

expF 1

s 2 (x

(t

Sj~x, t !Di~x, t !G , (7)

where Sj(x, t) was the signal with the jth phase out ofK 5 64 possible phases, Di(x, t) was a 2D array contain-ing a Monte Carlo realization of either signal plus noise ornoise only, s was the standard deviation of the noise, andH was a constant that included the signal energy. In thecase of unknown phase, the value of Li was the decisionmetric.


4. RESULTSFrom a typical experimental session, we obtain plots ofcontrast sensitivity and response times as a function oftrial number. The contrast sensitivity curves are ini-tially noisy but stabilize after approximately 30 trials inthe adaptive experiments. We obtain final estimates andstandard errors after 100 trials. Plots of typical experi-mental measurements are shown elsewhere.2,3

A. Experiment 1: Phase CuingAs a function of velocity, contrast sensitivity of a 1-pixel-diameter cylinder is shown in Fig. 3(a) with and withoutphase cuing. Results are very similar across subjects,and there is a consistent degradation of detection with in-

Fig. 3. Effect of phase cuing on the detection of a 1-pixel-diameter cylinder. Contrast sensitivity (1/C) values are plottedin (a) with and without phase cuing. At a velocity of 0 pixels/s,phase cuing is irrelevant, and only one data point is shown.With phase cuing there is a significant improvement in detection( p , 0.05). To reduce intersubject variability, we normalizedata from each subject by her, or his, 1/C value at zero velocity.After normalization, data points overlap and are not visible on aplot. Hence we plot responses averaged across subjects in (b).The lines are not theoretical and simply connect the averages.

creasing velocity. Every phase-cued measurement ex-ceeds the paired measurement without phase cuing.With phase cuing we obtain average enhancements of15% and 9% at 132 and 264 pixels/s, respectively. Whenwe pool 1/C data across observers, a standard two-factorial analysis of variance reveals a significant effect ofphase cuing and velocity on detection ( p , 0.05). Thisanalysis does not include the standard error estimates foreach measurement, and data at zero velocity are ex-cluded. Since data are obtained in a paired fashion withthe reference/test paradigm, we use a paired t-test andagain find a significant effect of phase cuing ( p , 0.05) ateach velocity. To reduce intersubject variability we nor-malize the data from each subject by her, or his, measure-ment at zero velocity. Normalized data are averagedacross subjects in Fig. 3(b). Again, there is a small butconsistent effect of phase cuing.

Detection improves with motion for a large cylinder of21-pixel diameter. Individual and average normalizedcontrast sensitivities are plotted as a function of velocityin Figs. 4(a) and 4(b), respectively. In this case there was

Fig. 4. Effect of phase cuing on the detection of a 21-pixel-diameter cylinder. The effect is not significant ( p . 0.05). (a)Individual observers and (b) averages for three observers areplotted.


no significant effect of phase cuing ( p . 0.05). Averageenhancements with phase cuing were only 2% and 1% at132 and 264 pixels/s, respectively.

Monte Carlo simulations with and without knowledgeof phase are shown in Fig. 5. With phase known, d8equals the matched filter output SNR. With no knowl-edge of phase, the value of d8 is reduced, and the plot isnonlinear near the origin. In the linear range, near ouroperating point of d80%8 5 1.893, the percent differencewith and without knowledge of phase is '38%, a valuelarger than that obtained experimentally (1–15%).

B. Experiment 2: Eye PursuitIn the second experiment, high-contrast markers movewith the cylinder to provide a moving fixation point foreye pursuit. Detection for this condition was comparedwith fixed markers with phase cuing. For the 1-pixel-diameter cylinder, with a stationary phase cue, detectiondegrades with increasing velocity [Fig. 6(a)]. When dataare pooled across observers, a two-factorial analysis ofvariance and a paired t-test show a significant effect ofeye pursuit on detection ( p , 0.05). Contrast sensitiv-ity data are normalized and compared with theory in Fig.6(b). When phase cuing is used, motion of the cylindercauses a significant drop in contrast sensitivity as com-pared with the stationary cylinder. The average de-crease in 1/C values are 64% and 73% at 132 and 264pixels/s, respectively. (These data compare closely withresults obtained in experiment 1.) The effect of velocity isgreatly reduced, and average sensitivity decreases by only10% and 18%, respectively, when eye pursuit is allowed.Using a gain of 0.8 for smooth pursuit, the normalizedSNR from the human observer model is plotted for eyepursuit (solid curve) and phase cuing (dashed curve).

In Fig. 6(b) the model overestimates the data withouteye pursuit. One reason is that the presence of a nearbymarker enhances detection of a thin stationary

Fig. 5. Monte Carlo simulation results for the ideal observermodel with and without phase cuing. The SNR on the x axis isthe prediction from a signal and phase known exactly (ideal ob-server calculation). The theoretical d8 values on the y axis com-pare phase known (solid curve) with phase unknown (dashedcurve). See Subsection 3.D for details of the simulation.

cylinder.31,32 This effect is measured by comparison ofthe contrast sensitivities of a thin stationary cylinderwith and without the marker present [Fig. 7(a)]. For allthree subjects sensitivity increases when the marker ispresent, and the average increase is 35%. When themeasurement without the marker is used to normalizedata, the agreement between data and theory is improved[Fig. 7(b)].

Motion improves detection of the 21-pixel-diameter cyl-inder (Fig. 8). Also, unlike the 1-pixel-cylinder case, eyepursuit significantly degrades detection. This result isexpected, since eye pursuit reduces the effective retinalvelocity and hence the enhancement of detection whenmotion is present. The two-factorial analysis of varianceand the paired t-test show that eye pursuit significantlyimproves detection at the two nonzero velocities ( p, 0.05). The average increase in 1/C with eye pursuit isonly 17% and 38% at the two velocities as compared with

Fig. 6. Effect of eye pursuit on the detection of a 1-pixel-diameter cylinder. Individual and averaged normalized con-trast sensitivities (1/C) are plotted in (a) and (b), respectively.Eye pursuit significantly improves detection ( p , 0.05). In (b),the solid curve is the normalized SNR for the human observermodel with eye pursuit, and the dashed curve is the human ob-server model prediction without eye pursuit. Standard errorsare plotted. Data are from three subjects (h, RM; n, AC; s,CR).


67% and 69%, respectively, without eye pursuit. Theo-retical curves compare favorably with normalized data[Fig. 8(b)]. In this case model predictions fit the datavery well because detection of the large stationary cylin-der is not affected by the nearby marker.31

C. Response TimesAverage response times across all the experiments were'9.8 s. Across sessions and subjects, average responsetimes vary considerably. Also, within an experimentalsession, there is a large standard deviation, sometimes ofthe order of 50% of the mean. To allow comparison, weuse the reference/test paradigm in which different typesof presentations are interspersed. This allows a rigorousapplication of statistical analyses like the paired t-test.

There was no effect of phase cuing (experiment 1) oreye pursuit (experiment 2) on response times ( p

Fig. 7. Effect of fixation markers on detection. In (a), contrastsensitivities are plotted for a 1-pixel-diameter stationary cylin-der with and without the fixation marker. All three subjectsconsistently show improved detection in the presence of themarker, and a paired t-test shows a significant effect ( p, 0.05). In (b), to reduce the effect of localization, motion datafrom Fig. 6 are normalized by the 1/C value at zero velocity with-out a fixation marker (crosshatched circles). The lower, un-shaded circles contain data points from Fig. 6. Data are fromthree subjects (h, RM; n, AC; s, CR).

. 0.10). There was an effect of motion. For the phase-cued/phase-not-cued experiment, ratios of response timesfor moving targets over stationary targets were averagedacross subjects. The average ratios (and 61 standard er-rors) were 1.17 6 0.03 and 1.14 6 0.03 at 132 and 264pixels/s, respectively. For the phase-cued/eye pursuit ex-periment, these ratios were 1.12 6 0.02 and 1.216 0.04, respectively. All the ratios were significantlygreater than unity, indicating that subject response timeswere increased for moving cylinders.

5. DISCUSSIONOur experiments investigate the ability of humans to useprior information in the detection of moving objects.Starting with known velocity and size, we progressivelyadd phase cuing and eye pursuit. Although knowledge ofphase can effectively be used by a computational ob-server, there is relatively little improvement in humandetection with our phase cue. However, the effect of eyepursuit is significant and consistent with a retinal veloc-ity reduced by smooth-pursuit eye motion. We now dis-

Fig. 8. Effect of eye pursuit on the detection of a 21-pixel-diameter cylinder. Eye pursuit significantly decreases the en-hancement in detection with motion ( p , 0.05). As in Fig. 6,(a) individual and (b) normalized averages are plotted.


cuss our results in more detail and compare them withprevious measurements and models.

A. Phase CuingWith phase cuing we obtain 1–15% enhancements in con-trast sensitivities, with an average increase of 6%. Eck-stein et al. reported larger effects of phase cuing, withenhancements ranging from 80% to 17% at low(d8 ' 0.47) and high (d8 ' 3.4) detectabilities,respectively.14 When we estimate a response at our per-formance level (d80%8 5 1.893) from Fig. 3 of the paper byEckstein et al.,14 the enhancement is '35% with phasecuing, a value 2.3 times our maximum result and 6 timesour average result.

This discrepancy is explained by differences in the ex-periments. Eckstein et al. used a small stationarysquare that alternated between dark and light as com-pared with the surround.14 The contrast of their signalvaried as a square wave with a period of 1 s, and noisyimages were displayed at 10 frames/s. By contrast, weused a cylinder moving at a constant velocity. The na-ture of the phase cues was also very different. In thestudy by Eckstein et al. the phase cue was a nearby copyof the stationary target. In our case the subjects’ taskwas more complex: They had to use information from astationary phase cue to aid detection of a moving target.Clearly, the spatiotemporal characteristics of the noiseand target are very different and explain the discrepancybetween the two experiments.

One might argue that our phase cue was inadequateand that with an improved phase cue there would be moreenhancement. We tested several alternatives. For ex-ample, we used a marker that moved with the target orwhich moved with the same phase and a smaller excur-sion. In this case it was difficult to rule out inadvertenteye motion, an undesirable effect when one is trying to se-lectively study phase cuing. We also created alternatinglight and dark phase cues placed directly above and belowthe stationary target. Such markers introduced an addi-tional confound because they greatly reduced location un-certainty in cylinder detection.31,32 After trying theseand other alternatives, we chose the method reportedhere.

With regard to phase cuing, it is interesting to comparehuman and computational observer detection. FromMonte Carlo simulations, the percent difference in SNRwith and without phase cuing is '38% (Fig. 5) near ouroperating point of d80%8 5 1.893. Eckstein et al. reportedsimilar differences for both an ideal observer calculationand a suboptimal, Max–Min model.14

We measured a smaller difference, and there are sev-eral potential explanations. It is probable that humanscannot simply incorporate the additional knowledge ofphase as effectively as a computational observer. Thismight be particularly true in our experiment, where thephase cue was stationary and the target was moving. Itis noteworthy that computational observers such as thosediscussed above, without a spatiotemporal contrast sensi-tivity response, do not explain many other experimentalobservations on image sequences.1,7,8,33 In principle, onecould calculate the effect of phase uncertainty by usingour human observer model, but this would require exten-

sive code modification and very computationally demand-ing Monte Carlo calculations. We anticipate that thespatiotemporal contrast sensitivity response in the hu-man observer model would blur the response and reducethe effect of phase known versus phase unknown for amoving object. Similarly, humans are limited in theirability to use all the temporal information in image se-quences. As compared with the ideal observer, efficien-cies over 64 frames are typically 3% for this target, avalue much smaller than the 30–50% usually obtainedwith a single image frame. This suggests that humansapproach detection of moving targets in a much moreadaptive fashion, using small windows of temporalinformation.7 With modifications such as those sug-gested here, the difference between theory and modelmight be significantly reduced.

B. Moving Fixation Marker for Eye PursuitThe presence of a moving fixation marker and eye pursuitgives consistent, significant effects. Without the movingmarker, motion degrades detection of small cylinders andenhances detection of large cylinders. Notably, a motionmarker reduces both the degradation and the enhance-ment in a manner consistent with a reduced retinal veloc-ity.

With a smooth-pursuit gain of 0.8 the agreement be-tween model and theory is good [Figs. 6(b) and 8(b)]. Again of 0.8 is less than the gains of 0.9–0.95 reported inthe literature for velocities less than 10 deg/s.18,19 How-ever, these higher gains were measured in noise-freebackgrounds. A noisy background causes an optokineticdrag that lowers the gain.34

The difference between eye pursuit and stationary fixa-tion can also be interpreted by use of the physiologicallydistinct magnocellular and parvocellular visual systemsas described by Livingstone and Hubel.35 The magnocel-lular system has receptors in the retinal periphery and isresponsible for the detection of moving objects with lowspatial frequencies (the thick cylinders in our experi-ments). The parvocellular system, with its receptors pri-marily in the fovea, is responsible for the detection of sta-tionary targets of high-spatial-frequency content (the thincylinders in our experiments). In our experiments withstationary fixation markers, the contrast sensitivity for athin cylinder moving across the fovea degrades becausethe parvocellular system has a sluggish temporal re-sponse. Eye pursuit of a thin cylinder stabilizes the cyl-inder on the fovea and enhances detection. By contrast,the detection of a thick moving cylinder is enhanced withfixation because receptors in the peripheral, magnocellu-lar system are stimulated.

As far as we know, there are no experiments in the lit-erature with which we can directly compare results witheye pursuit. However, the group at Cedars-Sinai Medi-cal Center, Los Angeles, California sometimes uses amoving model artery segment and a small target-fillingdefect.36–38 Since the artery segment is easily seen, itprovides a motion cue that can aid detection. Since theirphantom and experimental design are different fromours, no quantitative comparison can be made.

Our detection results with a compact cylindrical targetin noise are qualitatively similar to those of Flipse et al.,


who used sine waves.39 They examined noise-free, sine-wave detection under three conditions: The sine wavewas stationary (experiment A), the sine wave was movingand subjects fixated on a stationary point (experiment B),and the sine wave and the fixation point moved together(experiment C). For small-period sine waves, they foundthat contrast sensitivity ordered as follows: experimentA . experiment C . experiment B. Presumably, ex-periment B gave the worst sensitivity because the sinewave was motion blurred with fixation. Motion blur wasnot present when the eyes pursued the object (experimentC) or when the sine wave was stationary (experiment A).When experiments were corrected for the effective retinalvelocity, results were quantitatively consistent.

The above-mentioned ordering reported by Flipse et al.matches our measurements with the small cylinder, i.e.,stationary cylinder . moving cylinder with eye pursuit. moving cylinder with fixation at a point. Moreover, asin the study by Flipse et al., we modeled results with aneffective retinal velocity.

C. Other Modeling IssuesThe fit of the human observer model to data from a smallcylinder without a motion marker [Fig. 6(b)] is not as goodas other fits reported here and previously.7,8 For a1-pixel-diameter cylinder, Xue and Wilson7 reported sen-sitivities very close to the dashed curve in Fig. 6, whereasthe present data fall below the curve. A cause for confu-sion in Fig. 6 is that the presence of a fixation marker en-hances detection of the stationary cylinder and exagger-ates the percent decrease when the cylinder moves.Hence, in Fig. 7(b), data are normalized to measurementsobtained without the fixation markers, and measure-ments better match the model and previous experiments.A second explanation is that the experiments of Xue andWilson were performed under conditions of free responseto simulate clinical x-ray fluoroscopy viewing. In thepresent experiments subjects fixated their eyes on one lo-cation, and this degraded the ability to detect the smallmoving cylinder. The human observer model is a signal-known-exactly, phase-known-exactly model, and its con-struction most closely matches the experiment with phasecuing. Hence the remaining discrepancy between themodel and the data with phase cuing is disappointing[Fig. 7(b)]. Surprisingly, the agreement between this cal-culation and previous free-response measurements isbetter.7,8

Despite the difference with the small cylinder, detec-tion of the large cylinder is fairly similar between previ-ous and present experiments. One reason is that detec-tion of large objects is little affected by locationuncertainty.31,32

D. Other Dynamic Perception ExperimentsThere are other types of dynamic perception experimentsof interest. Stelmach and Hearty measured vernier acu-ity as a function of velocity.10 With the eyes fixated on apoint and no eye tracking of the object, vernier acuity be-gins to degrade at '4–8 deg/s. By contrast, with eyetracking there is no degradation at 16 deg/s. Both ourdetection experiments and the vernier acuity experimentsare explained by a reduction in effective retinal velocity.

Other experiments measure the ability to detect blur-ring of high-contrast, moving targets. Much of this re-search is inspired by compression of image sequences. Acurrently disputed idea is that moving objects can be spa-tially degraded because such degradation cannot be de-tected by humans. Hearty11 argued that this is not thecase with high-contrast objects that can be tracked by theeyes because one can measure very good dynamic visualacuity. Westerink and Teunissen12 concurred; theyfound that the perceived sharpness of moving unblurredimages was always greater than moving blurred images.As in our detection experiments, the interpretation is thateye tracking reduces the effective retinal velocity. This,in turn, permits good dynamic visual acuity.

E. Relationship to X-Ray FluoroscopyOur goal is to understand and optimize image quality inmedical imaging x-ray fluoroscopy. The experiments re-ported here are easily related to this goal. In the case ofcoronary artery angiography, targets are arterial lesions,arterial filling defects, and interventional devices such asstents and guide wires. The heart beats at a regular in-terval, and in an image sequence there are many phasecues that slightly improve detection. X-ray opaquemarkers are sometimes added to small guide wires.These high-contrast markers act very much like the mo-tion markers used in our experiments and greatly aid de-tection of thin guide wires.

ACKNOWLEDGMENTSThis work was supported by the Whitaker Foundationand by National Institutes of Health grant RO1-HL48918. We thank Andres C. Carrillo and CharulathaR. Ramanathan for being conscientious subjects.

D. L. Wilson, the corresponding author, can be reachedat the first address given on the title page, by telephone at216-368-4099, or by e-mail at [email protected].

REFERENCES1. R. Aufrichtig, P. Xue, C. W. Thomas, G. C. Gilmore, and D.

L. Wilson, ‘‘Perceptual comparison of pulsed and continu-ous fluoroscopy,’’ Med. Phys. 21, 245–256 (1994).

2. P. Xue and D. L. Wilson, ‘‘Pulsed fluoroscopy detectabilityfrom interspersed adaptive forced choice measurements,’’Med. Phys. 23, 1833–1843 (1996).

3. P. Xue, C. W. Thomas, G. C. Gilmore, and D. L. Wilson, ‘‘Anadaptive reference/test paradigm: Application to pulsedfluoroscopy perception,’’ Behav. Res. Methods Instrum.Comput. 30, 332–348 (1998).

4. R. Aufrichtig, C. Thomas, P. Xue, and D. L. Wilson, ‘‘Modelfor perception of pulsed fluoroscopy image sequences,’’ J.Opt. Soc. Am. A 11, 3167–3176 (1994).

5. D. L. Wilson, K. N. Jabri, P. Xue, and R. Aufrichtig, ‘‘Per-ceived noise versus display noise in temporally filtered im-age sequences,’’ J. Electron. Imaging 5, 490–495 (1996).

6. K. N. Jabri and D. L. Wilson, ‘‘Detection improvement inspatially filtered x-ray fluoroscopy image sequences,’’ J.Opt. Soc. Am. A 16, 742–749 (1999).

7. P. Xue and D. L. Wilson, ‘‘Detection of moving objects inpulsed x-ray fluoroscopy,’’ J. Opt. Soc. Am. A 15, 375–388(1998).

8. P. Xue and D. L. Wilson, ‘‘Effects of motion blurring in x-rayfluoroscopy,’’ Med. Phys. 25, 587–599 (1998).


9. R. N. McDonough and A. D. Whalen, Detection of Signals inNoise, 2nd ed. (Academic, San Diego, Calif., 1995).

10. L. B. Stelmach and P. J. Hearty, ‘‘Requirements for staticand dynamic spatial resolution in advanced television sys-tems: a psychophysical evaluation,’’ J. Soc. Motion Pict.Telev. Eng. 100, 5–9 (1991).

11. P. J. Hearty, ‘‘Achieving and confirming optimum imagequality,’’ in Digital Images and Human Vision, A. B. Wat-son, ed. (MIT, Cambridge, Mass., 1993).

12. J. H. D. M. Westerink and C. Teunissen, ‘‘Perceived sharp-ness in moving images,’’ in Human Vision and ElectronicImaging: Models and Applications, B. E. Rogowitz and J.P. Allebach, eds., Proc. SPIE 1249, 78–87 (1990).

13. A. E. Burgess and H. Ghandeharian, ‘‘Visual signal detec-tion. I. Ability to use phase information,’’ J. Opt. Soc.Am. A 1, 900–905 (1984).

14. M. P. Eckstein, J. S. Whiting, and J. P. Thomas, ‘‘Role ofknowledge in human visual temporal integration in spa-tiotemporal noise,’’ J. Opt. Soc. Am. A 13, 1960–1968(1996).

15. D. J. Lasley and T. E. Cohn, ‘‘Detection of a luminance in-crement: effect of temporal uncertainty,’’ J. Opt. Soc. Am.71, 845–850 (1981).

16. A. C. Guyton, Textbook of Medical Physiology, 9th ed.(Saunders, Philadelphia, Pa., 1996).

17. A. F. Fuchs, ‘‘Saccadic and smooth pursuit eye movementsin the monkey,’’ J. Physiol. (London) 191, 609–631 (1967).

18. R. W. Baloh, W. E. Kumley, A. W. Sills, V. Honrubia, andH. R. Konrad, ‘‘Quantitative measurement of smooth pur-suit eye movements,’’ Ann. Otol. Rhinol. Laryngol. 85, 111–119 (1976).

19. L. Schalen, ‘‘Quantification of tracking eye movements innormal subjects,’’ Acta Oto-Laryngol. 90, 404–413 (1980).

20. L. D. Loo, K. Doi, and C. E. Metz, ‘‘Investigation of basic im-aging properties in digital radiography. 4. Effect of un-sharp masking on the detectability of simple patterns,’’Med. Phys. 12, 209–214 (1985).

21. A. E. Burgess, ‘‘Statistically defined backgrounds: perfor-mance of a modified nonprewhitening observer model,’’ J.Opt. Soc. Am. A 11, 1237–1242 (1994).

22. G. S. Lisberger, E. J. Morris, and L. Tychsen, ‘‘Visual mo-tion processing and sensory-motor integration for smoothpursuit eye movements,’’ Annu. Rev. Neurosci. 68, 453–461(1987).

23. J. G. Robson, ‘‘Spatial and temporal contrast-sensitivityfunctions of the human visual system,’’ J. Opt. Soc. Am. 56,1141–1142 (1966).

24. D. H. Kelly, ‘‘Retinal inhomogeneity. I. Spatiotemporalcontrast sensitivity,’’ J. Opt. Soc. Am. 1, 107–113 (1984).

25. D. H. Kelly, ‘‘Motion and vision. II. Stabilized spatio-temporal threshold surface,’’ J. Opt. Soc. Am. 69, 1340–1349 (1979).

26. P. Xue, R. Aufrichtig, and D. L. Wilson, ‘‘Detectability ofmoving objects in fluoroscopy,’’ in Medical Imaging 1996:Image Perception, H. L. Kundel, ed., Proc. SPIE 2712, 2–8(1996).

27. A. E. Burgess, ‘‘Comparison of receiver operating character-istic and forced choice observer performance measurementmethods,’’ Med. Phys. 22, 643–655 (1995).

28. D. M. Green and J. A. Swets, Signal Detection Theory andPsychophysics (Wiley, New York, 1966).

29. H. Fujita, K. Doi, and M. L. Giger, ‘‘Investigation of basicimaging properties in digital radiography. 6. MTFs ofII-TV digital imaging systems,’’ Med. Phys. 12, 713–719(1985).

30. A. E. Burgess, ‘‘Visual signal detection. III. On Bayesianuse of prior knowledge and cross correlation,’’ J. Opt. Soc.Am. A 2, 1498–1507 (1985).

31. R. M. Manjeshwar and D. L. Wilson, ‘‘Effect of spatial loca-tion uncertainty on human observer performance in x-rayfluoroscopy noise,’’ Ann. Biomed. Eng. 26, Suppl. 1, S–13(1998).

32. H. L. Kundel, C. F. Nodine, L. Toto, and S. Lauver, ‘‘A circlecue enhances detection of simulated masses on mammo-gram backgrounds,’’ in Medical Imaging: Image Percep-tion, H. L. Kundel, ed., Proc. SPIE 3036, 81–84 (1997).

33. D. L. Wilson, K. N. Jabri, and P. Xue, ‘‘Modeling human vi-sual detection of low-contrast objects in fluoroscopy imagesequences,’’ in Medical Imaging 1997: Image Perception,H. L. Kundel, ed., Proc. SPIE 3036, 21–30 (1997).

34. H. Collewijn and E. P. Tamminga, ‘‘Human smooth and sac-cadic eye movements during voluntary pursuit of differenttarget motions on different backgrounds,’’ J. Physiol. (Lon-don) 351, 217–250 (1984).

35. M. Livingstone and D. Hubel, ‘‘Segregation of form, color,movement, and depth: anatomy, physiology, and percep-tion,’’ Science 240, 740–749 (1988).

36. J. S. Whiting, M. P. Eckstein, C. A. Morioka, and N. L. Ei-gler, ‘‘Effect of additive noise, signal contrast, and featuremotion on visual detection in structured noise,’’ in MedicalImaging 1996: Image Perception, H. L. Kundel, ed., Proc.SPIE 2712, 26–38 (1996).

37. M. P. Eckstein, J. S. Whiting, and J. P. Thomas, ‘‘Detectionand contrast discrimination of moving signals in uncorre-lated Gaussian noise,’’ in Medical Imaging 1996: ImagePerception, H. L. Kundel, ed., Proc. SPIE 2712, 9–25 (1996).

38. N. L. Eigler, M. P. Eckstein, K. N. Mahrer, and J. S. Whit-ing, ‘‘Improving detection of coronary morphological fea-tures from digital angiograms: effect of stenosis stabiliza-tion display,’’ Circulation 89, 2700–2709 (1994).

39. J. P. Flipse, G. D. Wildt, M. Rodenburg, C. J. Keemink, andP. G. M. Knol, ‘‘Contrast sensitivity for oscillating sinewave gratings during ocular fixation and pursuit,’’ VisionRes. 28, 819–826 (1988).

Role of phase information and eye pursuit in the detection of moving objects in noise

Documents

Transcript of Role of phase information and eye pursuit in the detection of moving objects in noise