Murapnihnleynl . ~ .,] 28, N , 11, pp I Cr I1J'- . 1990 .

Mll-1932 90 51 f1) 0(5)Rimed in (irmr nmu,a

1990 Pergamoo Prexa pit

SENSITIVITY TO EYE GAZE IN PROSOPAGNOSIC PATIENTSAND MONKEYS WITH SUPERIOR TEMPORAL SULCUS

ABLATION

R. CAMPRFLL,*+ C . A . HEYWOOD,* A . CowLy,* M . REGARDt and T . LANDISt

'Department of Experimental Psychology, Oxford University, U .K . ; and tUniversitaetspital, Zurich .Switzerland .

IReceired 17 October 1989: accepted 12 .lune 1990)

Abstract-Accuracy at perceiving frontal eye gaze was studied in monkeys and human subjects usinga forced-choice detection task on paired photographs of a single human face . Monkeys learned thetask readily, but after bilateral removal of the banks and floor of the superior temporal Sulcus (STS)they failed to perform the task efficiently . This result is consistent with the conclusion, based onrecordings from single cells in awake, behaving monkeys [PERRET e1 at. . Phrsiofoyfeal Aspects ofChuh-al Neurn-ophlhahnologp, Chapman & Hall . London . 1988] that this region of the temporal lobeis important for coding information about eye-gaze of a confronting animal . Human subjects weregiven identical stimuli in a task where they were asked to detect 'the face that is looking straight atyou'. Human performance is sensitive to the degree of angular deviation from the frontal gazeposition, being poorest at small angular deviations from 0 . This was also true of monkeys viewingthese stimuli, pre- and post-operatively .Compared with normal controls, two human prosopagnosics were impaired at this task . However

the extent of impairment was different in the two patients . These findings are related to earlier reports(including those for patients with right-hemisphere damage without prosopagnosia) . to normalperformance with upright and inverted face photographs, and to notions of independent subsystemsin face processing .

INTRODUCTION

SENSITIVITY to frontal eye gaze is important in sighted, social vertebrates [23] . While socialpsychologists have described the range, variation and functional significance of gaze in themutual control of social interactions [19], perceptual psychologists have attempted todetermine the accuracy with which we can detect whether we are looked at in order tomeasure thresholds for gaze detection [I, 10, 16, 30] . The present study investigates thisability to discriminate angle of regard in monkeys and in human subjects using photographsof a human face in a forced choice detection task. where the correct response depends onidentifying which one of a pair of faces is "looking at you" .

The human perceptual system is highly sensitive to deviations from frontal view whenphotographs of full faces are viewed under good illumination . ANTIS of al. [1] estimate that at122 cn) viewing distance . a horizontal "displacement of the iris of 0 .18 mm is just noticeableto the subject" (p . 475). This indicates that good visual acuity is a prerequisite of fine gaze

;Address for correspondence : R . Campbell, Department of Psychology, Goldsmiths' College, University ofLondon. London SF14 6NW . U .K . : or C . Heywood, Department of Experimental Psychology, University ofOxford South Parks Road . Oxford OX I 3UD, U .K .

11 2 3

1124

R. CApIPiirLL CI n/,

detection . Antis er at. further show that the contrast of the coloured iris with the whitehuman sclera contributes greatly to the ability to perform the task . In monkeys the sclera isnot usually visible in eye-contact : same-species eve-gaze computation must therefore dependon other factors, e .g. the relative displacement of the pupil in the (very dark) iris andreflections of incident light from the front of the eye probably determine perceptualsensitivity .

In this study we examined frontal eye gaze discrimination using photographs of a singledark-eyed man . The foil pictures (those of deviated eye-gaze 1 showed the face with horizontal(lateral) deviation of the eyes of 5 . 10 and 20 to the left or to the right . The angle of the headwas independently manipulated in the photographs . It could face forward or diverge 20 toeither the left or the right . A sample pair of photographs is shown in Fig . I .

Identical stimuli were used in a previous study to investigate frontal eye gaze . PF:RRF-rT eial . [25] . following DESIMONE et ill . [ 14], used single cell recording techniques to confirm thata population of cells in the superior temporal sulcus (STS) of the macaque monkey issensitive to frontal views of human and simian faces, and that other populations in the sameregion were tuned to profile and back views . Perrett er al . then found that. of the frontallytuned cells .. a majority show further sensitivity when the eves are directed to the viewer .compared with when the eyes are averted : that is, this cell population demonstrates aconjoint sensitivity to front views of faces and frontal eye gaze Most of the cells in thispopulation are not sensitive to facial identity . Other cells in the same cortical area are,however. sensitive to facial identity irrespective of lighting conditions, perspective view orfacial expression [2 . 26, 27], while yet another discrete cell population in the STS seems to hesensitive to facial expression . irrespective of identity [25] . These separate functionalsubsystems are in anatomically coherent clumps alongside other cell populations which arenot particulary face-sensitive [26] .

One natural inference from these studies is that impaired identification of faces andrecognition of their expression . resulting from temporal lobe damage in human patients . maybe associated with impaired discrimination of gaze . It is assumed that an area of the humantemporal lobes, analogous to STS in macaques . would have to be damaged or disconnectedfor these associated deficits of face processing to occur . One such case was reported by Perrettet al . [24] . R .B . . known to be impaired at recognizing facial identities and it recognizingexpressions in faces . was also impaired in the discrimination of frontal eve gaze . Hisperformance was out of range of normal performance and also signficantly worse than that ofa group Ell =39) of patients with right hemisphere lesions without prosopagnosia. His visualfunctions were otherwise good .

These associated impairments in one patient need not lead us to expect them in all patients .As Perrett's own studies show, discrete cell populations, though anatomically intermingled_may underlie these different aspects of facial perception . Lesions to segregated outputs fromthese subsystems could cause dissociated impairments in face processing, such as have beenobserved in some human cases . Identification of faces can be impaired in patients withlocalized lesions (prosopagnosia) when categorization of facial expression is intact [6 . 29 .32] . One aim of the present paper is to explore further the co-occurrence of identity,expression and gaze impairments in prosopagnosia . by ex aming two women with associatedimpairments of identity and expression classilicit lion on a task of gaze-lodgement .

A first aim of the present paper. however, is to test the chain of reasoning that runs from thesingle-cell studies of macaques to human cognitive deficits . Although single cell recordingssuggest a particular functional role for STS cells in the intact animal it need not follow that

Sample pair of face

se. the face

snNsrnvtTN TO EYE GAZF 1127

lesions of areas containing those cells will inevitably produce a deficit in the processingstimuli that they had been tuned to . Some lesions are "silent", suggesting that the particulartested function is supported by multiple systems in the intact animal . This question isparticularly important in considering face processing . Deficits in face processing followingdiscrete lesions are relatively rare . This has led several investigators to conclude that thefunction formerly subsumed by the lesioned area is readily performed by other areas of thebrain . This issue arises with particular force when the lateralization of face processing isunder discussion . for while in human populations there is general agreement that the righthemisphere has an important role in face processing [28] . it has been hard to demonstratethat localized right hemisphere lesions alone can produce face processing impairments (butsee [12, 20. 21]) .

The studies of Perrett et al . showed single cell sensitivity to frontal eve-gaze . but the precisebehavioural correlates of this sensitivity are unknown . While monkeys are undoubtedlysensitive to eye-gaze and head position in natural settings [23], the extent to which they maybe able to learn a laboratory task on the basis of this sensitivity is not known . We thereforetested the ability of monkeys to discriminate angle of regard with the same stimuli used forhuman subjects . If ablation of the so-called face cell area in STS impairs this ability, theextrapolations from single cell recording to human neuropsychological deficit will be firmer .If dissociated impairments in gaze sensitivity, using the same task, can then be demonstratedin humans with facial emotion and identity processing deficits, the functional separation ofdifferent face processing subsystems, suggested by the single-cell studies, would he furtherupheld .

EXPERIMENT WITH MONKEYSDo bilateral STS lesions impair the discrimination of angle of regard in monkeys'! PERRFTT et al . [24] It aveshown

that some STS cells are tuned to frontal eye-gaze in macaque monkeys . The present study examines the ability of themacaque to perform a task requiring the discrimination of frontal eye gaze before and after ablation of STS . Themonkeys also learned to discriminate patterns and colours .

SubjectsFive adult rhesus monkeys (Macao mulatto) were used [STS- I-STS-5 ) . All animals had previous experience u h

visual discriminations in a Wisconsin General Testing Apparatus (WGTA) . Two animals (STS-4 and STS-51 hadparticipated as unoperated controls in a previous experiment [17] . The remaining three had served as unoperatedcontrols in both an automatic apparatus for the testing of contrast sensitivity, vernier acuity, contrast matching andspatial frequency discriminations and in a WGTA for the testing of colour discrimination [ [8] . Thus, the monkeyshad rather different visual and learnin g experiences prior to this experimental study and this should he borne in mindin interpreting the results .

Sur,ger1' and histnlu .It

Surgical details are described in detail elsewhere 117] . Bilateral cortical ablations were made by aspiration underdeep general anaesthesia and under aseptic conditions . The ablations included both banks and the floor of thesuperior temporal sulcus . From a position 5 mm in front of the inferior occipital sulcus. the lesion extended some20 mm anteriorly and included the area in which a substantial proportion of neurons respond selectively to faces[27] . Histological reconstructions for STS-4 and STS-5 have been published elsewhere [17] . For the remaininganimals the lesions were as intended and did not differ in any essential way from the published accounts .Reconstructions are presented in Fig . 2 .

ApparatusA description of the apparatus and procedure is available elsewhere [17] . Briefly, testing was carried out in a

Wisconsin General Testing Apparatus . Stimuli mounted beneath clear perspex plaques were place side by side on asloping black hoard (56 cm wide x 30 cm high) that confronted the animal . The stimuli rested on a ledge and eachcould be displaced laterally to reveal a food well . The experimenter was concealed behind a one-way vision screen .Between trials a second opaque screen could be lowered between the animal and the discriminanda to allow the

I t2g

R . CAMIBLLL et al .

STS-1 STS-2 STS-3

5cmFig .'_ . Diagrams showing position and extent of cortical lesions. Ablated [issue is show n in Mark onboth the surface views of the brains and the drawings of coronat sections immedmtek beneath . Foreach animal, caudal is at the top and the left hemisphere is at the left . The lines alongside the brain,

indicate the position of the representative corona[ sections .

position of the rewarded stimulus to he varied according to a predetermined random schedule . the animalresponded through the hats of a transport cage and, if the choice was correct, retrieved a pcmut reward from thefood well .

METHODDiorrintinatlnrn learning : trot materials

'[he animals learned a sequence of eight two-choice visual discriminations each to a criterion of 9t) i correct, insequence . All discrintinanda were moonted on 12 .5 cm x 12 .5 em plaques . They were t I t a plain white plaque vs aplain Mack plaque with a I cm white border . (2) A plus square discrimination where each horizontal or verticalcomponent of tlic pattern measured I cot x 75 stn . 131 A red vsa green plaque, each with a I cm white border. (41A4-asclc honzontal square wove grating where each bar was 1 cm x 9 cm vs an identical but vertical grating t5 81Four scpurac "so-choice discriminations of monkey faces where each face was a black and white photograph(9 stn x 1'_ .5 urn) of a different monkey .

The second part of the experfmem required the animals to discriminate between G¢es on the basis of eve position .that is . it attempted to teach the monkeys the "rule" that an eyes-averted face was rewarded, using a set ofphotographic stimuli. For this a series of colour photographs was prepared as follows . 'I he photographst'-l em x 12 em (were all ofthe single face shown in Fig 1 . The face was photographed in one of threedificrent head

sENstrINI t 5 TO 5YF CALF ]129

orientations . 20 to the left or right or face on . For each position of the head the eyes could he in six possiblepositions . 5 . 10 or 20 to the left or right . This resulted in a set of 18 stimuli . In addition three identical sets ofdiscriminanda were prepared where the head was in one of the three possible positions but the eves looked directly atthe observer .

Discrimination learning : procedureFollowing preoperative training [17], the animals were tested on the eight sets of two-choice discriminations .

They were tested for 50 trials a day . 5 days a week . The preoperative criterion for learning each discrimination was 90correct responses out of 100 consecutive trials . The number of errors to reach the criterion was taken as the measureof performance .

The next stage of the procedure required the animal to discriminate eve position by learning six two-choicediscriminations between pair, of faces . Six pairs of photographs were selected in which the orientation of each headin each pair was identical (left, right or straight-ahead) and in which one of the faces looked ahead 0 ' angulardeviation I while the other had eyes averted 20, to the left or the right . In each case the choice of the stimulus wherethe eves were averted was rewarded (eye-contact can be a threatening stimulus to monkeys : eye-aversion is thereforemore compatible with a positive choice response) . Each discrimination was learned to the criterion described above .In this way the "learning rule" to be mastered by the monkey was taught using faces where head deviation was anirrelevant cue and where 20 eye deviation to the left or right was the only criterion for reward .

Appipiny the learning rule : concurrent discrimination a) ere-ya^eFinally, the animals were tested for 6 days on a concurrent discrimination task where . on any trial. the rewarded

stimulus was randomly selected from among the pool of 18 faces with eyes averted and the unrewarded stimulusfrom among the three faces with eye contact . The availability of the three sets of the latter enabled differentexemplars of the same physical stimulus to be used which reduced the possibility that animals were responding onthe basis of artifacts in a few specific stimuli . Within a session, each rewarded stimulus was paired once with eachunrewarded stimulus resulting in daily sessions of 54 trials . Over 6 days of testing this procedure resulted in 108 trialseach for the discrimination of 5 . 10 and 20' of eve deviation from eye contact . The measure of performance was thepercentage correct responses for each angle of eye deviation .

Post-operative testing was resumed 3 weeks after surgery . Animals were retested on the same tasks that werelearned preoperatively and which were presented in the same order .

RESULTSI'wo sets of data were examined; the first concerned the rate of learning of the various

visual discrimination tasks and the exemplars of the gaze-discrimination task (six face-pairs) .The second set of data, error on concurrent discrimination of gaze, showed how, once theyhad been taught the gaze-contingent learning rule they applied it to the concurrent gaze-detection task . Pre- and post-operative performance were compared for both dependentmeasures .

Visual discrimination learning

For each animal, the number of errors made (including errors made in the 100 criteriontrials) to reach pre-operative criterion on the six pattern discriminations was compared withthe number of errors needed to reach criterion post-operatively . Paired r-tests between pre-and post-operative mean error scores failed to show a significant difference (I= 1 .18, d .f. 4,P>0.2). This shows that, following bilateral removal of STS, pattern discriminations wererelearnt at about the same rate at which they were acquired pre-operatively . This was the casewhen either the 4 face or 2 pattern discriminations were analysed separately (t=2 .43, d .f . 4,P>0.05 : t=0.58 . d .f .4. P>0.25, respectively) . However, it should not he concluded thatSTS lesions fail to affect visual discrimination learning, for the following reasons : first, theresults for two of the animals (STS-4 and STS-5) have been reported previously [ 17] . Theseanimals showed substantial post-operative savings on the pattern discriminations . They hadpreviously served as unoperated controls for the same series of pattern discriminations andthis additional experience in the previous experimental task could account for their superior

1 13 0

R. CAMPBELL et dl,

performance . Second, there were no internal controls in this study ; that is, no animals withsham or other operation with which to compare the present animals . However, results areavailable from four unopcrated animals, including the two referred to above, for the series ofpattern discriminations reported previously [17] . While the animals in the present studyshowed mean (post-operative) savings of 35 .1%, (where savings refer to the differencebetween pre- and post-operative scores. divided by their sum) this may usefully he comparedwith mean savings of 62 .9% shown by unoperatcd animals for an identical series ofdiscriminations .

Eye-gaze discriminationThe six two-choice discriminations of eye gaze were readily learned by all animals . They

initially showed lip-smacking behaviour to the photographs . indicating that they saw thephotographs as faces rather than as meaningless patterns . The mean number of errors tolearn each discrimination was 77 (range 29-157) . Post-operatively . there was a mean savingof 37 .4% for the retention of the discriminations . However, it must he noted that savingsscores can be misleading when problems are learned or relearned with few errors since smalldifferences in error scores then lead to large differences in ratios . Once again, as there were nointernal controls in this study. we do not know how this savings rate might compare with thatfor animals with other lesions . At all events, following surgery, animals were not severelyimpaired in re-learning the rule exemplars that teach that averted eve discrimination isrewarded . How effectively do they apply this rule?

Figure 3 shows the accuracy of concurrent discrimination performance of eye gazediscrimination both pre- and post-operatively for the three tested angular eye deviations .

5

10

m •ANGULAR DEVIATION OF EYES

Fig 3 . Concurrent discrimination of gaze in monkeys: mean accuracy pre-and post-operatively as afunction of eye-deviation of the photographs ISF shown as vertical bare I .

SENSITIVITY TO EYE GAZE

Analysis of variance was carried out on the data using lesion and eye deviation as factors .Pre-operatively, animals performed at better than 50% correct in discriminating eye gaze at5, 10 and 20` of eye deviation. Post-operative performance showed a substantial reduction inthis ability (F=8 .78, d .f., 1, 4, P<0 .05) and the absence of a significant interaction betweenlesion and eye deviation (F= 1 .76, d.f ., 2, 8, P>0 .05) shows this to be true at each of thethree levels of difficulty . Performance overall was better as the degree of eye deviationincreased (F= 16.96, d .f., 2, 8, P< 0 .01). Furthermore, Fig. 4 shows percentage correct post-operative performance plotted as a function of the head deviation of the unrewardedstimulus . If animals were responding on the basis of head deviation . this would be reflected inan uneven distribution of scores at each of the three possible head positions . An analysis ofvariance using lesion and head deviation as factors reveals no main effect of head deviation(F= 0 .71, d.f.. 1, 4. P> 0.05) confirming that animals were using eye position to perform thediscrimination .

100

I

a 50

a"re-opPost-op

0woe

mr

enogn

aignr

HEAD DEVIATION OF S-

Fig . 4 . Concurrent discrimination of gaze in monkeys : accuracy as a function of head deviation ofphotograph (SE shown on vertical hars)

Thus, prior to operation, monkeys learn effectively to use angle of regard as a cue toobtaining rewards . They are sensitive to angular deviation of the eyes, performing less well atdeviations of 5` than at 10 or 20" . Following STS ablation, the rule is relearned but is appliedless accurately-at all angular deviations . The animals do not seem to use a differentrule/strategy (such as head deviation) .

DISCUSSIONExtending earlier studies of cortical lesions on visual discrimination learning [17, 18], the

present study shows that (a) rhesus monkeys readily learn to discriminate human faces on thebasis of angle of regard and (b) following STS ablation this ability is impaired-the animalsappear to retain the same strategy as before the lesion, but apply it less accurately .

1 13 1

1 13 2

R. Cnmpnat et al .

This study does not indicate whether the deficit following STS lesion is specific to face-related tasks, for the experiment did not investigate the learning and application of otherspecific cue-contingent rules . By inference from other studies, and considering the particularexperience of the monkeys in this study, we suggest that STS lesions can induce a generalized,but probably quite mild, impairment in relearning tasks where visual discriminations havebeen learned pre-operatively (e .g . gratings, black white, red green plaques ; pairs of differentmonkey faces) . We do not yet know whether this impairment may contribute to the deficit inlesioned animals in learning the rule-governed concurrent eve-gaze discrimination task orwhether it is independent of it .

Although there was marked variability among the animals with respect to their ability torelearn specific visual discriminations following operation (as there wass to learn them in thefirst place), the data on the concurrent eye-gaze discrimination task_ which involved learningand applying a "rule", are unambiguous, with small variability from animal to animal . Weare confident, therefore, that rhesus monkeys can readily learn to discriminate (human) faceson the basis of eye gaze, once they have been shown the salient stimuli . and that STS lesionsimpair this ability markedly and systematically .

Finally, it should he noted that despite total absence of the temporal neocortical area,where many cells are tuned to some aspects of faces . these monkeys were able promptly torelearn to discriminate between photographs of monkey faces . Even if mildly impaired, theywere not more impaired than when discriminating between colours or geometrical shapes .They therefore have no marked apperceptive prosopagnosia . Whether they can recognizefaces . i .e . identify them absolutely or perceive them as familiar or unfamiliar, is still notknown .

HUMAN STUDIESThe aim of the studies with human subjects was to explore their ability to detect frontal

eve-gaze in different populations and under different conditions . The same paired stimuliwere used as in the animal study of concurrent discrimination learning hut, rather thanlearning a rule by training, subjects were simply given verbal instructions to ' - choose the facethat is looking straight at you" in a forced choice task . We intended to discover the extent towhich patients with deficits in face processing may be impaired in this task- primarily incomparison with normal controls, but also with respect to other data . including the animaldata above .

YntaoPaypinn,ir „ this, .,,,

I ko prosopagnosic women were studied . Their details are summarized below .1, 1) belie! 198% I . K .D . is female and was born in 1922 . She suffered a right posterior cc rebraI artery infarct in

1982 . since w hen her condition has remained stable . Her prosopagnosia has been evident since first testing in 1983 .In I982 .she showed a left homonymous hemianopsia which resolved in a defect of the whole of superior quadrantand part of the adjacent inferior quadrant, Her perceptual psvehophvsical functions . Including sensitivity to allspatial frequcncie, were tested in 1986 and found to he normal (Dr l . Re ntsehIcrl . Shc has topographagnosiu . Shehas feelings of unfamiliarity with previously known places and people when exarnined by sight . Shc shows somemood depression . occasionally marked . There is EEG slowing over right posterior Temporal regions . Her reading .writing. speech . semantic and episodic memory are normal . She shows no neglect in reading or in any visual task .She i, not ;mosognostc There is no apraxla Fill] clinical and anatomical descriptions of K .D, can he foundelsewhere (9_ 20] . A CT scan taken in 1986 . showing the relevant corona) sections, is shown in Fig, s

1 13 I(csled 19%81. A.Bdis female and was horn in 1961 . She was referred for psychological testing alt lie age of 13because of"iumsinesi' and "poor recognition' at school . There was ill) reported trauma perinatally or in infancy .There i, a Cmtily history of - poor visual recognition', her mother being particularly impaired at face recognition(personal communication) following testing, .4 .B . was described by MncCnsArna 1'_2] a, n case of

Fig . 5 . Computer tomography coronal scan of K .D. (1986) showing area of lucency in right medio-temporal occipital region consistent with infarct in distribution of the right parietal artery .

SENSITIVITY TO EYE GAZC

1 13 5

developmental prosopagnosia . Posterior right parietal EEG abnormality was reported in 1976 . but no scans areavailable .

Visual and general cognitive function were tested again in 1981 (Dr M . W yke) . A .B . s colour vision was good . Hervisual acuity, measured with corrected vision on Snellen charts . was normal . Her performance on the Gollin test wasat the poor end of normal range . Line-length estimation was good . Despite persistent reports of general clumsinessthere was no clinical apraxia . On I.Q. tests (AHS and WAIST she showed a significant %erbal performancediscrepancy [e .g . WAIS (verbal) was 148 : WAIS (non-verbal) was 120] .

Her high IQ scores attest to A .B : s intellecutal abilities: she performed well in public school-leaving examinationsand studied at University . In 1985 she attained a research doctorate and now does independent research in a leadinguniversity .

Further tests (1986) showed normal contrast sensitivity over the range of spatial frequencies tested (CambridgeLow Contrast Gratings : [33]j . Topographagnosia and left--right confusion were noted . There were no signs ofneglect in any mode. Her reading and writing were excellent and her performance on Warrington s Old' New Wordrecognition test [31] was perfect . She also showed fast and accurate performance on Benton and van Allen s tests ofVisual Function [4] . including tests of line orientation matching and of visual form recognition .

Tests of face processing were administered to both women in 1988-1989 . These are summarized in Table I and arcdetailed below .

Table 1 . Summary of Face Processing Deficits in K .D . and A .B .

Key : - =impaired. -=unimpaired performance .In both people . the deficit in person recognition is limited to the seen face and

does not extend to identification by voice or description . In K .D. the expressiondeficit is face-specific .*K.D . was given a test of old new face recognition similar to Warrington and

showed impairmenttA.B . was able to lipread silent speech but was insensitive to the auditory-visual

fusion illusion (see [BJI .

Tests offuciol recnymliOnFatuous face recognition . Photographs (coloured and monochrome) of 10 celebrities which were readily and

reliably named by age and culture-matched controls were presented for identification . Subjects were encouraged Inname the face and . if that was not possible. t o say anything they could about the person portraye( . Fxamples ofsuchfamous faces used for both subjects are Ronald Reagan (then president of the USA) and Marilyn Monroe- Neithersubject named or reliably described a single face .

Familiar J'ore reroypinion . Recent photographs of family members and colleagues . including the experimentalteam. were shown -These subjects were unable to reiably identify (by name or description I any of the photographsNeither subject reliably recognized a photograph of herself .

Warrington's recognition nreniort' test for fates [31] . This test required inspection of 50 unfamiliar facephotographs (black and wane . approx 6 x 6 coil which were presented one at a time in abound hook formal . ] herecognition set comprised all SO faces intermixed with S(1 new faces . The subject had to class each of the 10(1 faces n,familiar or unfamiliar. Neither A .B . nor K .D. scored better than 5(1% correct-

K .D- A.B .

Face recognitionFamous face recognition + +Familiar face recognition + +Old-new face recognition (Warrington I +' +

Matching facesBenton and Van Allen 37 39

(mild) (borderline)Photos/line-drawings (Landis) + +

Structural aspectsScrambled: unscrambled faces -Agejudgemcnts + +Gender judgements r 4Expression (Ekman face sorting) + +

Lipreading - -t

1 13 6

R CAMPBELL '1 a! .

Trnrt of lace rnatrlarrll

BIsins and VAN ALLEN [5J . This test requires matching of a small 13 x 3 cm) monochrome face photograph tothose i n a set of six possible matches . The matching set is presented simultaneously with the target . The correct match isto a face transformed by angle of view or by Iighting from the original . A series of 20 such faces was presented formatching- Scores of 39 and 37 for A .B . and K.D . . respectively indicate mild impairments of face matching .

Landis For'- .blare king test . A highly schematic line drawing of a face Ihappv, sad, angry I is presented to which thesubject seeks a match from a set of photographs of profile-view real faces each showing one of thesc emotions . K .D-was unable to perform this task [7 . 9] . Similarly . A .B . performed poorly, making one correct match nut of livepossihle one,

Srramhled Jane detection . Photographic collage and Iine-drawings of full faces were made in which the internalfeatures eeyes, nose . mouth I were misplaced . Among these were several in which the features were correctly disposedbut the feature itself was wrong 0 .e . a nose in an eye position I . These were mixed with correctly configured stimuliand the subjects were asked to sort the malerial into pictures of real and unreal faces . Neither A .t3 . nor K . D .madeasingle error on this task .

Age and gender judgements . Using the previously presented photographs of unfamiliar faces, subjects were askedto estimate the age and gender of each face. While control subjects performed flawlessly . A . H . and K-D. made 3 and 4e rrors . respectively . i n judgements of 20 faces .

Expression cateyoricatiou . Ekman and Friesen's set of photographs of facial expression were used [15] . Thesecomprise more than 60 full face photographs nfseveral sitters of different age, gender and ethnicity . portraetug thefacial expressions reliably interpreted as Grief. Anger . Fear . Rage. Surprise, Disgust . Happiness and Sorrow .Subjects were given the appropriate expression labels and asked to sort the face photographs appropriately . A .BKand K.D. performed this task slowly and hesitantly . Their performance was 65-75"•'% correct and errors "ere notconsistent over repeated testing . Their performance was significantly worse than normal controls .

Lipreadiny The lipreading tests are described in derail elsewhere for K .D . [7, 8] and for A .B . [R] .

Control subjects : upright and inverted choice tusk . Ten right-handed subjects aged from 25to 60 yr were tested . Of these 10, 7 performed the task twice ; once with correctly orientedfaces, the other time with inverted face-pairs . The reason for examining inverted faces was totest the extent to which a full face configuration, comprising features in their correct relativepositions . i s necessary for adequate performance of the task . In inverted faces only therelative horizontal position of the eyes, which are themselves inverted, is maintained . Is thissufficient to enable gaze detection to occur accurately'?

METHODThe same picture pairs were presented to subjects as were used with the animals- They were presented on cards

measuring 20 x 12 cm . one at a time by the experimenter, at a viewing distance of 0 .5 m . at a rate determined by thesubjects' response Subjects were told the following : "One of each pair of the pictures I will show you will he of a facelooking directly at you- You must (point toor) tell the which face is looking directly at yon" . Subjects were not givenfeedback and were not allowed to re-view pictures-Three subjects who performed both inverted and upright conditionssaw the upright faces first, four the inverted ones .

For the prosopagnosic patients instructions were repeated three of four times during performance (if the task_ Carewas taken with K .D . to present the material in her intact visual field (there was no evidence of neglect or scutome inA .13 .) .

RESULTSFigure 6 shows the relationship between accuracy of forced choice detection of frontal eye

gaze and angular deviation of the foil picture for normal subjects viewing upright andinverted faces and for the prosopagnosic subjects viewing upright faces-

A within-subjects Analysis of Variance (n=7) examined angular deviation and orientationas factors affecting accuracy fit normal subjects. An overall effect of angular deviation wasconfirmed (F=7 .3, d .f ., 2, 6. P<0.02) which interacted with orientation (F=4.8, d .f ., 1, 6,P<0.05) . Post hoc tests showed that while deviations of 10 and 20" were discriminated fromfrontal view at ceiling levels of accuracy, deviations of 5` from ahead were significantly lessaccurately reported . Similarly . it was only at 5' that the interaction with orientation wassignificant, with inverted faces being less accurately perceived .

SFNSITIVITV TO EYE GAZE

mood controls' Imverted feresl

-I AB

2p

ANGULAR EEVIAP.ONOF EYES iFOILi

Fig . 6, Concurrent discrimination of gaze in human subjects : normal controls with upright andinverted faces and the prosopagnosic subjects K .D. and A .B .: mean response accuraco as a fund ion of

eve deviation (SF shown on vertical hars)

Thus, for normal viewers, inverting the face impairs accuracy of gaze detection only whensmall angular deviations from the centre need to be discriminated .

As Fig . 6 shows, K.D. showed some sensitivity to the angular deviation of the foil picture .She performed within normal range at 10 and 20" of eye deviation (upright faces) . She wasnot so good at discriminating 5 of eye deviation from the straight-ahead face : herperformance is more than 1 .8 SD's worse than that of the control group for upright faces(n=10) and within range of normals who viewed inverted faces at 5 .A .B. showed no sensitivity to angular deviation . She is out of range of normal subjects (> 2

SD's) at all angular deviations . Her performance was no better than would be expected fromguessing (50% accurate) when both heads faced forward or when both heads deviated . Bycontrast she was only 16% accurate when one head deviated and one faced straight ahead .always choosing the head facing forwards . In other words, she used only head deviation inorder to solve the task . A .B .'s discrepant performance was apparent during testing, butrepeated instructions to attend to eye-gaze and not to head orientation had no effect .

PFBBETT et al . [24] describe data which can be compared to these for a number of humansubjects . Those investigators, however, used a different paradigm in which each face waspresented individually for subjects to

judgewhether it was looking at them. Using this

technique . 39 right hemisphere-lesioned patients . without prosopagnosia, were found to heabout 7tL4, accurate in judging 5 of angular deviation and performed at ceiling levels ofaccuracy for the other angular deviations . This pattern of performance is very similar to thatof K .D. in the present study . A .B ., by contrast is extremely impaired at this task and isunaffected by eye deviation at any tested angle . Since her performance at gaze detection wasso bad we performed further tests to establish whether her judgement of line orientation oreye-like configurations was intact .

Line Orientation . Benton's test of Line Orientation [3] was administered in the course ofgeneral visual function testing . It was performed quickly and accurately . In this test a 10 cm

11?'

Ills

a

7cm

10cm

r

R . CAMVnrrl . PI alL

lint is presented at an orientation which has to be matched to a line of the same orientation ina "fan" of lines of the same size presented below the target line . We also tested her ability toidentify (point tot a target dot specified by the direction of a pointing arrow (see Fig . 7a) . Shewas perfect at both tasks .

Sample cards to test

concentric circle configuration

S points to the 'odd one out'

Sample cards to test

tine (arrow) orientation

S points to the dot that the arrow

points to

20 cards of this type presented

20 cards of this type presented

I ir.' . Sample tests of line-arrow and concentric circle discrimination for human xihleci AA3 .

< im/iyuraGon of ere-like stimuli . Triads of concentric circles . using both filled and unfilledones. were drawn on cards and given to her to pick the "odd one out" in which theconfiguration of the inner to the outer circle was changed (see Fig . 7b) . She performed thistask flawlessly and quickly .

DISCUSSION

Monkeys in which the rostra) STS has been removed failed to re-learn frontal eye-Lazediscrimination accurately in a forced-choice, paired presentation task . This is consistent withthe demonstration of single cells in STS that Lire sensitive to specific eyehead orientationsand suggests that this region is important, for the behavioural deficit was present manymonths after surgery . Nevertheless, operated animals did not behave in an unusual manneron this task : they were still sensitive to the angular deviation of the eyes irrespective of headdirection . Since these animals also showed indications of a mild impairment in visualdiscrimination ability following STS lesion the% may have a general but mild visualperceptual impairment . Impairment in visual discrimination ability was not more markedfor photographs of monkey faces than for any other sets of visual stimuli. Although facialrecognition )identification) is yet to he tested in such monkeys . they do not seem to haveprosopagnosia of the apperceptive type . Although the ability to continue to discriminatebetween different faces could have been relearned by treating the faces as patterns, they

b

SENSITIVITY TO EYr GAZE

1 13 9

showed evidence of perceiving them as faces by making the characteristic lip-smackingresponse when first presented with the photographs of faces after the post-operative restperiod of several weeks. This never occurred with non-face stimuli .Control (human) subjects performed this task at high levels of accuracy . and. like

monkeys, they were sensitive to the degree of angular deviation of the eyes .Direct comparisons between monkey and human performance arc tricky since the task

demands were so different . It is the systematically similar patterns of error as a function ofangular deviation and of lesion that are informative .

Inversion offaces and configural processingIn seven normal human subjects, discrimination of small angular deviations of the eyes

was less accurate when the faces were inverted than when they were upright, but the task wasstill performed at high levels of accuracy for the wider angular deviations of the eyes . Thismild effect of inversion contrasts with the extremely deleterious effect found when faces haveto be identified or recognized as old or new [34] . We conclude that the detection of frontalgaze need not depend on full configural information about the relative disposition of all theface features and that paired horizontal eye disposition (which is maintained in inversion)may be sufficient to support sensitivity to direction of gaze .

Prosopagnosia : dissociated impairments of gaze sensitivity

Two prosopagnosics who had very similar problems in identifying faces, in classifyingfacial emotion and in judging age and gender from the face, nevertheless showed dissociatedimpairments in the gaze task . K.D : s deficit in discriminating small differences in angle ofregard is mild and falls within range of that of RH patients without prosopagnosia [24] . Herperformance was worse at 5 than at wider angular deviations . In terms of the present study itfalls within range of normal performance on inverted faces, suggesting that a loss of someconfigurational skills may be reflected in slight deficits in the task when the angular deviationof the eyes is small .A.B . showed a complete inability to detect frontal eye gaze : her performance was entirely

predicted by angular deviation of the head . Thisis unlikely to be due to any underlying visualdeficit in detecting_ orientations or matching configurations of similar visuo-spatialproperties to the gaze photographs . This dissociated ability, in K .D. and A.B ., who otherwiseshow very similar associated face processing deficits may lead one to speculate that furtherdissociations may he forthcoming . It is possible, for example, that an isolated deficit in gazediscrimination may occur with intact face-affect and face-identity processing, thussupporting further the description of multiple independent subsystems in the STS suggestedby Perrctt and his colleagues in single cell studies .

Aetiology

A.B . is a developmental prosopagnosic . Is the congenital nature of her problem the causeof the dense deficit on the gaze task'' This seems unlikely, for patient R .B ., reported to have asimilarly dense problem by PERRFTT et al . [24] . had vascular lesions as an adult which causeda sudden onset of face processing difficulties . There are further interesting comparisonsbetween R.B . and A .B. In R .B ., as in AR, contrast sensitivity function was normal (seeDavidoff et al . [11]) . R.B . was densely agnosic for visually presented objects . Our currentinvestigations of A .B. suggest some limited agnosic deficits . She is able to name picturedobjects reliably, including within-category (sub-ordinate category) members of some classes,

1 140

R . CAuvxe1 .1 11 nL

such as flowers . She is not good . however, at discriminating within some other objectcategories (e .g . cars) . It is presently unclear whether her deficit is clinically subnormal . Sheuses and moves among objects correctly in manipulative space and in extra-personal space .

Lot aIiZuIion

K .D . has been repeatedly examined using spectral imaging techniques [9 . 20] . On CT scanshe shows no sign of left hemisphere cortical damage (see Fig . 5) : though the medial site of thelesion should be noted . The nature and extent of A .B.'s cerebral abnormalities are notknown-but she shows no obvious neurological sign of left hemisphere damage (i .e . no right-lateralized weakness, neglect or hemianopsia ; no problems in receptive speech or in reading) .It is possible . therefore, that bilateral regions which appear to he responsible for faceprocessing in monkeys could be more strongly localized to the right hemisphere in mosthumans, whether young or adult . This would leave open the possibility that the lefthemisphere may carry out some aspects of face processing, including gaze detection : forinstance in some people with surgically separated hemispheres and in patients with discreteright hemisphere lesions .

Aenitr and contrast sensitivity

The argument has been made [28] that human lateralization for face processing reflectsrelatively greater right than left hemisphere sensitivity to low spatial frequencies- at least atsome (critical) stage(s) of development [13] . The visual demands of the present task(particularly at detecting eye deviations of 10 - or less) indicate that it cannot be achieved withlow spatial frequencies alone . A.B . . with excellent acuity and good high spatial frequencysensitivity (when tested with gratings) was unable to perform the task, while K .D . was notbadly impaired . Moreover, we have pointed out that R .B . . who was also severely impaired atgaze detection, had a normal contrast sensitivity function . Thus, while good high spatialfrequency sensitivity must be necessary for adequate gaze discrimination it cannot hesufficient . The problem for R .B. and for A .B. must lie in deeper levels of processing : possiblyin the establishment of a detailed, spatially sensitive representation of face features in a facialframe. Such a deficit (essentially apperceptive in nature) could lead to problems with all theother aspects of face processing described in Table 1 . It cannot, however, explain why K .D .should be relatively unaffected in her gaze discrimination, when other aspects of her ahility toprocess faces (expression, age, identity) are as impaired as those of A .B .

The gaze detection task may thus prove particularly suitable for the further investigationof the relationship between visual function and face processing in the context of thelocalization of higher cognitive function . We hold in mind, however, the possibility thatdisturbances of visual function may contribute to some aspects of face processing debility .and we certainly cannot rule this possibility out on the basis of either the animal or thehuman data presented here.

CONCLUSIONSMonkeys and human subjects learned and performed a forced-choice (concurrent

discrimination) task of gaze-detection, using photographs of a single individual in which theangular deviation of the eyes and the head were independently varied . Monkeys in which therostral STS was removed showed a marked deficit in rc-learning this task accurately, thoughthe specificity of this deficit cannot yet be confidently asserted . Normal subjects performed

SENSITIVITY TO FYI GAZI

1141

the task accurately, though with loss of sensitivity as the angular deviation of the eyesdecreased . This same sensitivity to angular deviation of the eyes was apparent in themonkeys, both pre- and post-operatively .

The task can be performed by human subjects with upside-down faces, though with someloss of sensitivity for small angular deviations .

Two prosopagnosia subjects with similar face-processing impairments showed dissociatedimpairments in gaze discrimination : K .D. was no more impaired than normals viewinginverted faces or RH patients without prosopagnosia [24], but A.B. was unable to do thetask, and used the angular deviation of the head to guide her responses .

.4 ckuowleilqemems-This work was supported by the MRC . U .K . (Grant G 971 . 397'B .) . Dr M . Wyke kindlsprovided clinical details on A .B .

REFERENCESI . ANSTIS, S . M ., MAYHEW, J . W . and MORLEY, T . The perception of where a face or TV portrait is looking . An? . J .

Pst'ehof. 82 .474-489, 1969 .2 . BAYLIS, G . . ROLLS, E . T . R . and LEONARD, C . M . Selectivity between faces in the responses of populations of

neurons in the cortex of the superior temporal sulcus of the monkey . Brain Res . 342, 91-10^_, 1985 .3 . BENTON, A . L . and VAN ALLEN . M. W. Prosopagnosia and facial discrimination . J . .Neurol . Sri . 15, 167 172,

1972 .4 BENTON, A. L . and VAN ALLEN, M . W . Tests of Visual Function, Iowa, 1973 .5 . BENTON . A . L. and VAN ALLEN . M. W . Test ofFace Recognition Manual . Neurosensory Center Publication

number 287 . Department of Neurology, Iowa, USA ., 1973 .6 . BRE YER . R . . LATERRE . C . . SERON, X ., FEYEREISEN, P . . STRYPSTEIN . E ., PIERRARD . E . and RECTEML D . A case of

prosopagnosia with some preserved covert recognition of familiar faces . Brain Cognit . 2, 257 284, 1983 .7. CAMPBELL, R . . LANDIS . T . and REGARD . M . Face recognition and lipreading : a neurological dissociation . Brain

109, 509 521, 1986 .8 . CAMPBELL. R . . GARWOOD . J ., FRANKLIN . S . . HOWARD . D . . LANDIS, T. and REGARD . M . Neuropsvchologtcal

studies of the auditory-visual fusion illusion . Neu,opsrrhologia (in Press) .9 . CHRISTEN . L ., LANDIS, T. and REGARD, M . Left Hemispheric functional compensation in prosopagnosia' A

tachistoscopic study with unilaterally lesioned patients. Hum. Neurohiol . 4, 9-14, 1985 .10. CLINE . M . G . The perception of where a person is looking . Ain . J . Psvc'hol .80, 41 50 . 1967 .11 . DAVIDOFF . J , MArtHEWS. W . B . and NEwcomBE . F . Observations on a case of prosopagnosia- In Aspects of

Face Pro, essioq : o NATO Srvnposnnot, H . D. ELLIS . M . A . JEEVES, F . NEWCOMBE and A . Yotim; IEditorq .pp . 279 290 . Martinus Nilhoff. Dordrecht . 1986 .

12 . Dr RLNzi . E . Prosopagnosia in two patients with CT evidence of damage confined to the right hemisphere .Net, ropsrrhofoqua 24, 385-389 . 1986 .

13 . DE SciioNEN . S . and MATHIVET . E . First come. first served : a scenario about the development of hemisphericspecialization in face recognition during infancy . (ahiers de P,sichologie Cognitive 9, 3 46, 1989 .

14. DFSIM)NE . R . . Aimiun HT . T . D ., (Ross, C. G . and BRLCE . C . Stimulus selective properties of inferior temporalregions in the macaque . J . Neurosri . 8. 2051 2062 . 1984 .

15. EKMAN . P . and FRI LEN, W. Fu, iaf Action Coding ,System : a Technique for the Measurerntnt of FariaLMorrmaul .Palo Alto CA, Consulting Psychologists Press . 1978 .

16. GiBsoN, J . .1 . and PICK, A . D . Perception of another person's looking behavior . Am. J . Psrrhol .70, 386 394,1963 .

17 . HEYWOOD, C. A . and Cowes. A . On the role of cortical area V4 in the discrimination of hue and colour inmacaque monkeys . J. New. osci .7, 2601 2607, 1997 .

18 . HE) wfxID, C . A . . SHIELDS . C . AND Cowes' . A . The involvement of the temporal lobes in colour discrimination .Expl Brain Res . 71, 437-441- 1988 .

19. KLEINKE, C Gaze and eye contact : a research review . Psvchol . Bull . 100, 78 100, 1984 .20. LANDis . T ., CUMMINGS . J . L . . (. HRISTFN . L . . Btx ;EN . J . F . and IMII0F . I-l .-G . Are unilateral right posterior cerebral

lesions sufficient to cause prosopagnosia? Clinical and radiological findings in six additional patients . Cones22,243 252 . 1986 .

21 . LAN'DIS .T . .REGARD.M .,BLUSU.I: .A.andKLP.IHULs,P.Prosopagnosiaandagnosiafornon-canonicalviewsanautopsied case . Brain 111, 1265 1275 . 1988 .

22. Mt Co'Arumt, H . R- Developmental prosopagnosia . A single case report Cortex 12, 76 82, 1976 .

1143

R- CAntfnrLL et a!-

23 . PI:RRFTT . D . 1 . and NIISLIN . A .1 Perception of facial characteristics in monkeys . In Compuratire PinreptIon .W. SLEHINS and M . BERKELEY Editors.John Wiles . Ness York, in press .

1_4 . PERREFT . D . I . . MlsiIIN, A . .1 ('Hit], . .A . J ., HARRIES . M . H_ NEwCUMRL. . FF and DE HA,xN . F . Neuronalmechanisms of face perception and their pathology . In Phcsiofo,giroL9spes I, of Ciintral .Neuro-nphthuLnnln tl . r .C . KENNARD and F . CLIFFORD Ross (Editors) . pp . 136-154 . Chapman & Hall . London . 1988.

2 PERRFrT . D . 1 ., Mist IN . A . J ., POrruR . D. D ., SMIvu, P . A . .1 ., HEAD . A . S . . CHrrn, A . J . . 1JROFNIMANN . R . .MILNER, A . D. and )FINES, M . A . Functional organisation of visual neurons processing face identity . In _(speckofhate Prneessinq . H . ELLIS . M . A . Jssves, F- NEweostHe and A. W . YOUNG (Editors ) . pp . 187 198 . MartimuNijhofL Dordrecht, 1986 .

26 . PERRETI . D . I . . SMITH . P . A . J . . POTTER, D . I) ., MISTLIN . A . J ., HEAD . A . S ., MII NIIR . A . D . and Jr %Is . M . A .Neurons responsive to faces in the temporal cortex : studies offunctional organisation sensitivnv and relation toperception . Hurt Neurohiol . 3. 197-208 . 1984 .

27 . ROLLS . F- T . Neurons in the cortex of the temporal lobe and in the amvgdala of the monkey with respnmzsselective for faces . Hunt . Neurohinl . 3, 209-277 . 1984 .

28- SFRGFNT, J . Structural processing effaces . In llandhook of Research in Pare Proeessing . .A . W . Not No and H .ELLIS . (Editors) . pp . 57-92 . Elsevier . North Holland, 1989-

29- SIILTTLFWORTII . E-C . . SYRING . V . and ALLEN . N- Further observations on the nature of prosopagnosia . Bruin('opnit . 1, 307 322, 1982 .

3(1 . WADE. M. W- and JoNta . R . F . The accuracy of cyegaze judgement : a signal detection approach- Br- .I. Say' .PscchoL 21,293 299 .1982-

31- WARRINGTON, E . K . Recognition Memoir Teat- NFER-Nelson . U .K ., 198432 . WHITELSY, A . M. and WARRINGTON . E . K . Prosopagnosia : a clinical, psychological and anatomical study of

three patients . J . Neurol . Neurosurg . Psehiar . 40, 395 403, 1977,33, WILKINS . A . 1 . and ROBSON, J . G . The Cambridge Low' Contrast Gratings . Clement Clerk International. U.K- .

1986 .34 . YIN, R . looking at upside-down faces . J . exp . Psycho! . 81 . 141 145 . 1969 .