Access and Use of Contextual Expectations in Visual Search during Aging

Miguel Borges (m.borges@student.maastrichtuniversity.nl)1

Moreno I. Coco (micoco@psicologia.ulisboa.pt)University of Lisbon, Department of Psychology,

Alameda da Universidade, 1600-214 Lisboa, Portugal

Abstract

Expectations are routinely utilized by the cognitive system tofacilitate processing and optimize responses. However, as weage, the access and use of expectations might undergo signifi-cant changes. In an eye-tracking study, we investigate howsearch performance in young and old age is influenced bycontextual priming of expectations, and the contextual consis-tency of the target object. We demonstrate that older partici-pants congruently primed perform significantly worse, espe-cially when the target object is inconsistent with the scenecontext (e.g., exposed to a ‘restaurant’ scene before perform-ing the search in another ‘restaurant’ scene, but looking for an‘iron’ rather than ‘bread’). Even though congruency and con-sistency differentially impact on the eye-movement responsesof the two age groups, they do not display any interaction.Overall, this study suggests that reliance on mechanisms ofcontextual expectations becomes stronger when we get older.

Keywords: aging; active vision; visual search; contextualexpectations; priming; eye-movements.

IntroductionPlates and forks are more likely to be found in kitchens thanin bathrooms. Expectations of this sort are constantly gener-ated by our cognitive system to predictively anticipate thestate of events, and pro-actively generate appropriate re-sponses (Bar, 2007; Clark, 2013).

Visual attention is a cognitive modality that relies heavilyon such contextual expectations to maximize performance invisual tasks (Torralba, Oliva, Castelhano, & Henderson,2006). In particular, the allocation of visual attention insearch tasks improves when semantic information of thetarget is combined with the contextual information whereinthe target is embedded (Malcolm & Henderson, 2010).Moreover, this joint use of target and context informationcan occur at early stages of search, as long as the quality andquantity of the contextual information available is adequate(Spotorno, Malcolm, & Tatler, 2014).

However, as we age, the visual system deteriorates andperformance on visual tasks starts to decrease (Potter, Gre-aly, Elliott & Andrés, 2012; Brockmole and Logie, 2013). Inparticular, visual search for a target, manipulated on low-level features (e.g., color, shape and orientation) and embed-ded in an array of distractors (set of increasing sizes) wors-ens with increasing age (Humphrey & Kramer, 1997), eventhough the age-related differences do not manifest to same

extent in short visual-working memory (Brockmole, Parra,Sala, & Logie, 2008).

Most of the above studies on aging focused on low-levelfeatures, with few other studies approaching this topic froma naturalistic perspective, where knowledge-based ratherthan stimulus-based processing is examined (Park, 2012).Thus, it remains an open question whether knowledge-basedexpectancy mechanisms undergo similar changes to thoseobserved on low-level stimulus based information. In thisstudy, we precisely investigate the role of expectations bycomparing younger and older observers in a visual search,contextual priming task situated in naturalistic scenes.

Priming paradigms are used to probe, and activate, se-mantic representations in memory. For example, if a primeword is semantically related to a target word (plant-grass),participants' lexical decision responses are shorter thanwhen the prime word does not belong to the same semanticcategory (Stanovich & West, 1981).

Effects of priming manifest also during visual attention.Mudrik and Koch (2013), for example, show in a congru-ency detection task that contextual priming occurs uncon-sciously, with scenes presented for as little as 33ms. More-over, repeated exposure to the same visual scene can im-prove the speed of search, but only for the same target ob-ject (Vo & Wolfe, 2012). Additionally, as aging impacts onexecutive control and inhibition suppression, search per-formance in seniors would benefit from valid cues, andconversely, take longer to recover from invalid cues (e.g.,Müller-Oehring, et. al. 2013)

The main goal of the current study is to establish howcontextual expectations are utilized by the visual system byyounger (under 25 years) and senior (over 65 years) parti-cipants during a search task. In particular, using a primingparadigm, we probe the congruency of contextual informa-tion between two scenes (the prime scene and the searchscene), sequentially presented (e.g., bathroom-bathroom,versus, kitchen-bathroom), and the consistency of the targetobject with the search scene (e.g., a toothpaste versus afork). In line with research showing that seniors have diffi-culties with inconsistent cues, we expect them to display asignificantly worst search performance when the target ob-ject is inconsistent with the primed information. Their diffi-culties with inhibition suppression would imply an in-creased difficulty to detect a target object when it inconsist-ently deviates from the congruently primed contextual ex-pectations.

1MB is now at Maastricht University, FPN

358

Figure 1: Design. Participants are primed (left column) witheither a congruent (restaurant) or an incongruent scene (pool hall),and then see a search scene (right column) wherein a consistent orinconsistent target is placed. The target objects (consistent = ‘bas-ket of bread’; inconsistent = ‘an iron’) are highlighted in red, andzoomed in.

Our studyIn this eye-tracking study, we compare the behavioural per-formance (accuracy and response time), and attentional re-sponses (search latency, first fixation and total gaze) ofyounger and older participants in a visual search task situ-ated in naturalistic scenes. We investigate how contextualpriming and the semantic consistency of the search targetmediates performance during aging.

Method

ParticipantsFifty-four participants Portuguese native speakers, dividedinto two age groups: 32 younger (28 women), age 22.63(SD = 9.03), University students; and 22 older volunteers(19 women), age 67.9 (SD = 4.75), mostly recruited fromsenior universities, with no medical history ofneuropsychological illness Both groups had normal orcorrected to normal vision. The study was approved by theEthics Committee of the Department of Psychology,University of Lisbon (FPUL).

All participants underwent the Montreal Cognitive As-sessment Test - Portuguese Version (MoCA) (Simões et al.,2008), used as a criteria to detect potential mild cognitivedeficit, especially for the older group. The MoCA was ap-plied on paper with instructions given orally. The young

Figure 2: Trial run. Target object and Prime scene are congruentwith the search scene (restaurant-bread).

group scored an average of 27.1 out of 30 on the MoCA (SD= 1.83), while the elderly group scored an average of 24 outof 30 (SD = 2.81). The groups significantly differ in theirperformance, t(32.8) = 4.46, p < 0.001), however, none ofthe participants in the elderly group performed below thecut-off score of 19.

DesignOur 2x2x2 experimental design crosses object Consistency(Consistent or Inconsistent), contextual Congruency (Con-gruent, Incongruent) between the prime scene and searchscene, and a between-participants age Group condition(younger or older participant). Refer to Figure 1 for an ex-ample of the design material and Figure 2 for the trial run.For example, a participant could be primed with a Restaur-ant scene, then presented with another Restaurant scene, andpreviously asked to search for a basket of bread. This ex-ample corresponds to the Congruent and Consistent experi-mental condition.

Material We utilize 42 experimental scenes, 42 filler scenes and 84prime scenes, which are all naturalistic, real-worldphotographs. In particular, the 42 experimental items weredrawn from 7 different indoor scenarios (Restaurant,Church, Office, Waiting Room, Bathroom, Living Room,and Kitchen) with 6 items per scenario. The scenes werecaptured in Lisbon, and surrounding areas, with a congruentor an incongruent object placed by the photographer (referto section Validity pre-tests of experimental material fordetails on the validation of the consistency between thetarget objects and the search scene). The position of thetarget object was counterbalanced with left-right conditionsto avoid the development of search strategies. We decided totake photographs of the scenes to obtain a naturalistic visualsaliency, and avoid uncontrolled artifacts (e.g., brightness or

359

contrast) that could be introduced when the target object isdigitally inserted into the scene. Material was distributed in4 lists using a Latin-Square Design.

Primes and fillers were selected from various freelyavailable online databases, such as the SUN database (Xiao,Hays, Ehinger, Oliva, & Torralba, 2010) or Flickr. Theprimes were selected on the basis of their Congruency withrespect to the experimental scenes (refer to next section).Moreover, we made sure that the prime scene nevercontained the target object. The same databases were usedfor the filler scenes, and we have chosen scenes belongingto similar contexts. No filler trials did included the targetobject.

Validity pre-tests of experimental materialFirstly, we asked 10 students of the FPUL (6 women)between the ages of 20-25 years (mean = 21.5, SD = 1.433)to name 8 objects they considered typical of each of the 7contexts. 198 unique objects were produced overall, andamong those we selected the 8 most nominated objects foreach context (e.g., pillow for context bedroom), as theconsistent targets. The 8 inconsistent objects, instead, weredrawn from another pool of objects never mentioned by anyof the participants. A new group of 20 participants (12women, mean age = 21.2, SD = 1.23), was then asked tojudge the consistency between the target objects previouslyselected (8 consistent, 8 inconsistent) and the embeddingcontext on a 6-point Likert scale (1 – Totally inconsistent; 6– Totally consistent). We selected the 6 objects that hadhighest (consistent) and lowest (inconsistent) scores foreach context (e.g., Soap – Bathroom: mean = 5.9, SD =0.31; Flashlight-Bathroom: mean = 1.3, SD = 0.67). Finally,a new group of 12 participants (6 women, mean age = 22.5,SD = 1.87) was asked to respond to another Likert scalequestionnaire and rate: (a) the contextual congruencybetween the prime scene and the search scene, (b) thetypicality of the prime scene and search scene given theircontext, (c) the consistency of the target object within thesearch scene, and (d) the ease of recognizing the targetobject given its linguistic denotation.

The purpose of this last questionnaire was to empiricallyvalidate the Congruency and Consistency manipulations.Results show that: (a) the difference between congruousprime scene/search scene (mean = 5.4, SD = 1.09) and theirincongruent version (mean = 1.5, SD = 0.92), was statistic-ally significant (t(489) = 44, p < 0.001); (b) both primescene (mean = 5, SD = 0.87) and search scene (mean = 5 ,SD = 0.86) were considered to be highly representative ex-amples of their context (t(503) = 0.167, p = 0.87); (c) thetarget objects consistent with the search scene (mean = 5.1,SD = 1.6) were rated as significantly more likely than theirinconsistent counterparts (mean = 1.7, SD = 1.2) t(470) =28, p < 0.001); and (d) the target objects were rated ashighly identifiable by their linguistic denotation (mean = 5.3SD = 1.7).

EquipmentVisual stimuli were presented on a LG Flatron L194ws 19-inch LCD screen with a 60 Hz refresh rate, 55cm away fromparticipants' eyes. Eye-movements were recorded using anSMI IVIEW X™ HI-SPEED eye-tracker at a sampling rateof 1250 Hz on a 21” screen (1024 x 768 px. image resolu-tion). Viewing was binocular but only the participant’s dom-inant eye was tracked (determined by a prior parallax test).Chin rest, and forehead support were used to stabilize thehead position, and keep viewing distance constant. Parti-cipants logged their answers regarding the presence (or not)of the target object in the scene, using a Logitech CordlessRumblepad 2 controller. They used 3 buttons for: “Yes”,“No” and to advance to the next trial. The experimental pro-tocol was controlled by ExperimentCenter 3.2 (SMI).

ProcedureParticipants were first presented with the target object (awritten cue for 500ms). Then, a fixation cross appeared inthe center of the screen for 1000ms after which the primescene is presented for 250ms, followed by a mask2 of thesame scene for 250ms. The mask is utilized to disrupt feed-back between lower and higher visual areas and keep theperception of the prime scene on a subliminal level (Enns &Di Lollo, 2000). Finally, the search scene appears and theparticipants can inspect it until they self-terminate the trialby stating whether they saw (or not) the target object.

Each block contained 42 experimental trials and 42 fillertrials, for a total of 84 trials, which were randomly presentedto each participant. The target object was present in the ex-perimental trials, and absent in the filler items guaranteeinga balanced 50-50 distribution of yes/no responses, as well askeeping the participants engaged in the search task. 4 prac-tice trials were administered at the beginning of the experi-mental session to familiarize participants with the task. A15-points calibration procedure was performed at the begin-ning of the trial; and repeated every 10 trials (8-points) toensure accurate tracking. The calibration was accepted whenangle threshold was ≈ 0.8/1.2 for x and y respectively. Onaverage, the younger population had 0.61º ± 0.42º deviationerror on the x-axis. and 0.72º ± 0.58º on the y-axis. Theolder population had 0.82º ± 0.60º deviation error on the x-axis and 1.12º ± 0.79º on the y-axis. The experiment was ex-plained using written instructions and the eye-tracking ses-sion lasted for approximately twenty to thirty minutes.

AnalysesThe performance measures analyzed are: (a) response ac-curacy, a binomial variable for correct and incorrect re-sponses (1, 0), and on correct trials only, (b) the responsetime to pass to the next trial (in ms.). From the eye-move-ment responses to the target object, only for correct trials

2The masks were generated by splitting the prime scene into 12by 9 quadrants, 100x100 pixel squares, and randomly shuffle them.

360

Figure 3: Means and SE for accuracy and reaction time, groupedby Congruency of the prime scene with the target scene(Congruent/Incongruent), Consistency of target object(consistent/inconsistent) and age Group (older/younger). Wedistinguish between the different experimental conditions usingcolour and density of lines.

we compute: (c) the search latency in ms. (i.e., the timefrom scene onset until the target object is looked at for thefirst time), (d) the duration of the first fixation in ms., and(e) the total duration of fixations on target-object in ms.These eye-movement measures are commonly used tocharacterize attentional performance in search tasks(Malcolm & Henderson, 2010). Fixations to the targetobject AOI were mapped using BeGaze (SMI). Statisticaltests on the dependent measures were carried out usingmixed-ANOVAs where the between-participant independentvariables are Congruency between the prime scene and thesearch scene (2 levels: congruent and incongruent), objectConsistency (2 levels: consistent and inconsistent) and thewithin-participant Group variable (2 levels: younger andolder). All analyses are performed using R (ver. 3.0.2, RCore Team, Austria).

Results

AccuracyIn Table 1, we report means and standard deviation for eachdependent measure across all experimental conditions,which are also visualized in Figure 3 and 4. We observe asignificant main effect of Congruency, whereby primescenes congruent with search scenes improve the detectionof the target object [F (1, 1153) = 10.2, p < 0.01]. Younger

Figure 4: Means (SE) of search latency, first fixation and totalduration, grouped by Congruency of the prime scene with thesearch scene (congruent/incongruent), Consistency of target object(consistent/inconsistent) and age Group (older/younger). Wedistinguish between the different experimental conditions usingcolor and density of lines.

participants are overall more accurate than older participants[F (1,1153) = 44.4, p < 0.001]. Moreover, we find a signifi-cant two-ways interaction Consistency:Group, wherebyolder participants detect more accurately inconsistent thanconsistent object [F (1,1153) = 4.22 p < 0.05] Moreover, weobserve a significant three ways interactionCongruency:Consistency:Group, whereby response accu-racy of older participants drastically drops when primescene and search scene are Congruent, but the target objectis Inconsistent with the search scene [F (1,1153) = 14.07,p < 0.001].

Reaction timeScenes where the target object was consistent with thecontext of the search scene took longer to be responded [F(1, 1109) = 17.83, p < 0.0001]. Older participants tooklonger to respond than younger participants [F (1, 1109) =309.82, p < 0.0001]. When looking at the two-waysinteraction, we find that a search scene containing aConsistent target object, which was Congruently primed [F(1, 1109) = 8.98, p < 0.01] yielded faster response timesthan a scene with an Inconsistent target object; especially inolder participants [F (1, 1109) = 6.77, p < 0.01].

361

Table 1: Mean results. Means and standard deviations as a function of the two Prime context conditions and two Object consistency con-ditions for each of the age groups.

Search latencyYoung participants are overall faster than older participantsto look at target object for the first time [F (1, 1109) =95.81, p < 0.0001]. Search latency is faster when the objectis Inconsistent with the search scene, compared to when it isConsistent [F (1, 1109) = 24.06, p < 0.001]. However, whenprime scene and target scene are Congruent, and the targetobject is Consistent, we observe significantly shorter searchlatencies [F (1, 1109) = 4.19, p < 0.05].

First fixationOn the duration of the first fixation, we only find that: (1)consistent target objects are fixated the first time for longerthan inconsistent objects [F (1, 1109) = 4.42, p < 0.05]; and(2) older participants linger on the target object for longerthan younger participants [F (1, 1109) = 5.05, p < 0.05].

Total fixationOn the total fixation duration to the target object, we findmain effects of Consistency, whereby participants fixated anInconsistent target object for longer than when it was Con-sistent [F (1, 1109) = 9.10, p < 0.01]; and of Group, withyounger participants displaying shorter total duration of fix-ation than older participants [F (1, 1109) = 32.96,p < 0.001]. Furthermore, a two-way interaction effectbetween these two variables was also observed, wherebyolder participants fixated the target object overall less whenit was Consistent with the scene [F (1, 1109) = 19.92p < 0.001].

DiscussionBoth age groups are clearly influenced by contextual

priming, object consistency, as well as by their interaction.In particular, correct detections of the target object stronglydepend on the type of priming participants receive.Incongruent primes improve target detection, but this is

more strongly the case when the object is also inconsistent.By probing an unrelated contextual template, participantsmay become more aware of inconsistency within the searchscene. The most interesting result, however, is that thedetection performance of older participants abruptly dropswhen prime and search scenes are congruent, but the targetobject is inconsistent with the search scene. This resultconfirms previous literature showing that older adults havemore difficulties to inhibit invalid cued information (Müller-Oehring, 2013). Another possibly related explanation is thatolder participants might rely more strongly on contextualexpectations than younger participants. So, when primedwith an appropriate context (i.e., restaurant-restaurant), theygenerate strongly structured expectations for consistentinformation (e.g., finding the bread on the table of arestaurant). Thus, when the target object violates suchexpectations, they do not manage to inhibit contextualpriming, and the target escapes their attention. These resultsare largely corroborated by the response times on correctresponses, where again we find faster response times withincongruent primes for inconsistent target objects. Crucially,older participants are much slower to respond withinconsistent objects after being primed with contextuallycongruent information. As just said but from a slightlydifferent angle, this result might indicate that youngparticipants better adapt contextual expectations to theactual search scene, and hence are better able to resolve themismatch generated by the inconsistency of the target objectwith the primed expectations. On eye-movement responses,older participants compared to younger participants areoverall slower locating the target object and spend moretime processing its identity during the first fixation andoverall. Moreover, inconsistent objects are found faster thanconsistent object irrespective of the group, in line withprevious literature manipulating this experimental factor(Bonitz & Gordon, 2008). Both groups improve their searchlatency if the priming is congruent, and the object is

362

consistent. This indicates that appropriate contextualpriming can improve performance, but only if the object tobe looked at is consistent with the primed information.Crucially, however, we did not find a significant three-waysinteraction between Congruency, Consistency and Group;which might indicate that aging does not really mediate thejoint influence of Consistency and Congruency onattentional responses.

ConclusionExpectations are routinely used by the cognitive system to

anticipate information about real-word events, and optimizeappropriate action-responses (Bar, 2007; Clark, 2013).Visual attention is one of the components that can benefitfrom contextual expectations in tasks such as visual search,(Malcolm & Henderson, 2010; Spotorno et. al. 2014).Aging, however, might impair attentional performance, andconsequently change the way contextual expectations areaccessed, and subsequently used, to allocate attention onscenes. For example, visual performance in search tasks,manipulating low-level features, appears to degrade withincreasing age (Humphrey & Kramer, 1997). An openchallenge was to investigate whether knowledge-basedexpectations undergo similar degradation, and especially,how attentional mechanisms would be differently used.

In the current study, we used a contextual primingparadigm to probe expectations, and compared visualperformance in a search task between younger and olderobservers. We demonstrated that older observers rely morestrongly on consolidated expectations than youngerobservers. In particular, when the prime and search sceneoverlapped in context, older participants had a worseperformance than the younger, especially if the target objectwas inconsistent with the primed expectations. This resultsuggests that older observers might have more difficultiesre-computing the semantics of individual objects whenprimed, and hence compensate for unexpected information.Nevertheless, eye-movement responses did not show areliable difference in this interaction between congruencyand consistency due to aging. Both groups experiencedbetter search performance when the target object wasconsistent with expectations, but no difference was observedin this interaction due to age.

This study opens intriguing questions on the role ofprimed contextual expectations in clinical populationsaffected by neuro-degenerative disease (e.g., Alzheimer);and especially on how would behavioural patterns comparewith a normally aging population.

Overall, this work confirms the important role thatcontextual information plays on the active allocation ofvisual attention and shows that reliance on it may becomestronger as we age.

AcknowledgmentsFundação Amadeu Dias for student award to MB, andFundação para a Ciência e Tecnologia research fellowship

award to MIC (SFRH/BDP/88374/2012), are gratefullyacknowledged.

ReferencesBar, M., 2007. The proactive brain: using analogies and

associations to generate predictions. Trends in cognitivesciences 11 (7), 280–289.

Bonitz, V. S., & Gordon, R. D. (2008). Attention to smokingrelated and incongruous objects during scene viewing.Acta Psychologica, 129, 255–263.

Brockmole, J., Parra, M., Sala, S., Logie, R. (2008). Dobinding deficits account for age-related decline in visualworking memory? Psychonomic Bulletin & Review,15(3), 543-547.

Brockmole, J. R., & Logie, R. H. (2013). Age-relatedchange in visual working memory: a study of 55,753 par-ticipants aged 8–75. Frontiers in Psychology, 4.

Clark, A., 2013. Whatever next? predictive brains, situatedagents, and the future of cognitive science. Behavioraland Brain Sciences 36 (03), 310 181–204.

Enns, J. & Di Lollo, V. (2000). What's new in visualmasking? Trends in Cognitive Sciences, 4(9), 345-352.

Humphrey, D. G., & Kramer, A. F. (1997). Age differencesin visual search for feature,conjunction, and triple-con-junction targets. Psychology and Aging, 12, 704–717.

Malcolm, G. L., & Henderson, J. M. (2010). Combiningtop-down processes to guide eye movements during real-world scene search. Journal of Vision, 10(2), 4.

Mudrik, L., & Koch, C. (2013). Differential processing ofinvisible congruent and incongruent scenes: A case for un-conscious integration. Journal of Vision, 13(13), 24.

Müller-Oehring, E. M., Schulte, T., Rohlfing, T., Pfeffer-baum, A., & Sullivan, E. V. (2013). Visual search and theaging brain: discerning the effects of age-related brainvolume shrinkage on alertness, feature binding, and atten-tional control. Neuropsychology, 27(1), 48.

Park, Denise, ed. Cognitive aging: A primer. PsychologyPress, 2012.

Potter, L. M., Grealy, M. A., Elliott, M. A., & Andrés, P.(2012). Aging and performance on an everyday-based vis-ual search task. Acta Psychologica, 140(3), 208-217.

Spotorno, S., Malcolm, G. L., & Tatler, B. W. (2014). Howcontext information and target information guide the eyesfrom the first epoch of search in real-world scenes.Journal of Vision, 14(2), 7.

Stanovich, K. E., & West, R. F. (1983). On priming by asentence context. Journal of Experimental Psychology:General, 112(1), 1.

Torralba, A., Oliva, A., Castelhano, M., & Henderson, J.(2006). Contextual guidance of eye movements and atten-tion in real-world scenes:The role of global features in ob-ject search. Psychological review, 4(113), 766–786.

Võ, M. L. H., & Wolfe, J. M. (2012). When does repeatedsearch in scenes involve memory? Looking at versuslooking for objects in scenes. Journal of ExperimentalPsychology: Human Perception and Performance, 38(1),23.

363