Neural Similarity Between Encoding and Retrieval is Related to Memory Via HippocampalInteractions

Maureen Ritchey, Erik A. Wing, Kevin S. LaBar and Roberto Cabeza

Center for Cognitive Neuroscience, Department of Psychology and Neuroscience, Duke University, Durham, NC 27708, USA

Address correspondence to Maureen Ritchey, Center for Neuroscience, 1544 Newton Ct., Davis, CA 95618, USA. Email: meritchey@ucdavis.edu,mritchey@gmail.com

A fundamental principle in memory research is that memory is afunction of the similarity between encoding and retrieval oper-ations. Consistent with this principle, many neurobiological modelsof declarative memory assume that memory traces are stored incortical regions, and the hippocampus facilitates the reactivation ofthese traces during retrieval. The present investigation tested thenovel prediction that encoding–retrieval similarity can be observedand related to memory at the level of individual items. Multivariaterepresentational similarity analysis was applied to functional mag-netic resonance imaging data collected during encoding and retrie-val of emotional and neutral scenes. Memory success trackedfluctuations in encoding–retrieval similarity across frontal and pos-terior cortices. Importantly, memory effects in posterior regions re-flected increased similarity between item-specific representationsduring successful recognition. Mediation analyses revealed thatthe hippocampus mediated the link between cortical similarity andmemory success, providing crucial evidence for hippocampal–cortical interactions during retrieval. Finally, because emotionalarousal is known to modulate both perceptual and memory pro-cesses, similarity effects were compared for emotional and neutralscenes. Emotional arousal was associated with enhanced similaritybetween encoding and retrieval patterns. These findings speak tothe promise of pattern similarity measures for evaluating memoryrepresentations and hippocampal–cortical interactions.

Keywords: emotional memory, episodic memory, functional neuroimaging,multivariate pattern analysis

Introduction

Memory retrieval involves the reactivation of neural statessimilar to those experienced during initial encoding. Thestrength of these memories is thought to vary as a function ofencoding–retrieval match (Tulving and Thomson 1973), withstronger memories being associated with greater correspon-dence. This relationship has been formalized in modelslinking recognition memory success to the quantitative simi-larity of event features sampled during encoding and retrieval(Bower 1972), as well as in the principle of transfer-appropriate processing, which proposes that memory will beenhanced when the cognitive operations engaged during en-coding are related to those supporting memory discriminationat retrieval (Morris et al. 1977). Encoding–retrieval similarityalso bears special relevance to neurocomputational modelsthat posit a role for the hippocampus in guiding a replay ofprior learning events across the neocortex (Alvarez and Squire1994; McClelland et al. 1995; Nadel et al. 2000; Sutherlandand McNaughton 2000; Norman and O’Reilly 2003). In thesemodels, the hippocampus binds cortical representations

associated with the initial learning experience, and then facili-tates their reactivation during the time of retrieval.

Prior attempts to measure encoding–retrieval match havefocused on identifying differences in hemodynamic responsesto retrieval sets that vary only in their encoding history, suchas prior association with a task, context, or stimulus (reviewedby Danker and Anderson 2010). For example, when retrievalcues are held constant, words studied with visual imageselicit greater activity in the visual cortex at retrieval thanwords studied with sounds (Nyberg et al. 2000; Wheeler et al.2000), consistent with the reactivation of their associates.However, memory often involves discriminating between oldand new stimuli of the same kind. Within the context of rec-ognition memory, transfer-appropriate processing will bemost useful when it involves the recapitulation of cognitiveand perceptual processes that are uniquely linked to individ-ual items and differentiate them from lures (Morris et al.1977). Because previous studies have compared sets of itemswith a shared encoding history, their measures of processingoverlap have overlooked those operations that are idiosyn-cratic to each individual stimulus. The current experimentmakes the important advance of measuring the neural simi-larity between encoding and recognition of individual scenes(e.g. an image of a mountain lake vs. other scene images). Bylinking similarity to memory performance, we aim to identifyregions in which neural pattern similarity to encoding isassociated with retrieval success, likely arising from their par-ticipation in operations whose recapitulation benefitsmemory. These operations may be limited to perceptual pro-cesses supporting scene and object recognition, evident alongoccipitotemporal pathways, or may include higher-order pro-cesses reliant on frontal and parietal cortices.

Item-specific estimates also facilitate new opportunities forlinking encoding–retrieval overlaps to the function of the hip-pocampus. One possibility is that the hippocampus mediatesthe link between neural similarity and behavioral expressionsof memory. The relation between the hippocampus and corti-cal reactivation is central to neurocomputational models ofmemory, which predict enhanced hippocampal–neocorticalcoupling during successful memory retrieval (Sutherland andMcNaughton 2000; Wiltgen et al. 2004; O’Neill et al. 2010).Although these ideas have been supported by neurophysiolo-gical data (Pennartz et al. 2004; Ji and Wilson 2007), it hasbeen a challenge to test this hypothesis in humans. To thisend, the present approach newly enables the analysis of howthe hippocampus mediates the relationship between encod-ing–retrieval pattern similarity and memory retrieval on atrial-to-trial basis.

Finally, the neural similarity between encoding and retrie-val should be sensitive to experimental manipulations thatmodulate perception and hippocampal memory function.

© The Author 2012. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com

Cerebral Cortexdoi:10.1093/cercor/bhs258

Cerebral Cortex Advance Access published September 11, 2012 at M

edical Center L

ibrary, Duke U

niversity on April 2, 2013

http://cercor.oxfordjournals.org/D

ownloaded from

There is extensive evidence that emotional arousal increasesthe strength and vividness of declarative memories, thoughtto arise from its influence on encoding and consolidation pro-cesses (LaBar and Cabeza 2006; Kensinger 2009). Althoughthe neural effects of emotion on encoding (Murty et al. 2010)and retrieval (Buchanan 2007) phases have been separatelycharacterized, they have seldom been directly compared (butsee Hofstetter et al. 2012). Emotional stimuli elicit superiorperceptual processing during encoding (Dolan and Vuilleu-mier 2003), which may result in perceptually rich memorytraces that can be effectively recaptured during item recog-nition. Furthermore, arousal-related noradrenergic and gluco-corticoid responses modulate memory consolidationprocesses (McGaugh 2004; LaBar and Cabeza 2006), whichare thought to rely on hippocampal–cortical interactionssimilar to those supporting memory reactivation during retrie-val (Sutherland and McNaughton 2000; Dupret et al. 2010;O’Neill et al. 2010; Carr et al. 2011). One possibility is that theinfluence of emotion on consolidation is paralleled bychanges in encoding–retrieval similarity during retrieval. Wetest the novel hypothesis that emotion, through its influenceon perceptual encoding and/or hippocampal–cortical inter-actions, may be associated with increased pattern similarityduring retrieval.

Here, we use event-related functional magnetic resonanceimaging (fMRI), in combination with multivariate pattern simi-larity analysis, to evaluate the neural similarity between indi-vidual scenes at encoding and retrieval. Across several regionsof interest (ROIs), we calculated the neural pattern similaritybetween encoding and retrieval trials, matching individualtrials to their identical counterparts (“item-level pairs”) or toother stimuli drawn from the same emotional valence, encod-ing condition, and memory status (“set-level pairs”; Fig. 1). Toassess evidence for item-specific fluctuations in neural simi-larity, we specifically sought regions showing a differencebetween remembered and forgotten items that were augmen-ted for item-level pairs relative to set-level pairs. Given the rec-ognition design employed for retrieval, we anticipated thatevidence for item-specific similarity would be particularly pro-nounced in regions typically associated with visual processing.These techniques yielded estimates of the neural pattern simi-larity for each individual trial, newly enabling a direct test ofthe hypothesis that the hippocampus mediates the relation-ship between cortical similarity and behavioral expressions ofmemory. Finally, we tested the novel prediction that emotionenhances cortical similarity by comparing these effects foremotionally arousing versus neutral scenes.

Materials and Methods

ParticipantsTwenty-one participants completed the experiment. Two par-ticipants were excluded from analysis: One due to excessivemotion and one due to image artifacts affecting the retrievalscans. This resulted in 19 participants (9 female), ranging inage from 18 to 29 (M = 23.3, standard deviation [SD] = 3.1).Participants were healthy, right-handed, native English speak-ers, with no disclosed history of neurological or psychiatricepisodes. Participants gave written informed consent for aprotocol approved by the Duke University InstitutionalReview Board.

Experimental DesignParticipants were scanned during separate memory encodingand recognition sessions, set 2 days apart. During the firstsession, they viewed 420 complex visual scenes for 2 s each.Following each scene, they made an emotional arousalrating on a 4-point scale and answered a question related tothe semantic or perceptual features of the image. Duringthe second session, participants saw all of the old scenesrandomly intermixed with 210 new scenes for 3 s each. Foreach trial, they rated whether or not the image was old ornew on a 5-point scale, with response options for “definitelynew,” “probably new,” “probably old,” “definitely old,” and“recollected.” The fifth response rating referred to those in-stances in which they were able to recall a specific detailfrom when they had seen that image before. For all ana-lyses, memory success was assessed by collapsing the fourthand fifth responses, defining those items as “remembered,”and comparing them with the other responses, referred tohere as “forgotten.” This division ensured that sufficientnumbers of trials (i.e. >15) were included as rememberedand forgotten. In both sessions, trials were separated by ajittered fixation interval, exponentially distributed with amean of 2 s.

Figure 1. An overview of the experimental and analysis design. Participants viewedemotionally negative, positive, and neutral images during scene encoding andrecognition memory tasks; each image was classified as remembered or forgotten.Beta estimates were computed for each individual trial at both encoding andrecognition (A). For the full complement of encoding–retrieval pairs, includingitem-level pairs (same picture) and set-level pairs (same valence, task and memorystatus, different pictures), beta patterns were extracted from 31 separate anatomicalROIs and the similarity between patterns was computed (B). E, encoding; R, retrieval;rem, remembered; forg, forgotten; neg, negative; neut, neutral; pos, positive.

2 Encoding–Retrieval Similarity and Memory • Ritchey et al.

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

The stimuli consisted of a heterogeneous set of complexvisual scenes drawn from the International Affective PictureSystem (Lang et al. 2001) as well as in-house sources. Theyincluded 210 emotionally negative images (low in valenceand high in arousal, based on normative ratings), 210emotionally positive images (high in valence and high inarousal), and 210 neutral images (midlevel in valence and lowin arousal). The influence of arousal was assessed by compar-ing both negative and positive images to neutral. Additionaldetails about the encoding design and the stimulus set are de-scribed in Ritchey et al. (2011), which report subsequentmemory analyses of the encoding data alone.

fMRI Acquisition and Pre-ProcessingImages were collected using a 4T GE scanner, with separatesessions for encoding and retrieval. Stimuli were presentedusing liquid crystal display goggles, and behavioral responseswere recorded using a 4-button fiber optic response box.Scanner noise was reduced with earplugs and head motionwas minimized using foam pads and a headband. Anatomicalscanning started with a T2-weighted sagittal localizer series.The anterior (AC) and posterior commissures (PC) were ident-ified in the midsagittal slice, and 34 contiguous oblique sliceswere prescribed parallel to the AC–PC plane. Functional imageswere acquired using an inverse spiral sequence with a 2-s rep-etition time, a 31-m echo time, a 24-cm field of view, a 642

matrix, and a 60° flip angle. Slice thickness was 3.8 mm, result-ing in 3.75 × 3.75 × 3.8 mm voxels.

Preprocessing and data analyses were performed usingSPM5 software implemented in Matlab (www.fil.ion.ucl.ac.uk/spm). After discarding the first 6 volumes, the functionalimages were slice-timing corrected, motion corrected, andspatially normalized to the Montreal Neurological Institutetemplate. Functional data from the retrieval session werealigned with data from the encoding session, and normaliza-tion parameters were derived from the first functional fromthe encoding session. Data were spatially smoothed with an8-mm isotropic Gaussian kernel for univariate analyses andleft unsmoothed for multivariate pattern analyses.

fMRI Analysis

General Linear ModelAll multivariate pattern analyses were based on a generallinear model that estimated individual trials. General linearmodels with regressors for each individual trial were esti-mated separately for encoding and retrieval, yielding onemodel with a beta image corresponding to encoding trial andanother model with a beta image corresponding to each re-trieval trial. The postimage encoding ratings were alsomodeled and combined into a single regressor; thus, all ana-lyses reflect the picture presentation time only. Regressors in-dexing head motion were also included in the model. Thevalidity of modeling individual trials has been previously de-monstrated (Rissman et al. 2004), and the comparability ofthese models to more traditional methods was likewise con-firmed within this dataset.

Additional general linear models were estimated for uni-variate analysis of the retrieval phase, which were used todefine functional ROIs for the mediation analyses (describedbelow). The models were similar to that described above, butcollapsed the individual trials into separate regressors for

each emotion type, as in standard analysis approaches. Thesemodels also included parametric regressors indexing the5-point subsequent memory response for old items, as well asa single regressor for all new items. Contrasts correspondingto the effects of memory success, emotion, and the emotionby memory interaction were generated for each individual.Univariate analysis procedures and results are described ingreater detail in Supplementary Materials.

Multivariate Encoding–Retrieval Similarity AnalysisA series of 31 bilateral anatomical ROIs were generated fromthe Anatomical Automatic Labeling (AAL) system (Tzourio-Mazoyer et al. 2002) implemented in WFU Pickatlas (Maldjianet al. 2003). We expected that encoding–retrieval similarityeffects could arise not only from the reactivation of perceptual

Table 1All regions tested with ANOVA results for the effects of match and memory

Region Voxels Meanr

Main effect ofmatch

Main effect ofmemory

Match ×memoryinteraction

F1,18 P F1,18 P F1,18 P

OccipitalCalcarine 635 0.23 176.09 0.000* 1.99 0.088 8.74 0.004Cuneus 436 0.12 52.46 0.000* 3.05 0.049 2.62 0.062Lingual 609 0.17 222.77 0.000* 3.86 0.032 12.18 0.001*Inf occipital 283 0.19 55.93 0.000* 11.69 0.002* 6.56 0.010Mid occipital 780 0.18 131.33 0.000* 16.95 0.000* 15.29 0.001*Sup occipital 428 0.16 124.92 0.000* 21.78 0.000* 4.55 0.023

TemporalFusiform 699 0.21 89.36 0.000* 1.83 0.002 0.46 0.254Inf temporal 1019 0.05 119.06 0.000* 24.66 0.000* 0.01 0.468Mid temporal 1396 0.06 38.36 0.000* 26.92 0.000* 16.64 0.000*Mid temporalpole

287 0.01 0.26 0.309 2.15 0.080 0.02 0.448

Sup temporalpole

407 0.01 0.69 0.209 0.55 0.234 0.00 0.493

Sup temporal 803 0.02 0.22 0.323 14.12 0.001* 2.87 0.054Frontal

Inf frontalopercularis

382 10 39.50 0.000* 27.64 0.000* 2.61 0.062

Inf frontalorbitalis

526 0.03 1.26 0.138 7.30 0.007 0.00 0.499

Inf frontaltriangularis

691 0.08 26.38 0.000* 45.36 0.000* 0.47 0.251

Mid frontal 1472 0.04 32.05 0.000* 18.42 0.000* 2.47 0.067Mid frontalorbital

267 0.01 0.43 0.260 0.15 0.352 2.79 0.056

Sup frontal 1204 0.02 31.29 0.000* 2.26 0.075 0.15 0.353Sup medialfrontal

758 0.02 17.61 0.000* 4.76 0.021 0.63 0.218

Sup frontalorbital

294 0.01 6.04 0.012 1.40 0.126 0.26 0.308

Insula 549 0.02 0.16 0.348 3.74 0.034 0.04 0.418Parietal

Angular 409 0.06 23.63 0.000* 9.68 0.003 6.91 0.009Inf parietal 565 0.06 42.86 0.000* 28.00 0.000* 0.11 0.371Sup parietal 651 0.08 1.19 0.145 1.23 0.141 0.09 0.382Precuneus 1014 0.05 50.28 0.000* 22.54 0.000* 8.90 0.004Supramarginal 434 0.03 17.80 0.001* 49.54 0.000* 34.24 0.000*

MTLAmygdala 74 0.01 1.59 0.111 0.59 0.227 0.44 0.258Hippocampus 273 0.02 0.00 0.473 2.35 0.072 0.02 0.441

Parahippocampalgyrus

304 0.04 21.79 0.000* 3.72 0.000* 7.00 0.008

Primary sensory controlsHeschl 63 0.01 1.33 0.132 2.33 0.072 1.60 0.111Olfactory 68 0.01 0.01 0.471 0.17 0.342 0.01 0.472

Note: Mean r refers to the Pearson’s correlation coefficient between encoding and retrievalpatterns for remembered items, averaged across the group. The main effects and interactionsinvolving emotion are presented separately in Table 4.*Denote regions significant at Bonferroni-corrected threshold.MTL, medial temporal lobe; Inf, inferior; Mid, middle; Sup, superior.

Cerebral Cortex 3

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

processes, likely to be observed in occipital and temporal cor-tices, but also higher-order cognitive processes involving thelateral frontal and parietal cortices. Thus, the ROIs includedall regions comprised in the occipital, temporal, lateral pre-frontal, and parietal cortices (Table 1). Regions within themedial temporal lobe were also included due to their knownroles in memory and emotional processing. Two of theseregions (olfactory cortex and Heschl’s gyrus) were chosen asprimary sensory regions that are unlikely to be responsive toexperimental factors, and thus they serve as conceptual con-trols. Bilateral ROIs were chosen to limit the number of com-parisons while observing effects across a broad set of regions,and reported effects were corrected for multiple comparisons.

The full pattern of voxel data from a given ROI wasextracted from each beta image corresponding to each trial,yielding a vector for each trial Ei at encoding and each trial Rjat retrieval (Fig. 1). Each of the vectors was normalized withrespect to the mean activity and SD within each trial, thus iso-lating the relative pattern of activity for each trial and ROI.Euclidean distance was computed for all possible pair-wisecombinations of encoding and retrieval target trials, resultingin a single distance value for each EiRj pair. This distance valuemeasures the degree of dissimilarity between activity patternsat encoding and those at retrieval—literally the numerical dis-tance between the patterns plotted in n-dimensional space,where n refers to the number of included voxels (Kriegeskorte2008). An alternative metric for pattern similarity is the Pear-son’s correlation coefficient, which is highly anti-correlatedwith Euclidean distance; not surprisingly, analyses using thismeasure replicate the findings reported here. For ease ofunderstanding, figures report the inverse of Euclidean distanceas a similarity metric; regression analyses likewise flip the signof distance metric z-scores to reflect similarity.

Pair-wise distances were summarized according to whetheror not the encoding–retrieval pair corresponded to an item-level match between encoding and retrieval (i.e. trial Ei refersto the same picture as trial Rj). In comparison, set-level pairswere matched on the basis of emotion type, encoding task,and memory status, but excluded these item-level pairs. Theseexperimental factors were controlled in the set-level pairs tomitigate the influence of task engagement or success. Anydifferences observed between the item- and set-level pair dis-tances can be attributed to the reactivation of memories orprocesses specific to the individual stimulus. On the otherhand, any memory effects observed across both the item- andset-level pair distances may reflect processes general to en-coding and recognizing scene stimuli.

To assess neural similarity, mean distances for the item- andset-level match pairs were entered into a 3-way repeated-measures analysis of variance (ANOVA) with match level (itemlevel, set level), memory status (remembered, forgotten), andemotion (negative, neutral, positive) as factors. We first con-sidered the effects of match and memory irrespective ofemotion. Reported results were significant at a 1-tailed family-wise error rate of P < 0.05, with a Bonferroni correction foreach of the 31 tested ROIs (effective P < 0.0016). For regionsshowing a significant match by memory interaction, the spatialspecificity of these findings was interrogated further by bisect-ing the anatomical ROIs along the horizontal and coronalplanes, separately for each hemisphere. The resulting 8 subre-gions per ROI were analyzed identically to the methodsdescribed above and corrected for multiple comparisons.

To evaluate whether item-level similarity varied by whetherparticipants gave a recollected or definitely old response,mean distances for the item-level pairs were entered into a2-way repeated-measures ANOVA with memory response(recollected, definitely old) and emotion type (negative,neutral, positive) as factors. Finally, although the patternvectors were standardized prior to distance computation, it re-mained a concern that overall activation in the ROIs could bedriving some of these memory-related distance effects. Torule out this possibility, we conducted a logistic regressionusing trial-to-trial measures of mean ROI activation during en-coding, mean ROI activation during retrieval, and patternsimilarity to predict binary memory outcomes, within eachparticipant and for each of the memory-sensitive regions. Thebeta coefficients were tested with 1-sample t-tests to deter-mine whether they significantly differed from zero across thegroup. A significant result would indicate that mean activationestimates and encoding–retrieval similarity make separablecontributions to memory. All subsequent memory-related ana-lyses were restricted to regions for which encoding–retrievalsimilarity remained a significant predictor of memory in thisregression. Both sets of tests were corrected for multiple com-parisons across 31 ROIs, P < 0.0016.

Functional ConnectivityAnother goal was to test the hypothesis that the link betweencortical pattern similarity and memory is mediated by hippo-campal activity at retrieval. To address this hypothesis, multi-level mediation analysis was conducted via the MultilevelMediation and Moderation toolbox (Wager et al. 2009; Atlaset al. 2010), using encoding–retrieval similarity (from theitem-level pairs) as the independent variable, 5-point memoryresponses as the dependent variable, and hippocampalactivity at retrieval as the mediating variable. Each of thesetrial-wise vectors was normalized within participants prior toanalysis, and trials identified as outliers (>3 SDs from themean) were excluded. Hippocampal functional ROIs includedvoxels within the AAL anatomical ROI that showed parametricmodulation by memory strength for old items, defined separ-ately for each individual at the liberal threshold of P < 0.05 toensure the inclusion of all memory-sensitive voxels. Voxelswere limited to left hippocampus, where stronger univariateretrieval success effects were observed. Two participants wereexcluded from this analysis for not having above-thresholdresponses in the left hippocampus. Models were estimated viabootstrapping for each participant and then participants weretreated as random effects in a group model that tested the sig-nificance of each path. Significant mediation was identified bythe interaction of path a (similarity to hippocampal activity)and path b (hippocampal activity to memory, controlling forsimilarity). The mediation model was specified separately foreach of the 8 neocortical ROIs showing significant match andmemory effects, and statistical tests were thresholded atBonferroni-corrected P < 0.05, 1-tailed (effective P < 0.0063).It should be noted that it is not feasible to ascertain neuraldirectionality at the level of trial-wise estimates; thus, for thesake of completeness, the reverse mediation was also com-puted, with hippocampal activity as the independent variableand encoding–retrieval similarity as the mediating variable.

Finally, for ROIs showing evidence of hippocampalmediation, the correlation of pattern distance and hippocam-pal activity during retrieval was computed separately for

4 Encoding–Retrieval Similarity and Memory • Ritchey et al.

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

remembered and forgotten items, excluding trials identifiedas outliers. This enabled the direct test of whether patternsimilarity covaried with hippocampal activity within success-fully remembered or forgotten items only, and whether thiscorrelation was stronger for remembered than forgottenitems, using 1-sample and paired t-tests, respectively, onz-transformed r-values. This technique is similar to functionalconnectivity analyses based on beta series (Rissman et al.2004; Daselaar et al. 2006), but relates activity to pattern dis-tance rather than activity to activity.

Emotion EffectsFor regions showing significant modulation by match level,the influence of emotion was assessed in the 3-way repeated-measures ANOVA with match level (item level, set level),memory status (remembered, forgotten), and emotion (nega-tive, neutral, positive) as factors. Reported results were signifi-cant at a 1-tailed family-wise error rate of P < 0.05, with aBonferroni correction for each of the 19 tested ROIs (effectiveP < 0.0026). The region showing the maximal main effect ofemotion and match by emotion interaction (middle occipitalgyrus) was interrogated further using functional connectivitymethods similar to those described above to assess itsrelationship with amygdala activity during retrieval. Amygdalafunctional ROIs included voxels within the AAL anatomicalROI (limited to the right amygdala, where stronger univariateemotion effects were observed) that showed modulation byemotion or the interaction of emotion and memory for olditems, defined separately for each individual at the liberalthreshold of P < 0.05. Both the mediation tests and memory-modulated correlation tests were implemented similarly tothose described above, but with amygdala filling in the role ofthe hippocampus.

Results

Behavioral DataOverall memory performance was very good: Old items wereremembered 53.7% (SD = 19.2) of the time, whereas newitems were incorrectly endorsed as such only 3.3% (SD = 2.4)of the time, on average (Supplementary Table 2). These ratesresulted in an average d’ score of 2.04 (SD = 0.48), which iswell above chance, t(18) = 18.40, P < 0.001. Memory perform-ance was significantly modulated by the emotional content ofthe images, as measured by d’ (mean ± SD: negative, 2.33 ±0.52, positive, 1.91 ± 0.52, neutral, 1.92 ± 0.52). Negative pic-tures were recognized with greater accuracy than bothneutral, t(18) = 4.18, P = 0.001, and positive, t(18) = 5.69, P <0.001, pictures.

Evidence for Item-Level Encoding–Retrieval SimilarityPattern similarity between encoding and retrieval wasmeasured by calculating the pair-wise distance between eachretrieval trial and each encoding trial. Distances were sortedwith respect to whether the paired trials corresponded to thesame pictures (item-level pairs) or different pictures of thesame type (set-level pairs). As expected, many of the 31tested ROIs showed a main effect of match (Fig. 2A,B), exhi-biting greater encoding–retrieval similarity for item-level pairsthan for set-level pairs, P < 0.0016 (Table 1). Because thesame images were presented at encoding and recognition,this finding demonstrates that these pattern similaritymeasures are sensitive to the visual and conceptual differ-ences that distinguish individual images, consistent with priorevidence that individual scenes can be decoded using voxelinformation from the visual cortex (Kay et al. 2008). Not sur-prisingly, item-specific encoding–retrieval similarity is highestin regions devoted to visual perception; for example, the

Figure 2. Effects of item match and memory on encoding–retrieval pattern similarity. Among regions displaying greater encoding–retrieval similarity for item- than set-levelpairs, the correlation between encoding and retrieval patterns was strongest in regions devoted to visual processing (A and B). Color-coding reflects the strength of thiscorrelation, averaged across remembered trials. For example, the calcarine ROI showed a main effect of match, in that item-level pairs were more similar than set-level pairs,regardless of memory (C). Many regions were sensitive to the main effect of memory (D), in which greater encoding–retrieval similarity was associated with better memory,regardless of whether it was an item-level or set-level match. This main effect is represented in the inferior frontal gyrus opercularis ROI (E). Error bars denote standard error ofthe mean. Rem, remembered; forg, forgotten; d, distance; inf, inferior; mid, middle; sup, superior.

Cerebral Cortex 5

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

mean correlation between encoding and retrieval patterns incalcarine ROI was 0.23 (subject range: 0.10–0.34) for remem-bered items (Fig. 2C).

Several regions, including inferior frontal gyrus, precuneus,inferior parietal, occipital gyri, and inferior and middletemporal gyri, were characterized by a main effect of memorysuccess such that encoding–retrieval similarity was enhancedduring successful recognition regardless of match level, P <0.0016 (Fig. 2D,E). This similarity may reflect the re-engagement of processes that are generally beneficial to bothscene encoding and retrieval (e.g. heightened attention), com-patible with meta-analytic data marking frontal and parietalregions as consistent predictors of memory success at bothphases (Spaniol et al. 2009).

In the critical test of memory-related pattern similarity, sig-nificant interactions between match and memory status wereidentified in middle occipital gyrus, middle temporal gyrus,and supramarginal gyrus, P < 0.0016 (Table 1). Marginaleffects were also noted in precuneus (P = 0.004). In each ofthese regions, memory success was associated with greaterencoding–retrieval similarity, particularly when the retrievaltrial was compared with its identical encoding trial. This inter-action supports the idea that similar item-specific processesare engaged during both encoding and recognition, and thatthis processing overlap is affiliated with successful memorydiscrimination. To further interrogate spatial localization, eachleft or right ROI showing a significant interaction was splitinto quadrants. Analysis of these subregions revealed thatinteraction effects were concentrated within the left anterior-dorsal subregion of middle occipital gyrus and the bilateralposterior-dorsal subregions of middle temporal gyrus(Fig. 3A,B; Supplementary Table 3). These findings are con-sistent with our prediction that processes supported by occi-pitotemporal regions, such as visual scene or objectprocessing, are engaged during both encoding and retrieval,and that the memory benefits of this overlap can be observedat the level of the individual item. There were no significantincreases in neural similarity for trials accurately endorsedwith the recollected versus definitely old response, althoughthe middle occipital gyrus and hippocampus exhibited trendsin that direction (Supplemental Materials).

Across-trial logistic regression analysis confirmed that, fornearly all regions identified as memory sensitive, encoding–

retrieval pattern similarity significantly predicted memory per-formance even when mean retrieval activation estimates (forboth encoding and retrieval) were included in the model(Fig. 3C; Table 2). These results also clarify that, although wedid not observe pattern similarity effects in the hippocampus,mean retrieval activity in this region predicted memorysuccess; the amygdala showed a similar trend (P = 0.002).Thus, we can be sure that these regions are sensitive tomemory in the present design; however, standard spatial res-olution parameters may be too coarse to detect consistent pat-terns in these structures (cf. Bonnici et al. 2011). Thedissociability of these effects suggests that similarity betweentrial-specific patterns during encoding and retrieval is associ-ated with enhanced memory in a manner that goes beyondthe magnitude of hemodynamic response.

Role of the Hippocampus in Mediating CorticalSimilarityIt was hypothesized that if encoding–retrieval pattern simi-larity reflects the reactivation of the memory trace, then itsrelationship with memory performance should be mediatedby hippocampal activity at retrieval (Fig. 4A). Indeed, hippo-campal activity was a significant statistical mediator of therelationship between encoding–retrieval similarity andmemory, P < 0.0063, for occipital, inferior frontal, and inferiorparietal cortices (Fig. 4B; Table 3). After accounting for theinfluence of the hippocampus, the relationship between en-coding–retrieval similarity and memory remained intact foreach of these regions, providing evidence for partialmediation. There was also evidence that encoding–retrievalsimilarity in the occipital ROIs mediates the relationshipbetween hippocampal activity and memory. These findingsspeak to the dynamic influences of hippocampal activity andcortical pattern similarity at retrieval on each other and onmemory. Finally, for inferior frontal and occipital cortices, thecorrelation between pattern similarity and hippocampalactivity at retrieval was significant across trials even when re-stricted to remembered items only, P < 0.0063. Correlationsdid not differ between remembered and forgotten items,P > 0.05. Taken together, activity in the hippocampus isassociated with greater fidelity between neocortical represen-tations at encoding and retrieval, and this relationship facili-tates successful memory performance.

Figure 3. The influence of item-specific similarity on memory. Encoding–retrieval similarity at the level of individual items predicted memory success above and beyond set-levelsimilarity in the middle occipital, middle temporal, and supramarginal gyri, evidence by the interaction of memory and match. The spatial localization of these effects wasascertained by splitting these ROIs into subregions (A), among which the maximal interaction was observed in the left anterior-dorsal subregion of middle occipital gyrus (B).Finally, mean ROI retrieval activation and encoding–retrieval similarity independently predicted across-trial variation in memory success, demonstrating a lack of redundancyamong these measures (C). Error bars denote standard error of the mean. D, distance; rem, remembered; forg, forgotten; mid, middle.

6 Encoding–Retrieval Similarity and Memory • Ritchey et al.

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

Emotional Modulation of Cortical SimilarityEmotion globally increased the similarity of encoding and re-trieval patterns (main effect of emotion, P < 0.0026) in manyregions including the occipital, inferior and middle temporal,and supramarginal gyri (Table 4), in that encoding–retrieval

similarity was greater for negative and positive than neutralitems, consistent with an influence of arousal. Emotion alsoaffected the difference in similarity between item- andset-level match pairs (interaction of emotion and match, F2,36= 6.31, P = 0.004) in middle occipital gyrus (Fig. 5A,B). The

Figure 4. Hippocampal mediation of the link between cortical similarity and memory expression. Mediation analysis tested the hypothesis that the hippocampus mediates therelationship between encoding–retrieval cortical similarity and memory success (A). Among regions showing memory-modulated encoding–retrieval similarity effects, theinfluence of encoding–retrieval similarity in inferior frontal, inferior parietal, and occipital ROIs on memory was significantly mediated by the hippocampus (B). Most of theseregions were marked by a significant correlation between encoding–retrieval cortical similarity and hippocampal activation at retrieval, across remembered items; color-codingreflects the strength of this relationship. E–R, encoding–retrieval.

Table 2Within-subjects logistic regression of memory performance on ROI activity and encoding–retrieval similarity

Region Encoding activation Retrieval activation Encoding–retrieval similarity

Mean beta t(18) P Mean beta t(18) P Mean beta t(18) P

OccipitalCalcarine −0.02 −0.49 0.315 0.06 2.56 0.010 0.04 1.19 0.124Cuneus −0.15 −4.49 0.000 0.14 4.15 0.000* 0.05 1.94 0.034Lingual 0.04 1.09 0.145 0.02 0.72 0.240 0.06 2.20 0.021Inf occipital 0.12 2.82 0.006 0.08 2.69 0.008 0.13 3.29 0.002Mid occipital 0.02 0.62 0.273 0.07 2.15 0.023 0.15 4.23 0.000*Sup occipital −0.01 −0.24 0.408 0.05 1.31 0.103 0.12 4.53 0.000*

TemporalFusiform 0.12 2.64 0.008 0.12 4.01 0.000* 0.04 0.89 0.192Inf temporal 0.07 2.19 0.021 0.13 5.06 0.000* 0.13 4.56 0.000*Mid temporal −0.02 −0.57 0.287 0.13 4.27 0.000* 0.15 4.96 0.000*Mid temporal Pole 0.04 1.62 0.062 −0.01 −0.23 0.410 0.01 0.18 0.43Sup temporal Pole 0.04 1.65 0.058 −0.01 −0.66 0.257 0.01 0.51 0.308Sup temporal −0.13 −3.33 0.002 0.05 1.97 0.032 0.06 3.35 0.002

FrontalInf frontal opercularis 0.00 0.17 0.433 0.11 2.63 0.008 0.14 5.57 0.000*Inf frontal orbitalis 0.02 0.68 0.252 0.17 4.77 0.000* 0.05 1.69 0.054Inf frontal triangularis 0.02 0.73 0.236 0.13 3.14 0.003 0.14 4.69 0.000*Mid frontal −0.14 −3.92 0.001 0.01 0.46 0.326 0.11 3.36 0.002Mid frontal orbital −0.13 −3.92 0.001 0.09 2.85 0.005 −0.03 −0.92 0.185Sup frontal −0.07 −2.46 0.012 0.01 0.32 0.375 0.03 0.84 0.205Sup medial frontal 0.03 0.97 0.174 0.15 4.76 0.000* 0.04 1.40 0.089Sup frontal orbital −0.05 −2.11 0.025 0.08 3.00 0.004 0.02 0.87 0.197Insula −0.07 −2.26 0.018 0.07 3.43 0.002 0.03 1.28 0.108

ParietalAngular −0.12 −4.04 0.000 0.13 4.69 0.000* 0.12 3.57 0.001*Inf parietal −0.12 −3.66 0.001 0.04 0.94 0.181 0.15 4.81 0.000*Sup parietal −0.01 −0.25 0.404 −0.03 −0.76 0.229 0.11 3.84 0.001*Precuneus −0.09 −2.27 0.018 0.04 1.91 0.036 0.10 3.31 0.002Supramarginal −0.06 −1.62 0.061 0.12 4.06 0.000* 0.16 8.19 0.000*

MTLAmygdala 0.07 2.44 0.016 0.08 3.28 0.002 0.01 0.41 0.344Hippocampus 0.07 2.29 0.017 0.12 6.07 0.000* 0.00 −0.06 0.476Parahippocampal gyrus 0.07 2.29 0.017 0.12 4.37 0.000* −0.01 −0.28 0.390

Note: Mean beta refers to the logistic regression beta coefficient, averaged across the group.*Denote regions significant at Bonferroni-corrected threshold.MTL, medial temporal lobe; Inf, inferior; Mid, middle; Sup, superior.

Cerebral Cortex 7

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

relationship between memory and encoding–retrieval simi-larity was consistent across emotion types. The middle occipi-tal region was further explored via functional connectivityand mediation analyses linking amygdala activity at retrievalto encoding–retrieval similarity. The correlation betweenamygdala activity and encoding–retrieval similarity in middle

occipital gyrus showed a significant emotion by memoryinteraction, F2,32 = 4.10, P = 0.026 (Fig. 5C). This interactionreflected stronger correlations for remembered than forgottentrials among negative items, t(16) = 2.07, P = 0.027, 1-tailed,but not positive or neutral items, Ps > 0.05. Mediation analysisshowed that amygdala activity did not mediate the relation-ship between encoding–retrieval similarity and memory, orvice versa (Table 3), indicating that the hippocampus remainsthe primary link between cortical similarity and memory.Altogether, these findings suggest that emotional arousal isassociated with heightened encoding–retrieval similarity, par-ticularly in posterior cortical regions, and the recapitulation ofnegative information tends to engage the amygdala duringsuccessful retrieval.

Discussion

These results provide novel evidence that successful memoryis associated with superior match between encoding andretrieval at the level of individual items. Furthermore, hippo-campal involvement during retrieval mediates this relationshipon a trial-to-trial basis, lending empirical support to theoriespositing dynamic interactions between the hippocampal andneocortex during retrieval. These findings generalize to mem-ories associated with emotional arousal, which magnifies thesimilarity between encoding and retrieval patterns.

Neural Evidence for Encoding–Retrieval SimilarityPrior reports have highlighted the reactivation of set-levelinformation from the initial learning episode during memoryretrieval, marked by spatial overlap between task-related

Table 3Mediation analysis linking encoding–retrieval similarity to memory

Mean beta coefficients Mean X–Mcorrelation

X→M (a) M→ Y (b) Mediated X→ Y (c’) Unmediated X→ Y (c) Mediation term (a × b) Rem Forg

E–R similarity (X) to memory (Y), mediated by left HC activity at retrieval (M)Mid occipital 0.104* 0.115* 0.091* 0.102* 0.009* 0.132* 0.069*Sup occipital 0.089* 0.119* 0.064* 0.073* 0.008* 0.100* 0.084 *Inf temporal 0.049 0.120* 0.054* 0.058* 0.004 0.051 0.050Mid temporal 0.022 0.122* 0.073* 0.074* 0.002 0.031 0.005Inf frontal

opercularis0.087* 0.117* 0.061* 0.070* 0.007* 0.099* 0.065*

Inf frontaltriangularis

0.090* 0.114* 0.077* 0.086* 0.007* 0.111* 0.064*

Inf parietal 0.040* 0.122* 0.082* 0.085* 0.004* 0.047 0.011Supramarginal −0.005 0.123* 0.070* 0.069* −0.010 0.003 −0.033

Left HC activity at retrieval (X) to memory (Y), mediated by E–R similarity (M)Mid occipital 0.110* 0.091* 0.115* 0.124* 0.005* — —

Sup occipital 0.093* 0.064* 0.119* 0.124* 0.003* — —

Inf temporal 0.051 0.054* 0.120* 0.123* 0.000 — —

Mid temporal 0.023 0.073* 0.122* 0.123* 0.000 — —

Inf frontalopercularis

0.091* 0.061* 0.117* 0.124* 0.003 — —

Inf frontaltriangularis

0.095* 0.077* 0.114* 0.123* 0.005 — —

Inf parietal 0.040* 0.082* 0.122* 0.123* 0.001 — —

Supramarginal −0.005 0.070* 0.123* 0.123* 0.000 — —

E–R similarity (X) to memory (Y), mediated by right amygdala activity at retrieval (M)Mid occipital 0.044 0.036* 0.103* 0.103* 0.000 0.070 0.046

Right amygdala activity at retrieval (X) to memory (Y), mediated by E–R similarity (M)Mid occipital 0.046 0.103* 0.036* 0.039* 0.002 — —

*Denote regions significant at Bonferroni-corrected threshold.Inf, inferior; Mid, middle; Sup, superior; rem, remembered; forg, forgotten; E–R, encoding–retrieval; HC, hippocampus.

Table 4Regions showing emotional modulation of encoding–retrieval similarity

Effect Region Match × memory× emotion ANOVA

F2,36 P

Main effect of emotion Mid occipital 19.45 0.000*Inf temporal 17.61 0.000*Inf occipital 16.46 0.000*Angular 15.38 0.000*Inf parietal 12.43 0.000*Supramarginal 9.05 0.000*Mid temporal 8.29 0.001*Sup occipital 6.51 0.002*Sup temporal 5.11 0.006Sup parietal 4.95 0.006Inf frontal opercularis 4.92 0.006Fusiform 4.26 0.011Sup frontal 4.13 0.012Inf frontal triangularis 3.88 0.015

Emotion × match interaction Mid occipital 6.31 0.002*Fusiform 3.41 0.022

Emotion × memory interaction Inf frontal opercularis 3.85 0.015Emotion × match × memory interaction Fusiform 3.52 0.020

Note: Regions showing marginal effects (P< 0.025) are also shown. The main effects andinteractions not involving emotion are presented separately in Table 1.*Denote regions significant at Bonferroni-corrected threshold.Inf, inferior; Mid, middle; Sup, superior.

8 Encoding–Retrieval Similarity and Memory • Ritchey et al.

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

hemodynamic responses at each phase (reviewed by Rugget al. 2008; Danker and Anderson 2010). Multivariate patternanalysis methods have advanced these findings by demon-strating that neural network classifiers, trained to associateexperimental conditions with hemodynamic patterns from en-coding, can generate predictions about which type of infor-mation is reactivated at retrieval (reviewed by Rissman andWagner 2012), including patterns associated with the memor-andum’s category (Polyn et al. 2005), its encoding task(Johnson et al. 2009), or its paired associate (Kuhl et al.2011). Importantly, despite these methodological improve-ments, evidence thus far has been limited to broad set-level dis-tinctions tied to the encoding manipulation. Overlaps betweenencoding and retrieval will most benefit discrimination betweenold and new items when they carry information specific to indi-vidual stimuli.

The present study advances this line of research by provid-ing essential evidence that the similarity between encodingand retrieval operations can be tracked at the level of individ-ual items. For some regions, particularly in occipitotemporalcortices, encoding–retrieval similarity is most affected bymemory success when trials are matched at the level of indi-vidual items. The presence of this interaction provides directevidence for the theory of encoding–retrieval match, in thatincreased similarity is associated with superior memory per-formance. Item similarity may reflect cognitive and perceptualoperations particular to the individual stimulus that occurduring both encoding and retrieval or the reactivation of pro-cesses or information associated with initial encoding. Bothforms of information may serve to induce brain states at retrie-val that more closely resemble the brain states of their encod-ing counterparts. Unlike previously reported results, thesemeasures are not constrained by category or task and aretherefore ideal for flexibly capturing instances of the encod-ing–retrieval match tailored to each individual person andtrial. In many regions spanning frontal, parietal, and occipitalcortices, encoding–retrieval similarity predicted memorysuccess when mean activation did not, suggesting that inthese regions, pattern effects may be more informative thanunivariate estimates of activation.

Recent investigations using pattern similarity analysis haverelated variation across encoding trials to memory success, ex-ploiting the ability of similarity measures to tap into therepresentational structure underlying neural activity

(Kriegeskorte 2008). Jenkins and Ranganath (2010) usedpattern distance to link contextual shifts during encoding tosuccessful memory formation. Another set of experiments de-monstrated that increased similarity between encoding rep-etitions, calculated separately for each individual stimulus,predicts memory success, suggesting that memories benefitfrom consistency across learning trials (Xue et al. 2010). Incontrast, the present study focuses on the similarities betweenprocessing the same mnemonic stimuli at 2 separate phasesof memory—namely, the overlap between encoding and ex-plicit recognition processes. Because a different task is re-quired at each phase, the similarity between encoding andretrieval trials likely arises from either perceptual attributes,which are common between remembered and forgotten trials,or from information that aids or arises from mnemonic recov-ery, which differ between remembered and forgotten trials.Thus, any differences between remembered and forgottenitems in the item-level pairs should reflect item-specific pro-cesses that benefit memory or the reactivation ofencoding-related information during the recognition period.

Hippocampal–Cortical Interactions During SuccessfulMemory RetrievalBecause pattern similarity measures yield item-specific esti-mates of encoding–retrieval overlap, they offer new opportu-nities to investigate the role of the hippocampus in supportingneocortical memory representations. Novel analyses relatingtrial-by-trial fluctuations in cortical pattern similarity to hippo-campal retrieval responses revealed that the hippocampus par-tially mediates the link between encoding–retrieval similarityand retrieval success across a number of neocortical ROIs.Many theories of memory have posited dynamic interactionsbetween the hippocampus and neocortex during episodic re-trieval (Alvarez and Squire 1994; McClelland et al. 1995;Sutherland and McNaughton 2000; Norman and O’Reilly2003), such that hippocampal activation may be triggered bythe overlap between encoding and retrieval representations ormay itself promote neocortical pattern completion processes.The mediation models tested here suggest some combinationof both mechanisms. However, evidence for the hippocampalmediation account tended to be stronger and distributedacross more regions, possibly stemming from our use of a rec-ognition design. Interestingly, in our mediation models,

Figure 5. Influence of emotion on encoding–retrieval similarity and amygdala responses during retrieval. Emotion significantly enhanced overall encoding–retrieval similarityeffects in a number of posterior regions and specifically enhanced item-level effects within the middle occipital gyrus (A). The middle occipital gyrus showed greater encoding–retrieval similarity for negative and positive relative to the neutral trials (B). Encoding–retrieval similarity in middle occipital gyrus was additionally correlated across trials withretrieval activity in the amygdala; this correlation was modulated by memory success for negative but not positive or neutral trials (c). Error bars denote standard error of themean. Neg, negative; pos, positive; neut, neutral; MOG, middle occipital gyrus.

Cerebral Cortex 9

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

hippocampal activation did not fully account for the linkbetween encoding–retrieval similarity and memory, implyingthat both hippocampal and neocortical representations arerelated to memory retrieval (McClelland et al. 1995; Wiltgenet al. 2004).

Implications of Emotional Modulation of CorticalPattern SimilarityEmotion is known to modulate factors relevant to encoding–retrieval similarity, such as perceptual encoding (Dolan andVuilleumier 2003) and memory consolidation (McGaugh2004), in addition to enhancing hippocampal responsesduring both encoding (Murty et al. 2010) and retrieval (Bu-chanan 2007). However, its influence on encoding–retrievalmatch has never before been investigated. Here, we presentnovel evidence that, across all trials, emotion heightens thesimilarity between encoding and retrieval patterns in severalregions in the occipital, temporal, and parietal cortices. Therelationship between encoding–retrieval similarity andmemory was consistent across levels of emotion, suggestingthat memory-related pattern similarity is affected by emotionin an additive rather than interactive way. One possible expla-nation for these findings is that emotion might generally in-crease the likelihood or strength of hippocampal–corticalinteractions that contribute to both consolidation and retrieval(Carr et al. 2011). Another possible explanation is thatemotion might facilitate perceptual processing during bothencoding and retrieval, thus resulting in heightened similaritybetween the encoding and retrieval of emotional items, par-ticularly among regions devoted to visual perception. Consist-ent with the latter interpretation, the amygdala appears tomodulate emotional scene processing in extrastriate regionsincluding middle occipital gyrus but not in early visual areaslike the calcarine sulcus (Sabatinelli et al. 2009).

Functional connectivity analyses revealed that neural simi-larity in the occipital cortex correlated with amygdala activityduring successfully retrieved negative trials. Negative memoriesare thought to be particularly sensitive to perceptual infor-mation (reviewed by Kensinger 2009), and enhanced memoryfor negative visual details has been tied to the amygdala and itsfunctional connectivity with late visual regions (Kensinger et al.2007). Increased reliance on perceptual information, alongwith observed memory advantages for negative relative to posi-tive stimuli, may account for these valence-specific effects.Unlike the hippocampus, however, the amygdala did notmediate the link between similarity and memory, suggestingthat amygdala responses during rapid scene recognition maybe driven by the recovery and processing of item-specificdetails. These observations stand in contrast to results obtainedin a more effortful cued recall paradigm, in which amygdalaactivation appeared to play a more active role in the successfulrecovery of autobiographical memories (Daselaar et al. 2008).In the present study, the hippocampus remained the primaryroute by which encoding–retrieval similarity predicted memoryperformance, for both emotional and neutral trials.

Future DirectionsThe present study used a recognition task during the retrievalphase, which has the benefit of matching perceptual infor-mation and thus encouraging maximal processing overlapbetween encoding and retrieval. Through this approach, we

demonstrated that encoding–retrieval similarity varies withmemory success and is related to hippocampal activation, con-sistent with the positive influence of encoding–retrieval overlapor transfer-appropriate processing. However, it is worth notingthat considerable task differences and the collection of dataacross multiple sessions may have limited the sensitivity of thecurrent design to event similarities. Although we are encour-aged by the results reported here, future experiments usingclosely matched tasks might be better suited to uncoveringsubtle changes in pattern similarity, for example, as a functionof stimulus type. Furthermore, the use of a recognition designlimits our ability to ascertain the direction of hippocampal–cortical interactions, in that processing overlaps in corticalregions might trigger hippocampal activation or vice versa.Another approach would be to measure cognitive and neuralresponses reinstated in the absence of supporting information,as in a cued- or free-recall design. This control would ensurethat the similarity between encoding and retrieval patternswould be driven by information arising from hippocampalmemory responses, rather than being inflated by perceptualsimilarities. This test would be an important step toward deter-mining whether hippocampal pattern completion processescan drive cortical similarities between encoding and retrieval.

Conclusions

In conclusion, these data provide novel evidence that episo-dic memory success tracks item-specific fluctuations in en-coding–retrieval match, as measured by the patterns ofhemodynamic activity across the neocortex. Importantly, thisrelationship is mediated by the hippocampus, providing criti-cal evidence for hippocampal–neocortical interactionsduring the reactivation of memories at retrieval. We alsoreport new evidence linking emotion to enhancements inencoding–retrieval similarity, a mechanism that may bridgeunderstanding of how emotion impacts encoding, consolida-tion, and retrieval processes. Altogether, these findingsspeak to the promise of pattern similarity measures for evalu-ating the integrity of episodic memory traces and for measur-ing communication between the medial temporal lobes anddistributed neocortical representations.

Supplementary MaterialSupplementary material can be found at: http://www.cercor.oxfordjournals.org/.

Funding

This work was supported by National Institutes of Healthgrants #NS41328 (R.C. and K.S.L.) and #AG19731 (R.C.). M.R.was supported by National Research Service Award#F31MH085384.

NotesWe thank Alison Adcock, Vishnu Murty, Charan Ranganath, andChristina L. Williams for helpful suggestions. Conflict of Interest:None declared.

10 Encoding–Retrieval Similarity and Memory • Ritchey et al.

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from

ReferencesAlvarez P, Squire LR. 1994. Memory consolidation and the medial

temporal lobe: a simple network model. Proc Natl Acad Sci.91:7041–7045.

Atlas LY, Bolger N, Lindquist MA, Wager TD. 2010. Brain mediators ofpredictive cue effects on perceived pain. J Neurosci. 30:12964–12977.

Bonnici HM, Kumaran D, Chadwick MJ, Weiskopf N, Hassabis D,Maguire EA. 2011. Decoding representations of scenes in themedial temporal lobes. Hippocampus. 22:1143–1153.

Bower GH. 1972. Stimulus-sampling theory of encoding variability.In: Melton AW, Martin E, editors. Coding processes in humanmemory. Washington (DC): V.H. Winston & Sons. p. 85–123.

Buchanan T. 2007. Retrieval of emotional memories. Psychol Bull.133:761–779.

Carr MF, Jadhav SP, Frank LM. 2011. Hippocampal replay in theawake state: a potential substrate for memory consolidation andretrieval. Nat Neurosci. 14:147–153.

Danker JF, Anderson JR. 2010. The ghosts of brain states past: remem-bering reactivates the brain regions engaged during encoding.Psychol Bull. 136:87–102.

Daselaar SM, Fleck MS, Cabeza R. 2006. Triple dissociation in themedial temporal lobes: recollection, familiarity, and novelty. J Neu-rophysiol. 96:1902–1911.

Daselaar SM, Rice HJ, Greenberg DL, Cabeza R, LaBar KS, Rubin DC.2008. The spatiotemporal dynamics of autobiographical memory:neural correlates of recall, emotional intensity, and reliving. CerebCortex. 18:217–229.

Dolan RJ, Vuilleumier P. 2003. Amygdala automaticity in emotionalprocessing. Ann N Y Acad Sci. 985:348–355.

Dupret D, O’Neill J, Pleydell-Bouverie B, Csicsvari J. 2010. The reor-ganization and reactivation of hippocampal maps predict spatialmemory performance. Nat Neurosci. 13:995–1002.

Hofstetter C, Achaibou A, Vuilleumier P. 2012. Reactivation of visualcortex during memory retrieval: content specificity and emotionalmodulation. NeuroImage. 60:1734–1745.

Jenkins LJ, Ranganath C. 2010. Prefrontal and medial temporal lobeactivity at encoding predicts temporal context memory. J Neurosci.30:15558–15565.

Ji D, Wilson MA. 2007. Coordinated memory replay in the visualcortex and hippocampus during sleep. Nat Neurosci. 10:100–107.

Johnson J, McDuff S, Rugg M, Norman K. 2009. Recollection, famili-arity, and cortical reinstatement: a multivoxel pattern analysis.Neuron. 63:697–708.

Kay KN, Naselaris T, Prenger RJ, Gallant JL. 2008. Identifying naturalimages from human brain activity. Nature. 452:352–355.

Kensinger EA. 2009. Remembering the details: effects of emotion.Emotion Rev. 1:99–113.

Kensinger EA, Garoff-Eaton RJ, Schacter DL. 2007. How negativeemotion enhances the visual specificity of a memory. J CognNeurosci. 19:1872–1887.

Kriegeskorte N. 2008. Representational similarity analysis – connectingthe branches of systems neuroscience. Front Syst Neurosci. 2:1–28.

Kuhl B, Rissman J, Chun MM, Wagner AD. 2011. Fidelity of neuralreactivation reveals competition between memories. Proc NatlAcad Sci. 108:5903–5908.

LaBar KS, Cabeza R. 2006. Cognitive neuroscience of emotionalmemory. Nat Rev Neurosci. 7:54–64.

Lang PJ, Bradley MM, Cuthbert BN. 2001. International affectivepicture system (IAPS): instruction manual and affective ratings. In:Technical report A-5. Gainesville (FL): The Center for Research inPsychophysiology, University of Florida.

Maldjian JA, Laurienti PJ, Kraft RA, Burdette JH. 2003. An automatedmethod for neuroanatomic and cytoarchitectonic atlas-basedinterrogation of fMRI data sets. NeuroImage. 19:1233–1239.

McClelland JL, McNaughton BL, O’Reilly RC. 1995. Why there arecomplementary learning systems in the hippocampus and neo-cortex: insights from the successes and failures of connectionistmodels of learning and memory. Psychol Rev. 102:419–457.

McGaugh JL. 2004. The amygdala modulates the consolidation ofmemories of emotionally arousing experiences. Ann Rev Neurosci.27:1–28.

Morris CD, Bransford JD, Franks JJ. 1977. Levels of processing versustransfer appropriate processing. J Verbal Learning Verbal Behav.16:519–533.

Murty V, Ritchey M, Adcock RA, LaBar KS. 2010. fMRI studies of suc-cessful emotional memory encoding: a quantitative meta-analysis.Neuropsychologia. 48:3459–3469.

Nadel L, Samsonovich A, Ryan L, Moscovitch M. 2000. Multiple tracetheory of human memory: computational, neuroimaging, and neu-ropsychological results. Hippocampus. 10:352–368.

Norman KA, O’Reilly RC. 2003. Modeling hippocampal and neocorti-cal contributions to recognition memory: a complementary-learning-systems approach. Psychol Rev. 110:611–646.

Nyberg L, Habib R, McIntosh AR, Tulving E. 2000. Reactivation ofencoding-related brain activity during memory retrieval. Proc NatlAcad Sci. 97:11120–11124.

O’Neill J, Pleydell-Bouverie B, Dupret D, Csicsvari J. 2010. Play itagain: reactivation of waking experience and memory. TrendsNeurosci. 33:220–229.

Pennartz CMA, Lee E, Verheul J, Lipa P, Barnes CA, McNaughton BL.2004. The ventral striatum in off-line processing: ensemble reacti-vation during sleep and modulation by hippocampal ripples. JNeurosci. 24:6446–6456.

Polyn SM, Natu VS, Cohen JD, Norman KA. 2005. Category-specificcortical activity precedes retrieval during memory search. Science.310:1963–1966.

Rissman J, Gazzaley A, D’Esposito M. 2004. Measuring functional con-nectivity during distinct stages of a cognitive task. NeuroImage.23:752–763.

Rissman J, Wagner AD. 2012. Distributed representations in memory: in-sights from functional brain imaging. Ann Rev Psychol. 63:101–128.

Ritchey M, LaBar KS, Cabeza R. 2011. Level of processing modulatesthe neural correlates of emotional memory formation. J CognNeurosci. 23:757–771.

Rugg MD, Johnson JD, Park H, Uncapher MR. 2008. Encoding-retrie-val overlap in human episodic memory: a functional neuroimagingperspective. Progress in Brain Research. 169: 339–352.

Sabatinelli D, Lang PJ, Bradley MM, Costa VD, Keil A. 2009. Thetiming of emotional discrimination in human amygdala andventral visual cortex. J Neurosci. 29:14864–14868.

Spaniol J, Davidson PS, Kim AS, Han H, Moscovitch M, Grady CL.2009. Event-related fMRI studies of episodic encoding and retrie-val: meta-analyses using activation likelihood estimation. Neurop-sychologia. 47:1765–1779.

Sutherland GR, McNaughton B. 2000. Memory trace reactivation inhippocampal and neocortical neuronal ensembles. Curr Opin Neu-robiol. 10:180–186.

Tulving E, Thomson DM. 1973. Encoding specificity and retrieval pro-cesses in episodic memory. Psychol Rev. 80:352–373.

Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O,Delcroix N, Mazoyer B, Joliot M. 2002. Automated anatomical label-ing of activations in SPM using a macroscopic anatomical parcella-tion of the MNI MRI single-subject brain. NeuroImage. 15:273–289.

Wager TD, Waugh CE, Lindquist M, Noll DC, Fredrickson BL, Taylor SF.2009. Brain mediators of cardiovascular responses to social threat:Part I: reciprocal dorsal and ventral sub-regions of the medial pre-frontal cortex and heart-rate reactivity. NeuroImage. 47:821–835.

Wheeler ME, Petersen SE, Buckner RL. 2000. Memory’s echo: vividremembering reactivates sensory-specific cortex. Proc Natl AcadSci. 97:11125–11129.

Wiltgen BJ, Brown RAM, Talton LE, Silva AJ. 2004. New circuits forold memories: the role of the neocortex in consolidation. Neuron.44:101–108.

Xue G, Dong Q, Chen C, Lu Z, Mumford JA, Poldrack RA. 2010.Greater neural pattern similarity across repetitions is associatedwith better memory. Science. 330:97–101.

Cerebral Cortex 11

at Medical C

enter Library, D

uke University on A

pril 2, 2013http://cercor.oxfordjournals.org/

Dow

nloaded from