Following the music to the memoriesPetr Janata

Center for Mind and Brain & Dept. of Psychology, Univ. of California, Davis, Davis, CA.

0.25 0.5 1 2 4

1000

2000N=13

MPFC(peak: x=-20, y=60, z=22)

0.25 0.5 1 2 4

600

1200N=13

Occipital/Temporal(peak: x=-18, y=-68, z=24)

0.25 0.5 1 2 4

170

340N=13

left VLPFC(peak: x=-42, y=38, z=6)

0.25 0.5 1 2 4

185

370N=8

left STS(peak: x=-50, y=-40, z=6)

0.25 0.5 1 2 4

60

120N=8

left prCG(peak: x=-46, y=-10, z=48)

0.25 0.5 1 2 4

50

100

N=9

right MFG(peak: x=40, y=20, z=38)

0.25 0.5 1 2 4

20

40

N=8

SFG(peak: x=6, y=28, z=54)

0.25 0.5 1 2 4

15

30N=8

FO/Insula(peak: x=34, y=24, z=-2)

0.25 0.5 1 2 4

Tonality-Tracking Bias

Autobiographical salience ratio(log10)

SalientNon-salient

-60mm -50 -40

-30 -20 -10

-5 +5 +10

+20 +30 +40

+50 +60mm3

4

5

6

#sub

ject

s

60 40 20 0 -20-40-60-8060 40 20 0 -20-40-60-8060 40 20 0 -20-40-60-80-40-20

0204060

-40-20

0204060

-40-20

0204060

-40-20

0204060

-40-20

0204060

+40 +25

+15 +5

-10 -40

-70mm

L R

-60 -40 -20 0 20 40 60-60 -40 -20 0 20 40 60-40

-20

0

20

40

60

-40

-20

0

20

40

60

-40

-20

0

20

40

60

-40

-20

0

20

40

60

VLPFC

VLPFC

MPFC

MPFC

STS

Music Playing

Question/AnswerPeriod

AutobiographicalSalience

Familiarity

Constant

Time (s)

Affective Valence

Music Playing

Question/AnswerPeriod

AutobiographicalSalience

Familiarity

Constant

Affective Valence

Subject 4

Subject 1

0 124 248 372 496 620 744 868

0 120 240 360 480 600 720 840

Run 1 Run 2

Time (s)Run 1 Run 2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Music Playing

Question/

Answ

er

Autobio

Familiarity

Valence

A B

Cor

rela

tion

Coe

ffici

ent

Figure 1. Design matrices for two subjects illustrate the statistical models that were used. The grayscale codes the regressor values, with black representing the most negative values and white the most positive. The ubiquitous gray represents zero. A) 30s music epochs alter-nated with the question/answer periods. Answers on the multipoint scalse were used to code parametric regressors for Autobiographical Salience, Familiarity and Affective Valence. Black bars represent song excerpts that were unfamiliar or not autobiographically salient, whereas white bars represent songs that were familiar and strongly autobiographically salient. Valence ratings range from very displeasing (black) to very pleasing (white) with neither pleasing nor displeasing represented by zero. B) Correlation matrices of the design matrices shown in A. The Familiarity, Autobiographical Salience, and Valence regressors showed moderate to strong correlations among each other. This was true across all subjects.

IntroductionThe evocation of autobiographical memories and associated emotions by music counts among the most poignant experiences associated with music. Accordingly, excerpts of music can serve as potent retrieval cues with which to study the functional architecture of autobiographical memory. Converging evidence suggests that the rostral aspect of the medial prefrontal cortex (BA10) is a region where music and autobiographical memories might be integrated. This study tested the hypothesis that experiencing autobiographically salient musical excerpts will result in increased activation in the medial prefrontal cortex, and that the medial prefrontal cortex will track the tonal (harmonic) structure of the musical excerpts, particularly the autobiographically salient ones.

MethodsPrescreening and memory characterization• Thirty 30s excerpts from Billboard Top 100 Pop and R&B charts • Randomly selected from time when subject was between 7 and 19 years of age. • Following each excerpt, a subject reported whether the song was familiar and if so whether it evoked a weakly or strongly autobiographical memory or no clear memory at all.• Subjects were invited to participate in the fMRI version of the experiment if >30% of the excerpts evoked an autobiographical memory.

fMRI experiment• 13 subjects (11 female) • 30 stimuli (selected according to the same criteria as above) across two scanning runs. • No repeats of stimuli from prescreen. • Ratings following each excerpt: • Affective valence (5-point scale) • Arousal (5-pt scale) • Familiarity (2 pt scale) • Autobiographical salience (3 pt scale) • Degree of orientation of their attention toward the memories (5 pt scale) • Degree of orientation of their attention toward the music (5 pt scale).

Scan parameters: Siemens 3T Trio, 34 slices (4 mm thick, 0 skip), TR=2.0s, in-plane resolution: 3.4x3.4 mm.

Analyses: Analyses were performed using SPM5. Data were spatially normalized, and variance associated with estimated movement parameters and linear trends was removed. The residuals were analysed using the models shown in Figures 1 and 4.

Modeling music’s movement in tonal spaceDesign matrices for testing general task effectsand parametric variation in subjective variables

Group-level analyses of each subjectivevariable contrast

Group-level analyses of task effects and combination of

Familiarity, Autobiographical Salience, Affect

Converging evidence from music theory, cognitive psychology, and self-organizing neural networks indi-cates that the system of major and minor keys in West-ern tonal music can be represented on the surface of the torus.As a piece of music unfolds, the changing harmonies and chord progressions create a changing pattern of activity on the surface of the torus, providing a signa-ture of a piece of music.

Subject 4

cc10

cc01

cs01cs02cs03sc10sc21sc13sc21

cc02cc03

cc11cc12cc13cs11cs12cs13sc20sc12sc23ss12ss23

constant

Time (s)

0 124 248 372 496 620 744 868

Run 1 Run 2

Tona

lity

Reg

ress

ors

(sph

eric

al h

arm

onic

s)

Aut

obio

grap

hica

lN

on-

Aut

obio

grap

hica

l

Figure 4. Design matrix in which tonality tracking is estimated separately for songs that did and didn’t elicit autobiographical associations. False-positives were controlled for by comparing the residual variance of the veridical model against the distribution of residual variances obtained from 100 models in which the ordering of the songs was permuted.

f ( , ) = amncc cos(m )cos(n ) +

m= 0,n= 0

M, N

amncs cos(m )sin(n ) +

m= 0,n= 0

M, N

amnsc sin(m )cos(n ) +

m= 0, n= 0

M, N

amnss sin(m )sin(n )

m=0,n =0

M ,N

cs01 0.73

Spherical harmonic Cumulative sum

sc11 -0.30

sc10 -0.22

cc11 -0.19

cs11 -0.19

Original activation

The activation pattern on the torus at any given moment in time can be described as a weighted-sum of a set of spherical harmonics (spatial frequencies) using the equation below.

momentary activationpattern

θ

φ

unfolded torus

As the activation pattern changes during the course of a song, so do the weights for each spherical harmonic. These changing weights are entered as regressors for each of 34 spherical harmonics.

100 200 300 400 500 600 700

-1.5

-1

-0.5

0

0.5

1

1.5

Volume (TR=3s)

Z-va

lue

Spherical harmonic regressor 4

Time

1s

3s

5s

7s

9s

11s

Subject 1 Subject 2 Subject 3

Tona

lity

Trac

king

-55mm -40

+0 +40

+55

-55mm -35

+0 +35

+55

-45mm -35

-10 +15

+35

60 40 20 0 -20 -40 -60 -80

-40-20

0204060

-40-20

0204060

60 40 20 0 -20 -40 -60 -80

-40-20

0204060

60 40 20 0 -20 -40 -60 -80

32

1

23

ratio

autobio

non-autobio

Tonality-tracking responses

Figure 5. These tonality tracking images show the proportion of variance explained by the tonality tracking regressors modeling autobiographically salient pieces of music relative to those modeling pieces of music that elicited no autobio-graphical association. The color scale represents the ratio of these variances, with each variance corrected by the number of music excerpts contributing to its estimate. Red values indicate a bias toward tonality tracking associated with an autobiographical memory, and blue denotes tonality tracking without an associated autobiographical memory.

Figure 6. A summary of brain areas exhibiting tonality-tracking behavior. The color of each voxel indicates the number of subjects in whom that voxel showed significant tonality-tracking behavior. The cyan contours demarcate the borders of the FAV-related activations shown in Figure 2, and illustrate the co-localization of the FAV and tonality-tracking effects in MPFC. The insets show distributions of tonality tracking biases for all voxels in clusters of voxels that showed tonality tracking for a majority of subjects. A ratio of 1 indicates equal tracking of autobiographically salient and non-salient songs, whereas larger ratios indicate a bias toward tracking of autobiographically salient songs. The box and whisker plots provide information about the distribution of mean tonality-tracking biases across subjects.

-60mm -50 -40

-30 -20 -10

-5 +5 +10

+20 +30 +4060 40 20 0 -20-40-60-80

-40-20

0204060

+50 +60mm

+40mm +25

+15 +5

-10 -40-60 -40 -20 0 20 40 60

-40

-20

0

20

40

60

-70mm

60 40 20 0 -20-40-60-8060 40 20 0 -20-40-60-80

-40-20

0204060

-40-20

0204060

-40-20

0204060

-40-20

0204060

L R

-60 -40 -20 0 20 40 60

-40

-20

0

20

40

60

-40

-20

0

20

40

60

-40

-20

0

20

40

60

STS

IPS

OFC

SFGIFS

VLPFC

VMPFC

MPFC

DLPFC

FO FO

Ins

STG

STG

pSMA

pSMA

ACC

PCC

pSMA

AG

AG

Cb

SPL

Music QAP

FAV

all contrastsp<0.005

Figure 2. Group analyses for 3 basic contrasts: Music Playing (green), the Question/Answer Period (blue) and a contrast in which the Familiarity, Autobiographical Salience, and Valence effects were combined. They were combined because of the strong correlations among these variables for all subjects. Thus, areas in red, yellow, and magenta showed greater activation when the song was familiar, strongly autobiographically salient and evoked a positive emotion.

-60mm -50 -40

-30 -20 -10

-5 +5 +10

+20 +30 +40

+50 +60mm

60 40 20 0 -20-40-60-80-40-20

0204060

60 40 20 0 -20-40-60-8060 40 20 0 -20-40-60-80

-40-20

0204060

-40-20

0204060

-40-20

0204060

-40-20

0204060

3.54

4.55

t-val

ue

+40mm +25

+15 +5

-10 -40

-70

L R

-60 -40 -20 0 20 40 60-40

-20

0

20

40

60

-40

-20

0

20

40

60

-40

-20

0

20

40

60

-40

-20

0

20

40

60

-60 -40 -20 0 20 40 60

PHG

DMPFC

IFG/FO

MTG

ITS

vltn

vltn

Cun

Cun

VLPFCVLPFC

PTIns

ins

DMPFC

VMPFC

nAcc

nAcc

PHG

pSMA

pSMA

all contrastsp<0.005

Figure 3. Individual effects of familiarity (green), autobiographical salience (red), and af-fective (positive) valence (blue). For purposes of comparison, the cyan outlines are the borders of the FAV contrast shown in red in Figure 2.

Vale

nce

Fam

iliar

Aut

obio

ConclusionsDorsal medial prefrontal cortex exhibits properties of an area that binds together music with autobiographically salient memories, in that it responds more strongly during autobiographically salient songs and it also follows the structure of the music, predominantly for songs that are autobiographically salient. Left VLPFC shows similar properties. The effects of familiarity and autobiographical salience (lateralized to the left hemisphere) and tonality tracking were distributed across a number of prefrontal, temporal, and occipital areas, as would be expected given the interactions of frontal memory retrieval mechanisms with content representations in posterior cortices.

Acknowledgments: I wish to thank John Ryan, Mike Buonocore, Sonja Rakowski, Susannah Cohen, and Cheryline Mancusi for assistance in data collection. Supported in part by NIH R03 DC05146-01 and a Templeton Advanced Research Program grant from the Metanexus Institute