The Role of Prior Knowledge in Human Reconstructive Memory

Mark Steyvers

Pernille Hemmer

University of California, Irvine

1

Interaction between Prior Knowledge & Episodic Memory

2

Episodic MemoryPrior Knowledge,

Semantic Memory,“Schemas”

“What objects do you remember from your hotel room?”“What coffee did you order from Starbucks last week?”

reconstructive memory

Research questions

Machine learning How can we formalize prior knowledge (“schemas”)?

Psychology How do humans integrate memory and prior knowledge? What errors do people make?

Interface between human & machine learning Can we explain human error with rational inference principles? Can we build better information retrieval systems by taking

cognitive processes into account?

3

Remembering Objects from a Graduate Office

Brewer & Treyens (1981)

chairdeskskull

books

(30% of subjects)

Experiments

perception: list objects you see in a scene memory: list objects you remember from a scene guessing: list objects you might see in a kitchen/office/…

5

Dining Hotel Kitchen

Human Data: Precision vs. Output Position

6

2 4 6 8 10 12 14 160.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Output Position

Cu

mu

lativ

e A

ccur

acy

Time = 10secsTime = 2secsScene Cue

guessing

10 Secs.

2 Secs

Pre

cisi

on

Bayesian Analysis of Reconstructive Memory

7

( | ) ( | ) ( )p objects memory p memory objects p objects

Posterior

What objects were studied given noisy memory contents?

Prior

How likely are these objects a

priori?

Likelihood

How likely is this memory content

given these objects were studied?

Problem 1: Infer prior knowledge

8

kitchen1 kitchen3kitchen2

PriorKnowledge ?

Hierarchical Beta processes (Thibaux & Jordan, 2007); related to Indian Buffet process

Hierarchical Beta Process(Thibaux & Jordan, 2007)

9

“Kitchen”

Kitchen65

objects

pa

rtic

ipa

nts

Kitchen45

0 0 0 0~ Beta( , (1 ))b c b c b

Generalknowledge

Object probs. for specific

image

Reported objects

y

ja

b

~ Beta( , (1 ))j j ja c b c b

~ Bernoulli( )ij jy a

Problem 2: Recall using Prior + Episodic M.

10

?

PriorKnowledge b

Actual objects in image

Noisy Memory Content

y

x

semantic memory

episodic memory

11

Data Model

2 4 6 8 10 12 14 160.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Output Position

Cu

mu

lativ

e A

ccur

acy

=0.200=0.150=0.000

2 4 6 8 10 12 14 160.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Output Position

Cumu

lative

Acc

uracy

Time = 10secsTime = 2secsScene Cue10 Secs.

2 Secs

Pre

cisi

on

prior only

Limitations of Model

Independence assumption between objects

Recall often clusters correlated objects “computer” “mouse” “keyboard”

12

Reconstructing the Style of Written Digits

13

Study

Reconstructed

studied digit

reconstructed

personalprior

Digit space from “Earth Mover” distances

Reconstructing Drawings from Memory

14

Studied Drawing Reconstruction

Reconstructing Lists of Words

15

Study this list:PEAS, CARROTS, BEANS, SPINACH, LETTUCE, HAMMER, TOMATOES, CORN, CABBAGE, SQUASH

HAMMER,PEAS,

CARROTS,...

Conclusion

Prior knowledge has strong effect on memory Guessing based on prior knowledge leads to quite good

performance

Machine learning methods help formalize “schemas”

Useful to understand human memory when designing information retrieval systems

16

Thanks!

17