Conceptual Hierarchies Arise from the Dynamics of Learning and Processing:Conceptual Hierarchies Arise from the Dynamics of Learning and Processing:Insights from a Flat Attractor NetworkInsights from a Flat Attractor Network

Christopher M. O’ConnorChristopher M. O’Connor Ken McRaeKen McRae George S. CreeGeorge S. CreeUniversity of Western OntarioUniversity of Western Ontario University of Western OntarioUniversity of Western Ontario University of Toronto at ScarboroughUniversity of Toronto at Scarborough

London, Ontario, CanadaLondon, Ontario, Canada London, Ontario, CanadaLondon, Ontario, Canada Toronto, Ontario, CanadaToronto, Ontario, Canadacmoconno@uwo.ca mcrae@uwo.ca gcree@utsc.utoronto.ca

Superordinate & Basic-level Superordinate & Basic-level RepresentationsRepresentations

Superordinate Priming & Temporal Dynamics of Superordinate Priming & Temporal Dynamics of SimilaritySimilarity

both spreading activation (Collins & Loftus, 1975) and attractor both spreading activation (Collins & Loftus, 1975) and attractor networks predict that magnitude of semantic priming is networks predict that magnitude of semantic priming is determined by degree of semantic similaritydetermined by degree of semantic similarity supported experimentally using basic-level concepts (McRae & supported experimentally using basic-level concepts (McRae &

Boisvert, 1998) Boisvert, 1998) simulated using feature-based attractor nets (Cree, McRae, & simulated using feature-based attractor nets (Cree, McRae, &

McNorgan, 1999)McNorgan, 1999)

therefore, the degree that an exemplar target is primed by its therefore, the degree that an exemplar target is primed by its superordinate should vary as a function of typicalitysuperordinate should vary as a function of typicality high typicality > medium typicality > low typicalityhigh typicality > medium typicality > low typicality

however, Schwanenflugel and Rey (1986) found that short SOA however, Schwanenflugel and Rey (1986) found that short SOA superordinate priming does not vary as a function of target superordinate priming does not vary as a function of target exemplar typicalityexemplar typicality

replicated and simulated their experimentreplicated and simulated their experiment

ExperimentExperiment 72 superordinate-exemplar pairs, e.g., 72 superordinate-exemplar pairs, e.g., vegetablevegetable paired with paired with peaspeas, ,

turnipturnip, , garlicgarlic 12 superordinate primes with 2 exemplars each of low, 12 superordinate primes with 2 exemplars each of low,

medium, and high typicalitymedium, and high typicality 200ms superordinate prime, 50ms ISI, exemplar target until 200ms superordinate prime, 50ms ISI, exemplar target until

response (concrete object?)response (concrete object?)

Results: replicated Schwanenflugel and Rey (1986)Results: replicated Schwanenflugel and Rey (1986) main effect of relatedness, main effect of relatedness, FF1(1, 42) = 8.09, 1(1, 42) = 8.09, pp < .01, < .01, FF2(1, 66) = 2(1, 66) =

3.52, 3.52, pp < .07 < .07 no interaction between typicality & relatedness, no interaction between typicality & relatedness, FF1 < 1, 1 < 1, FF2 < 12 < 1

Feature VerificationFeature Verification

ConclusionsConclusions semantic memory can be represented as a single layer of semantic memory can be represented as a single layer of

semanticssemantics without a transparent hierarchical structurewithout a transparent hierarchical structure

accounts for graded structure of categoriesaccounts for graded structure of categories predicts online superordinate verification latencies; novel resultpredicts online superordinate verification latencies; novel result due to the temporal dynamics of similarity, accounts for due to the temporal dynamics of similarity, accounts for

counterintuitive and seemingly inconsistent results regarding counterintuitive and seemingly inconsistent results regarding basic-level vs. superordinate primingbasic-level vs. superordinate priming results counter to hierarchical spreading activation theoriesresults counter to hierarchical spreading activation theories

IntroductionIntroduction people’s conceptual knowledge structure for concrete nouns people’s conceptual knowledge structure for concrete nouns

traditionally viewed as hierarchical traditionally viewed as hierarchical (Collins & Quillian, 1969)(Collins & Quillian, 1969)

superordinate concepts (superordinate concepts (vegetablevegetable) represented at a ) represented at a different level in hierarchy than basic-level concepts different level in hierarchy than basic-level concepts ((carrotcarrot, or , or pumpkinpumpkin))

flat attractor networks – i.e., models with a single layer of flat attractor networks – i.e., models with a single layer of semantics – have provided insight to a number of phenomena semantics – have provided insight to a number of phenomena regarding basic-level conceptsregarding basic-level concepts

semantic primingsemantic priming statistically-based feature correlationsstatistically-based feature correlations concept-feature distributional statisticsconcept-feature distributional statistics

unclear how these networks could learn and represent unclear how these networks could learn and represent superordinate conceptssuperordinate concepts

can such a network account for established results and can such a network account for established results and provide novel insights?provide novel insights?

GoalsGoals demonstrate that a flat attractor network can learn demonstrate that a flat attractor network can learn

superordinate conceptssuperordinate concepts simulate typicality ratings to show model accounts for graded simulate typicality ratings to show model accounts for graded

structurestructure simulate feature verification latencies to demonstrate simulate feature verification latencies to demonstrate

superordinate representations may be computed similarly to superordinate representations may be computed similarly to basic-level conceptsbasic-level concepts

simulate superordinate semantic priming to provide insight into simulate superordinate semantic priming to provide insight into the temporal dynamics of similaritythe temporal dynamics of similarity

ModelModelStructureStructure input:input: 30 wordform units representing spelling/sound of a 30 wordform units representing spelling/sound of a

wordword output:output: 2349 semantic feature units representing features 2349 semantic feature units representing features

taken from McRae et al.’s (2005) feature production normstaken from McRae et al.’s (2005) feature production norms e.g., <has wings>, <made of metal>, <is red>, <has seeds>e.g., <has wings>, <made of metal>, <is red>, <has seeds>

single layer of semantics; taxonomic features removed; all single layer of semantics; taxonomic features removed; all semantic features were interconnectedsemantic features were interconnected

thus, no hierarchy built into the modelthus, no hierarchy built into the model

TrainingTraining model learned to map random 3-unit wordform for each model learned to map random 3-unit wordform for each

concept to semantic features for that conceptconcept to semantic features for that concept

basic-level concepts trained in 1-to-1 manner: basic-level concepts trained in 1-to-1 manner: 3-unit wordform paired with same set of semantic features 3-unit wordform paired with same set of semantic features

on every learning trialon every learning trial

superordinate concepts trained in 1-to-many mannersuperordinate concepts trained in 1-to-many manner wordform paired with semantic features of one of its wordform paired with semantic features of one of its

exemplars on each trialexemplars on each trial e.g., wordform for e.g., wordform for vegetablevegetable paired with features of paired with features of

carrotcarrot on one trial, on one trial, spinachspinach on another, etc. on another, etc.

each exemplar was presented equally ofteneach exemplar was presented equally often thus, typicality was NOT built into the modelthus, typicality was NOT built into the model

WordformWordform(30 units)(30 units)

SemanticSemanticFeaturesFeatures

(2349 units)(2349 units)

activation of features activation of features influenced by:influenced by:

Feature Frequency:Feature Frequency: if many exemplars if many exemplars

possess a feature, it is possess a feature, it is strongly activatedstrongly activated

Category Cohesion:Category Cohesion: degree of featural degree of featural

overlap of exemplars overlap of exemplars determines activation of determines activation of superordinate featuressuperordinate features

more overlap = more more overlap = more activationactivation

Feature Correlations:Feature Correlations: activate one another activate one another

during the computation during the computation of meaningof meaning

Superordinate Superordinate representations:representations: most features have most features have

intermediate intermediate activationsactivations

Basic-level Basic-level representations:representations: all features have all features have

activations close to activations close to 1 (on)1 (on)

FeatureFeature ActivationActivation

<is edible><is edible> .67.67<grows in gardens><grows in gardens> .45.45<is green><is green> .45.45<eaten by cooking><eaten by cooking> .44.44<is nutritious><is nutritious> .33.33<eaten in salads><eaten in salads> .31.31<is round><is round> .31.31<is small><is small> .31.31<tastes good><tastes good> .29.29<has seeds><has seeds> .24.24<is crunchy><is crunchy> .24.24<is white><is white> .24.24<used for cooking><used for cooking> .23.23<grows in ground><grows in ground> .22.22<has leaves><has leaves> .22.22

VegetableVegetableFeatureFeature ActivationActivation

<is crunchy><is crunchy> .94.94<is edible><is edible> .94.94<is green><is green> .93.93<is nutritious><is nutritious> .93.93<eaten with dips><eaten with dips> .92.92<grows in gardens><grows in gardens> .92.92<has fibre><has fibre> .92.92<has leaves><has leaves> .92.92<is stringy><is stringy> .92.92<tastes bland><tastes bland> .92.92<eaten in salads><eaten in salads> .91.91<has stalks><has stalks> .91.91<is long><is long> .91.91<tastes good><tastes good> .90.90

CeleryCelery

CategoryCategory NN Cosine/Cosine/ Fam Res/Fam Res/ Cosine/Cosine/TypicalityTypicality TypicalityTypicality Fam ResFam Res

furniturefurniture 1717 .76**.76** .62**.62** .78**.78**fruitfruit 2929 .71**.71** .69**.69** .91**.91**applianceappliance 1414 .61*.61* .73**.73** .89**.89**weaponweapon 3939 .58**.58** .70**.70** .76**.76**utensilutensil 2222 .57**.57** .52**.52** .68**.68**birdbird 2929 .57**.57** .49**.49** .69**.69**insectinsect 1313 .52*.52* .69**.69** .77**.77**carnivorecarnivore 1919 .52*.52* .45*.45* .83**.83**containercontainer 1414 .46*.46* .50*.50* .51**.51**vegetablevegetable 3131 .45**.45** .50**.50** .90**.90**musical musical instrumentinstrument 1818 .44*.44* .54*.54* .94**.94**clothingclothing 3939 .43**.43** .50**.50** .73**.73**tooltool 3434 .41**.41** .38*.38* .65**.65**fishfish 1111 .41.41 .36.36 .93**.93**

animalanimal 133133 .18*.18* .12.12 .55**.55**petpet 2222 .15.15 -.01-.01 .86**.86**herbivoreherbivore 1818 .04.04 .21.21 .78**.78**predatorpredator 1717 -.14-.14 .06.06 .60**.60**mammalmammal 5757 -.03-.03 .14.14 .64**.64**vehiclevehicle 2727 -.14-.14 .18.18 .72**.72**

* p* p < .05, ** < .05, ** pp < .01 < .01Fam Res = Family ResemblanceFam Res = Family Resemblance

Typicality RatingsTypicality Ratings important for any semantic important for any semantic

memory model to simulate graded memory model to simulate graded structurestructure

ExperimentExperiment collected behavioral typicality collected behavioral typicality

ratings for all 20 categories (7-ratings for all 20 categories (7-point scale)point scale)

SimulationSimulation superordinate wordform superordinate wordform

presented & representation presented & representation recordedrecorded

basic-level wordform basic-level wordform presented & representation presented & representation recordedrecorded

computed cosine similarity computed cosine similarity between each superordinate & between each superordinate & exemplarexemplar

computed correlation between computed correlation between typicality ratings & cosines for typicality ratings & cosines for each categoryeach category

correlation between typicality correlation between typicality ratings & family resemblance ratings & family resemblance served as baselineserved as baseline

ResultsResults

models predicts typicality ratings at least as well as family models predicts typicality ratings at least as well as family resemblanceresemblance

therefore, the model was successful in simulating graded structuretherefore, the model was successful in simulating graded structure

Feature VerificationFeature Verification similar “flat” attractor networks have simulated basic-level feature similar “flat” attractor networks have simulated basic-level feature

verificationverification model can also simulate verification of superordinate featuresmodel can also simulate verification of superordinate features

ExperimentExperiment 54 superordinate-feature pairs such as: 54 superordinate-feature pairs such as: furniturefurniture <made of wood> <made of wood>

& & fruitfruit <tastes sweet> <tastes sweet> superordinate name for 400 ms, feature name until participant superordinate name for 400 ms, feature name until participant

respondedresponded "Is the feature characteristic of the category?""Is the feature characteristic of the category?"SimulationSimulation present superordinate wordform and record feature's activation present superordinate wordform and record feature's activation

over 20 time ticksover 20 time ticks correlated model's feature activation with human verification correlated model's feature activation with human verification

latencylatency feature activation in model predicts human verification from ticks 6 feature activation in model predicts human verification from ticks 6

- 20- 20

ReferencesReferencesCollins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning Journal of Verbal Learning and Verbal Behavior, 8and Verbal Behavior, 8, 240-247., 240-247.Collins, A. M., & Loftus, E. F. (1975). A spreading activation theory of semantic processing. Collins, A. M., & Loftus, E. F. (1975). A spreading activation theory of semantic processing. Psychological Psychological Review, 82Review, 82, 407-428., 407-428.Cree, G. S., McRae, K, & McNorgan, C. (1999). An attractor model of lexical conceptual processing: Cree, G. S., McRae, K, & McNorgan, C. (1999). An attractor model of lexical conceptual processing: Simulating semantic priming. Simulating semantic priming. Cognitive Cognitive Science, 23Science, 23, 371-414., 371-414.McRae, K. & Boivert, S. (1998). Automatic semantic similarity priming. McRae, K. & Boivert, S. (1998). Automatic semantic similarity priming. Journal of Experimental Psychology: Journal of Experimental Psychology: Learning, Memory and Cognition,Learning, Memory and Cognition, 2424, 558-572., 558-572.McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature production norms for a McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving large set of living and nonliving things. things. Behavior Research Methods, 37Behavior Research Methods, 37, 547-559., 547-559.Schwanenflugel, P. J., & Rey, M. (1986). Interlingual semantic facilitation: Evidence for a common Schwanenflugel, P. J., & Rey, M. (1986). Interlingual semantic facilitation: Evidence for a common representational system in the bilingual representational system in the bilingual lexicon. lexicon. Journal of Memory and Language, 25Journal of Memory and Language, 25, 605-618., 605-618.

SimulationSimulation superordinate prime wordform superordinate prime wordform

presented to model for 15 tickspresented to model for 15 ticks exemplar target presented for 20 exemplar target presented for 20

ticksticks cross entropy error recorded over cross entropy error recorded over

last 20 tickslast 20 ticks

ResultsResults typicality & relatedness did not typicality & relatedness did not

interact, interact, FF < 1 < 1

main effect of relatedness, main effect of relatedness, FF(1, 66) (1, 66) = 187.27, = 187.27, pp < .001 < .001

related lower than unrelated for related lower than unrelated for ticks 1 to 13ticks 1 to 13

0

5

10

15

20

25

30

35

40

45

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Time Tick

Cross Entropy Error

Related - Low Typicality

Related - Medium Typicality

Related - High Typicality

Unrelated - Low Typicality

Unrelated - Medium Typicality

Unrelated - High Typicality

ExplanationExplanation why is priming from superordinate to why is priming from superordinate to

exemplar different than priming between exemplar different than priming between basic-level concepts?basic-level concepts?

superordinate features have superordinate features have intermediate activations, which (due to intermediate activations, which (due to the sigmoid activation function) require the sigmoid activation function) require less change in net input to be turned on less change in net input to be turned on or offor off

basic-level priming:basic-level priming: features in prime but not in target features in prime but not in target relatively difficult to turn offrelatively difficult to turn off prime & target must have high degree of featural overlap to prime & target must have high degree of featural overlap to

produce primingproduce priming superordinate priming:superordinate priming: activation of prime's features more activation of prime's features more

easily changedeasily changed priming still results (vs. unrelated superordinate), but less priming still results (vs. unrelated superordinate), but less

sensitive to similaritysensitive to similarity therefore, same amount of facilitation for exemplars of therefore, same amount of facilitation for exemplars of

all typicality levelsall typicality levels

AcknowledgementsNSERC grant OGP0155704 &NIH grant R01-MH6051701 to Ken McRae