Representation, Development and Disintegration of Conceptual Knowledge: A Parallel-Distributed...
-
Upload
toby-sparks -
Category
Documents
-
view
219 -
download
1
Transcript of Representation, Development and Disintegration of Conceptual Knowledge: A Parallel-Distributed...
Representation, Development and Disintegration of Conceptual Knowledge:
A Parallel-Distributed Processing Approach
James L. McClellandDepartment of Psychology and
Center for Mind, Brain, and ComputationStanford University
• Representation is a pattern of activation distributed over neurons within and across brain areas.
• Bidirectional propagation of activation underlies the ability to bring these representations to mind from given inputs.
• The knowledge underlying propagation of activation is in the connections.
language
Parallel Distributed Processing Approach to Semantic Cognition
A Principle of Learning and Representation
• Learning and representation are sensitive to coherent covariation of properties across experiences.
What is Coherent Covariation?
• The tendency of properties of objects to co-occur in clusters.
• e.g.– Has wings– Can fly– Is light
• Or– Has roots– Has rigid cell walls– Can grow tall
Development and Degeneration
• Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge.
• Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.
Some Phenomena in Development
• Progressive differentiation of concepts• Overgeneralization • Illusory correlations
The Rumelhart Model
The Training Data:
All propositions true of items at the bottom levelof the tree, e.g.:
Robin can {grow, move, fly}
Target output for ‘robin can’ input
aj
ai
wij
neti=ajwij
wki
Forward Propagation of Activation
k ~ (tk-ak)
wij
i ~ kwki
wki
aj
Back Propagation of Error ()
Error-correcting learning:
At the output layer: wki = kai
At the prior layer: wij = jaj
…
ai
Experience
Early
Later
LaterStill
• Waves of differentiation reflect coherent covariation of properties across items.
• Patterns of coherent covariation are reflected in the principal components of the property covariance matrix.
• Figure shows attribute loadings on the first three principal components:
– 1. Plants vs. animals– 2. Birds vs. fish– 3. Trees vs. flowers
• Same color = features covary in
component• Diff color = anti-covarying
features
What Drives Progressive
Differentiation?
CoherenceTrainingPatterns
No labels are providedEach item and each property occurs with equal frequency
Properties
Coherent Incoherent
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Items
is can has is can has …
Effect of Coherence on Representation
Overgeneralization of Frequent Names to Similar Objects
“dog”
“goat”“tree”
Illusory Correlations
• Rochel Gelman found that children think that all animals have feet.– Even animals that look like small furry balls
and don’t seem to have any feet at all.• A tendency to over-generalize properties
typical of a superordinate category at an intermediate point in development is characteristic of the PDP network.
A typical property thata particular object lackse.g., pine has leaves
An infrequent,atypical property
Sensitivity to Coherence Requires Convergence
A
A
A
Another key property of the model
• Sensitivity to coherent covariation can be domain- and property-type specific, and such sensitivity is acquired as differentiation occurs.
• Obviates the need for initial domain-specific biases to account for domain-specific patterns of generalization and inference.
Differential Importance (Marcario, 1991)
• 3-4 yr old children see a puppet and are told he likes to eat, or play with, a certain object (e.g., top object at right)– Children then must choose
another one that will “be the same kind of thing to eat” or that will be “the same kind of thing to play with”.
– In the first case they tend to choose the object with the same color.
– In the second case they will tend to choose the object with the same shape.
Adjustments to Training
Environment
• Among the plants:– All trees are large– All flowers are small– Either can be bright or
dull• Among the animals:
– All birds are bright– All fish are dull– Either can be small or
large• In other words:
– Size covaries with properties that differentiate different types of plants
– Brightness covaries with properties that differentiate different types of animals
Testing Feature Importance
• After partial learning, model is shown eight test objects:– Four “Animals”:
• All have skin• All combinations of bright/dull and large/small
– Four “Plants”:• All have roots• All combinations of bright/dull and large/small
• Representations are generated by usingback-propagation to representation.
• Representations are then compared to see which animals are treated as most similar, and which plants are treated as most similar.
Similarities of Obtained Representations
Size is relevant for Plants
Brightness is relevant for Animals
Development and Degeneration
• Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge.
• Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.
Disintegration of Conceptual Knowledge in Semantic Dementia
• Progressive loss of specific knowledge of concepts, including their names, with preservation of general information
• Overgeneralization of frequent names• Illusory correlations
Picture namingand drawing inSem. Demantia
Grounding the Model in What we Know About The Organization of Semantic
Knowledge in The Brain
• There is now evidence for specialized areas subserving many different kinds of semantic information.
• Semantic dementia results from progressive bilateral disintegration of the anterior temporal cortex.
• Rapid acquisition of new knowledge depends on medial temporal lobes, leaving long-term semantic knowledge intact.
language
Proposed Architecture for the Organization of Semantic Memory
colorform
motion
action
valance
Temporal pole
name
Medial Temporal Lobe
Rogers et al (2005) model of semantic dementia
• Gradually learns through exposure to input patterns derived from norming studies.
• Representations in the temporal pole are acquired through the course of learning.
• After learning, the network can activate each other type of information from name or visual input.
• Representations undergo progressive differentiation as learning progresses.
• Damage to units within the temporal pole leads to the pattern of deficits seen in semantic dementia.
name assocfunction
temporal pole
vision
Severity of Dementia Fraction of Neurons Destroyed
omissions
within categ.
superord.
Patient Data Simulation Results
Errors in Naming for As a Function of Severity
Simulation of Delayed Copying
• Visual input is presented, then removed.
• After several time steps, pattern is compared to the pattern that was presented initially.
• Omissions and intrusions are scored for typicality
name assocfunction
temporal pole
vision
Omissions by feature type Intrusions by feature type
IF’s ‘camel’ DC’s ‘swan’
Simulation results
Development and Degeneration
• Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge.
• Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.
Sensitivity to Coherence Requires Convergence
A
A
A