ABSURDIST II: A Graph Matching Algorithm and its Application to Conceptual System Translation

Ying FengYing FengRobert GoldstoneRobert GoldstoneVladimir MenkovVladimir Menkov

Computer Science DepartmentComputer Science DepartmentPsychology DepartmentPsychology Department

Indiana UniversityIndiana University

How concepts get their meaningsHow concepts get their meanings

Conceptual webConceptual web– A concept’s meaning comes from its A concept’s meaning comes from its

connections to other concepts in the connections to other concepts in the same conceptual systemsame conceptual system

External groundingExternal grounding– A concept’s meaning comes from its A concept’s meaning comes from its

connection to the external worldconnection to the external world

Conceptual WebConceptual Web

PhilosophyPhilosophy– Conceptual role semanticsConceptual role semantics– Conceptual incommensurabilityConceptual incommensurability

PsychologyPsychology– Isolated and interrelated conceptsIsolated and interrelated concepts– Latent semantic analysisLatent semantic analysis

Computer ScienceComputer Science– Semantic networksSemantic networks– Intrinsic meaning in large databasesIntrinsic meaning in large databases

Externally Grounded ConceptsExternally Grounded Concepts

PhilosophyPhilosophy– The symbol grounding problemThe symbol grounding problem

PsychologyPsychology– Perceptual symbol systemsPerceptual symbol systems

Computer scienceComputer science– Embodied cognitionEmbodied cognition

Translation Across Conceptual SystemsTranslation Across Conceptual Systems

How can we determine that two people share a How can we determine that two people share a matching concept of something (such as matching concept of something (such as MushroomMushroom)?)?– The publicity of concepts: we want to say that two The publicity of concepts: we want to say that two

people both have a concept of people both have a concept of MushroomMushroom even though even though they know different things (Fodor, 1998)they know different things (Fodor, 1998)

Cross-person translation as a challenge to Cross-person translation as a challenge to conceptual web accounts of meaning (Fodor & conceptual web accounts of meaning (Fodor & Lepore, 1992)Lepore, 1992)– If a concept’s meaning depends on its role in its system, If a concept’s meaning depends on its role in its system,

and if two people have different systems, then they and if two people have different systems, then they can’t have the same meaningcan’t have the same meaning

Problem DefinitionProblem Definition

Given Conceptual Systems Given Conceptual Systems AA and and BB– Each has a set of conceptsEach has a set of concepts– Both have the same set of relation typesBoth have the same set of relation types

Match the concepts in Match the concepts in AA with those in with those in B B based onbased on– Internal relations between concepts in each Internal relations between concepts in each

systemsystem– Partially known (external) correspondence Partially known (external) correspondence

between the two systems between the two systems

Graph RepresentationGraph Representation

Represent conceptual systems as Represent conceptual systems as graphsgraphs– Concepts --> nodes in graphConcepts --> nodes in graph– Relations --> links in graphRelations --> links in graph

Featured graphFeatured graph– Directed vs undirectedDirected vs undirected– Labeled vs unlabeled Labeled vs unlabeled – Weighted vs unweightedWeighted vs unweighted

Example of Conceptual SystemsExample of Conceptual Systems

seal reindeer moose

polar bear

wolf

Coexists (weighted)

Hunts (unweighted)

0.9 0.1 0.6 0.8

0.4

A sample of Manitoba wildlife

phoque renne élan

l’ours polaire

loup

0.9 0.3 0.8

0.6

Un sample de fauna Québécoise

Graph MatchingGraph Matching

InputsInputs– Graphs representing Graphs representing AA and and BB– External correspondence matrix External correspondence matrix EE

OutputOutput– Correspondence matrix Correspondence matrix C (m x n)C (m x n)– 0 ≤ C(A0 ≤ C(Aq, q, BBxx) ≤ 1) ≤ 1 indicating the correspondence between indicating the correspondence between

concept concept AAqq andand BBxx

Principle of AlignmentPrinciple of Alignment– Aligning nodes so that the relations between Aligning nodes so that the relations between

each pair of nodes in one graph are similar to each pair of nodes in one graph are similar to those between the aligned nodes in the other those between the aligned nodes in the other graphgraph

Measuring Similarity of Relation BundlesMeasuring Similarity of Relation Bundles

Relation bundleRelation bundle– All relations between a given pair of nodes represented All relations between a given pair of nodes represented

as an M-dimensional vectoras an M-dimensional vector Measuring similarity between Relation BundlesMeasuring similarity between Relation Bundles

Sim(ASim(Aqq,A,Arr;B;Bxx,B,Byy) = 1 - Diff(A) = 1 - Diff(Aqq,A,Arr;B;Bxx,B,Byy))

Diff(ADiff(Aqq,A,Arr;B;Bxx,B,Byy) = ∑|w) = ∑|wii(A(Aqq,A,Arr)-w)-wii(B(Bxx,B,Byy)| / M)| / M 0 ≤ Sim ≤ 10 ≤ Sim ≤ 1

Aq

Ar

0.2 1.0

Bx

By

0.6 0.5

Correspondence MatrixCorrespondence Matrix

C = C =

00000011AA44

00110000AA33

11000000AA22

00001100AA11

BB44BB33BB22BB11

A permutation matrix A permutation matrix

Describing a one-to-one matchingDescribing a one-to-one matching

Correspondence MatrixCorrespondence Matrix

C = C =

0.40.40.70.70000AA44

0.70.70.40.40000AA33

00000011AA22

00001100AA11

BB44BB33BB22BB11

Not a permutation matrixNot a permutation matrix

Used to find the best one-to-one matchingUsed to find the best one-to-one matching

ABSURDIST II: An Optimization AlgorithmABSURDIST II: An Optimization Algorithm

Match quality of a permutation: global Match quality of a permutation: global edge similarity measure to maximizeedge similarity measure to maximizeGlobalEdgeSim(P) = GlobalEdgeSim(P) =

ββ∑∑q,rq,rSim(ASim(Aqq,A,Arr;B;BP(q)P(q),B,BP(r)P(r)) + α∑) + α∑q,rq,rE(AE(Aqq,B,BP(q)P(q)))

AB

Energy FunctionalEnergy Functional Generalization to an arbitrary correspondence Generalization to an arbitrary correspondence

matrix matrix CC: energy functional to maximize: energy functional to maximize

K(C) = α (E ∙ C) + β Excitation(C) - χ Inhibition(C)K(C) = α (E ∙ C) + β Excitation(C) - χ Inhibition(C) External( External( CC ) = ( ) = (E ∙ CE ∙ C)) Reward forReward for CC matching matching EE Excitation(Excitation( CC ) = ) =

∑∑q,r,x,y q,r,x,y Sim(ASim(Aqq,A,Arr;B;Bxx,B,Byy) C(A) C(Aq, q, BBxx) C(A) C(Ar, r, BByy) / (n-1)) / (n-1)

Reward for internal similarity matchingReward for internal similarity matching InhibitionInhibition( ( CC ) = ) =

(∑(∑q,r,xq,r,xC(AC(Aq,q,BBxx)C(A)C(Ar,r,BBxx) + ∑) + ∑q,x yq,x yC(AC(Aq,q,BBxx) C(A) C(Aq,q,BByy) )/(2(n-1))) )/(2(n-1))

Penalty for non-orthogonality of rows or columnsPenalty for non-orthogonality of rows or columns

Iterative Optimization ProcessIterative Optimization Process

Maximize Maximize K(C)K(C) on cube on cube QQ in in correspondence matrix spacecorrespondence matrix spaceCC00 = starting point = starting pointNNtt = = grad K(Cgrad K(Ctt) = net input) = net inputVVtt = = Damp( LNDamp( LNtt, C, Ct t )) = update = updateCCt+1t+1 = = CCt t + V + Vtt

Damp()Damp() = damping function= damping functionKeepsKeeps CCt+1t+1 inin QQ

LL = learning rate= learning rate Net input has external similarity, Net input has external similarity,

excitation, and inhibition termsexcitation, and inhibition terms

ConvergenceConvergence

Similar to steepest descent, but Similar to steepest descent, but damping ensures convergence to a damping ensures convergence to a maximum on cube maximum on cube QQ

Always converges to a maximum Always converges to a maximum (not necessarily global)(not necessarily global)

Alternatives: quadratic programming Alternatives: quadratic programming methods (NP-hard)methods (NP-hard)

Choosing ParametersChoosing Parameters

Choose learning rate Choose learning rate LL – Stay in cube Stay in cube QQ– Ensure convergence Ensure convergence – Maximize convergence speedMaximize convergence speed

Choosing ParametersChoosing Parameters

Influence of Influence of χ /βχ /β on the convergence on the convergence points of points of CC– Low Low χχ: progress along the principal : progress along the principal

eigenvector, eventualy converge to a matrix of eigenvector, eventualy converge to a matrix of all 1’s all 1’s

– High High χχ: converge to the closest permutation : converge to the closest permutation matrix matrix

Heuristic solution: choose Heuristic solution: choose χ /βχ /β to better to better balance excitation and inhibitionbalance excitation and inhibition- Start with a lower - Start with a lower χχ, adaptively vary (generally, , adaptively vary (generally,

increase) it to avoid convergence to a matrix of increase) it to avoid convergence to a matrix of all 1’sall 1’s

Iteration CostsIteration Costs

O(nO(n44)) terms in the formula for terms in the formula for NNtt Sparse conceptual systems: average Sparse conceptual systems: average

degree of node is degree of node is d << nd << n Exploiting sparsity to compute Exploiting sparsity to compute

update in update in O(nO(n2 2 dd 2 2)) operations operations

Testing ABSURDIST IITesting ABSURDIST II• Create conceptual systems for two peopleCreate conceptual systems for two people

– Create a set of Create a set of NN concepts in Person concepts in Person AA– Define relations radnomly between concepts of various Define relations radnomly between concepts of various

labels and with random weightslabels and with random weights– Copy these concepts to Person Copy these concepts to Person BB– Add noise to Add noise to BB's relations and/or their weights's relations and/or their weights

• Measure ABSURDIST II’s ability to recover true Measure ABSURDIST II’s ability to recover true alignmentsalignments– 200 separate runs200 separate runs– Initialize each correspondence unit to a certain value (Initialize each correspondence unit to a certain value (0.5)0.5)– Activation passing for a set number of iterationsActivation passing for a set number of iterations– Any concepts connected by a unit with more than a Any concepts connected by a unit with more than a

threshold (threshold (0.80.8) activity are assumed to be aligned) activity are assumed to be aligned

ABSURDIST II GUIABSURDIST II GUI

Noise Tolerence vs Graph SizeNoise Tolerence vs Graph Size

Noise Tolerence vs Graph DensityNoise Tolerence vs Graph Density

Coverage Noise vs Intensity NoiseCoverage Noise vs Intensity Noise

Iteration Steps vs Graph SizeIteration Steps vs Graph Size

Iteration Steps vs Graph DensityIteration Steps vs Graph Density

External Similarity SeedingExternal Similarity Seeding

Potential Applications of ABSURDIST IIPotential Applications of ABSURDIST II

• Object recognitionObject recognition– Within-object relations provide a strong Within-object relations provide a strong

constraint for aligning objectsconstraint for aligning objects• Analogical reasoning and similarityAnalogical reasoning and similarity

– Most models of analogy require highly Most models of analogy require highly structured, propositional representations structured, propositional representations ABSURDIST II can be applied when similarities ABSURDIST II can be applied when similarities are known, but structured representations are are known, but structured representations are hard to find: pictures, words, etc.hard to find: pictures, words, etc.

• Translating across large databasesTranslating across large databases – Ontologies, dictionaries,thesauri, etc.Ontologies, dictionaries,thesauri, etc.

ConclusionsConclusions• Connecting concepts to both the world and Connecting concepts to both the world and

each other is an attractive optioneach other is an attractive option– These connections are mutually supportive, not These connections are mutually supportive, not

antagonisticantagonistic– Within-system relational information may have Within-system relational information may have

surprisingly large influencessurprisingly large influences

• Absurdist II improvements on Absurdist IAbsurdist II improvements on Absurdist I– Graph representation of arbitrary conceputal Graph representation of arbitrary conceputal

systemssystems– Optimization frameworkOptimization framework– Better convergence behaviorBetter convergence behavior– Cost reduction from O(nCost reduction from O(n44) down to O(n) down to O(n2 2 dd 2 2))

Thank YouThank You

ABSURDIST II websiteABSURDIST II websitehttp://www.cs.indiana.edu/~yingfeng/ABSURDIST/http://www.cs.indiana.edu/~yingfeng/ABSURDIST/

Contact InfoContact Info– Ying Feng: Ying Feng: yingfeng@cs.indiana.eduyingfeng@cs.indiana.edu

– Robert Goldstone: Robert Goldstone: rgoldsto@indiana.edurgoldsto@indiana.edu

– Vladimir Menkov: Vladimir Menkov: vmenkov@cs.indiana.eduvmenkov@cs.indiana.edu