Learning to Rank Typed Graph Walks: Local and Global Approaches
description
Transcript of Learning to Rank Typed Graph Walks: Local and Global Approaches
![Page 1: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/1.jpg)
Learning to Rank Typed Graph Walks:
Local and Global Approaches
Einat Minkov and William W. Cohen
Language Technologies Institute and Machine Learning Department School of Computer ScienceCarnegie Mellon University
![Page 2: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/2.jpg)
Did I forget to invite anyone for this meeting?
![Page 3: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/3.jpg)
Did I forget to invite anyone for this meeting?
What is Jason’s personalemail address ?
![Page 4: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/4.jpg)
Did I forget to invite anyone for this meeting?
What is Jason’s personalemail address ?
Who is “Mike” who is mentioned in this email?
![Page 6: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/6.jpg)
Q: “what are Jason’s email aliases?”
“Jason”
Msg5
Msg18
Sent fromEmail
Sent toEmail
JasonErnst
Sent-to
Similar to
Msg 2
Sent To
einat
Has terminverse
![Page 7: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/7.jpg)
Search via lazy random graph walks An extended similarity measure via graph walks:
![Page 8: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/8.jpg)
Search via lazy random graph walks An extended similarity measure via graph walks:
Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths.
![Page 9: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/9.jpg)
Search via lazy random graph walks An extended similarity measure via graph walks:
Fixed probability of halting the walk at every step – i.e., shorter connecting paths have greater importance (exponential decay)
Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths.
![Page 10: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/10.jpg)
Search via lazy random graph walks An extended similarity measure via graph walks:
Fixed probability of halting the walk at every step – i.e., shorter connecting paths have greater importance (exponential decay)
Finite graph walk, applied through sparse matrix multiplication
(estimated via sampling for large graphs)
Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths.
![Page 11: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/11.jpg)
Search via lazy random graph walks An extended similarity measure via graph walks:
Fixed probability of halting the walk at every step – i.e., shorter connecting paths have greater importance (exponential decay)
Finite graph walk, applied through sparse matrix multiplication
(estimated via sampling for large graphs)
The result is a list of nodes, sorted by “similarity” to an input node distribution (final node probabilities).
Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths.
![Page 12: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/12.jpg)
The graph Graph nodes are typed.
Graph edges - directed and typed (adhering to the graph schema)
Multiple relations may hold between two given nodes.
Every edge type is assigned a fixed weight.
![Page 13: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/13.jpg)
Graph walks
graph walk controlled by edge weights Θ , walk length K and stay probability γ
The probability of reaching y from x in one step: the sum of edge weights from x to y, out of the total outgoing weight from x.
The transition matrix assumes a stay probability at the current node at every time step.
A query language:
Q: { , }
The graph
Nodes
Node type
Edge label
Edge weightx
y2
3
3
Probability of following blue edge out of x is
2/ (2+3+3)
x
y2
3
3
Probability of following blue edge out of x is
2/ (2+3+3)
Returns a list of nodes
(of type ) ranked by
the graph walk probs.
![Page 14: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/14.jpg)
TasksPerson namePerson namedisambiguationdisambiguation
ThreadingThreading
Alias findingAlias finding
[ term “andy” file msgId ]
“person”
[ file msgId ]
“email-file”
What are the adjacent messages in this thread?
A proxi for finding generally related messages.
What are the email-addresses of Jason ?...
[ term Jason ]
“email-address”
![Page 15: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/15.jpg)
Learning to Rank
Typed Graph Walks
![Page 16: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/16.jpg)
Learning settings
Query a
node rank 1
node rank 2
node rank 3
node rank 4
…
node rank 10
node rank 11
node rank 12
…
node rank 50
Query b Query q
node rank 1
node rank 2
node rank 3
node rank 4
…
node rank 10
node rank 11
node rank 12
…
node rank 50
node rank 1
node rank 2
node rank 3
node rank 4
…
node rank 10
node rank 11
node rank 12
…
node rank 50
…
GRAPH WALK
+ Rel. answers a + Rel. answers b + Rel. answers q
Task T (query class)
![Page 17: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/17.jpg)
Graph walk
Weightupdate
Theta*
Learning approachesEdge weight tuning:
![Page 18: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/18.jpg)
Graph walk
Weightupdate
Graph walk
Learning approachesEdge weight tuning:
Theta*
task
![Page 19: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/19.jpg)
Graph walk
Graph walk
Feature generation
Weightupdate
Updatere-ranker
Re-rankingfunction
Graph walk
Learning approachesEdge weight tuning:
Node re-ordering:
Theta*
task
![Page 20: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/20.jpg)
Graph walk
Graph walk
Feature generation
Weightupdate
Updatere-ranker
Re-rankingfunction
Graph walk
Graph walk
Feature generatio
n
Score byre-ranker
Learning approachesEdge weight tuning:
Node re-ordering:
Theta*
task
task
![Page 21: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/21.jpg)
Learning approaches
• Exhaustive local search over edge type (Nie et-al, 05)
• Gradient descent (Chang et-al, 2000)
• Hill climbing error backpropagation (Dilligenti et-al, IJCAI-05)
• Gradient descent approximation for partial order preferences (Agarwal et-al, KDD-06)
• Re-ranking (Minkov, Cohen and NG, SIGIR-06)
Graphparameters’tuning
Nodere-ordering
• Can be adapted from extended PageRank settings to finite graph walks.
• Strong assumption of first-order Markov dependencies
• A discriminative learner, using graph-paths describing features.
• Loses some quantitative data in feature decoding. However, can represent edge sequences.
![Page 22: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/22.jpg)
Error Backpropagation
Cost function:
Weight updates:
Where,
following Dilligenti et-al, 2005
![Page 23: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/23.jpg)
follows closely on (Collins and Koo, Computational Linguistics, 2005)
Scoring function:
Adapt weights to minimize (boosted version):
, where
Re-ranking
![Page 24: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/24.jpg)
Path describing Features
K=0 K=1 K=2
X1
X2
X3
X4
X5
x2 x1 x3
x4 x1 x3
x4 x2 x3
x2 x3
‘Edge unigram’was edge type l used in reaching x from Vq?
‘Edge (n-)bigram’ were edge types l1 and l2 traversed (in that order) in reaching x from Vq?
‘Top edge (n-)bigram’ same, where only the top k contributing paths are considered.
‘Source count’ indicates the number of different source nodes in the set of connecting paths.
Paths [x3, k=2]:
![Page 25: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/25.jpg)
Learning to Rank Typed Graph Walks:
Local vs. Global approaches
![Page 26: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/26.jpg)
Experiments
Gradient descent: Θ0 ΘG
Reranking: R(Θ0)
Combined: R(ΘG)
Methods:
Tasks &Corpora :
![Page 27: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/27.jpg)
The results (MAP)
0.4
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
M.game sager Shapiro
0.4
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
M.game Farmer Germany
0.4
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
Meetings
Namedisambiguation
Threading
Alias finding
MAP
*
*
*
*
*
*
*
*
** *
+
+
+
+ +
*
![Page 28: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/28.jpg)
Nam
ed
isam
big
uati
on
Th
read
ing
Ali
as
fin
din
g
![Page 29: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/29.jpg)
Our Findings Re-ranking often preferable due to ‘global’ features:
Models relation sequences.
e.g., threading: sent-from sent-to-inv
Re-ranking rewards nodes for which the set of connecting paths is diverse.
source-count feature informative for complex queries
The approaches are complementary
Future work:
Re-ranking: large feature space.
Re-ranking requires decoding at run-time.
Domain specific features
![Page 30: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/30.jpg)
Related papersEinat Minkov, William W. Cohen, Andrew Y. Ng Contextual Search and Name Disambiguation in Email using GraphsSIGIR 2006
Einat Minkov, William W. CohenAn Email and Meeting Assistant using Graph Walks CEAS 2006
Alekh Agarwal, Soumen ChakrabartiLearning Random Walks to Rank Nodes in GraphsICML 2007
Hanghang Tong, Yehuda Koren, and Christos Faloutsos Fast Direction-Aware Proximity for Graph Mining KDD 2007
![Page 31: Learning to Rank Typed Graph Walks: Local and Global Approaches](https://reader035.fdocuments.net/reader035/viewer/2022062517/56813cfe550346895da69f8e/html5/thumbnails/31.jpg)
Thanks! Questions?