Jong Y. Choi, Joshua Rosen, Siddharth Maini, Marlon E. Pierce, and Geoffrey C. Fox Community Grids...
-
Upload
edwina-mccormick -
Category
Documents
-
view
215 -
download
0
Transcript of Jong Y. Choi, Joshua Rosen, Siddharth Maini, Marlon E. Pierce, and Geoffrey C. Fox Community Grids...
Jong Y. Choi, Joshua Rosen, Siddharth Maini, Marlon E. Pierce, and Geoffrey C. Fox
Community Grids LaboratoryIndiana University
Delicious example
2
Bookmark
Tags
SocialNetwork
s
People-generate
d
Collaborative Tagging Online bookmarking
with annotations Create social networks Utilize power of
people’s knowledge Pros and cons
High-quality classifier by using human intelligence
But lack of control or authority
3
4
5
Search ResultSOAP, REST, …
Repository
Query with various options
RDFRSSAtomHTML
Populate Bookmarks/ tags
Distributed Tagging Data
CCT System
Data Coordinator
User Service
Data Importer
Collective Collaborative Tagging (CCT) System
6
1st - Service and algorithm development Identify services and algorithms
2nd - Interface development Web2.o style interface REST, SOAP, …
3rd – Export/import service development Merging distributed data sets Export data to build mesh-up sites
So far, we are mainly in 1st stage and do some experiments in 2nd stage
7
Different Data Sources
Various IR algorithms
Flexible Options
Result Comparison
8
SearchingSearching
Given input tags, returning the most relevant X (X = URLs, tags, or users)
Given input tags, returning the most relevant X (X = URLs, tags, or users)
Latent Semantic Indexing (LSI), FolkRank
Latent Semantic Indexing (LSI), FolkRank
II
Recommendation
Recommendation
Indirect input tags, returning undiscovered XIndirect input tags, returning undiscovered XIIII
ClusteringClustering
Community discovering. Finding a group or a community with similar interests
Community discovering. Finding a group or a community with similar interests
K-Means, Deterministic Annealing Clustering
K-Means, Deterministic Annealing Clustering
IIIIII
Trend detection
Trend detection
Analysis the tagging activities in time-series manner and detect abnormality
Analysis the tagging activities in time-series manner and detect abnormality
Time Series AnalysisTime Series AnalysisIVIV
Service DescriptionAlgorithm
Type
Vector-space model (bag-of-words model) Assume n URLs and q tags A URL can be represented by q-dimension
vector, di = (t1, t2, … , tq)
A total data set can be represented by n-by-q matrix
Pairwise Dissimilarity Matrix n-by-n symmetric matrix Distance (Euclidean, Manhattan, … ) Angles, cosine, sine, … O(n2) complexity
9
10(Source : MSI-CIEC)
Graph model Building a graph with nodes and edges Edges are indicating relationship Becoming complex networks (tag graph)
Dissimilarity Related with path distance Finding path is important
(Shortest path problem) Naive approach :
O(n3) complexity
Latent Semantic Indexing Using vector-space model, find the most
similar URLs with user’s query tags Dimension reduction from high q to low d (q
>> d) Removing noisy terms, extracting latent
concepts
11Precision
Reca
ll
2 terms4 terms8 terms20% dim. reductionNone
Ideal Line
Discover the group structures of URLs Non-parametric learning algorithm
Non-trivial optimization problem Should avoid local minima/maxima solution
12
Deterministically avoid local minima Tracing global solution by changing level of
energy Analogy to physical annealing process (High
Low)
13
Classification To response more quickly to user’s requests Training data based on user’s input and
answering questions based on the training results
Artificial Neural Network, Support Vector Machine,…
Trend Detection Can be used for prediction/forecasting Time-series analysis of tagging activities Markov chain model, Fourier transform, …
14
The goal of our Collective Collaborative Tagging (CCT) system Utilize various data sets Provide various information retrieval (IR)
algorithms Help to utilize people-powered knowledge
Currently various models and algorithms are being investigated
Service interfaces and import/export function will be added soon
15
16
17
-. Distances, cosine, …-. O(N2) complexity-. Distances, cosine, …-. O(N2) complexity
Dis-similarity
Dis-similarity
Vector-space Model
-. Paths, hops, connectivity, …-. O(N3) complexity
-. Paths, hops, connectivity, …-. O(N3) complexity
Graph Model
-. Latent Semantic Indexing-. Dimension reduction schemes-. PCA
-. Latent Semantic Indexing-. Dimension reduction schemes-. PCA
AlgorithmAlgorithm-. PageRank, FolkRank, …-. Pairwise clustering-. MDS
-. PageRank, FolkRank, …-. Pairwise clustering-. MDS
-. q-dimensional vector-. q-by-n matrix-. q-dimensional vector-. q-by-n matrix
Represen-tation
Represen-tation
-. G(V, E) -. V = {URL, tags, users}-. G(V, E) -. V = {URL, tags, users}
Pairwise clustering Input from vector-based model vs. graph
model How to avoid local minima/maxima? (e.g, K-
Means)
18
Graph modelVector-space model