Post on 23-Aug-2014
description
Physics Inspired Approachesto Community Detection
2012-09
1 / 52
Abstract
Community structure is one of the most relevantfeatures of graphs in sociology, biology, computerscience and so on.
In this talk, we review the following methods forcommunity detection:
synchronizationspinglass
2 / 52
Outline
...1 Introduction
...2 Synchronization
...3 Spinglass
...4 Summary
3 / 52
1. Introduction
4 / 52
Communities in real-world networks
biological networkprotein interaction, gene regulatory, metabolic, food chainsocial networkSNS, collaborators, phone/email, organizationtechnical systemweb graph, Internet, power gridother networkcitation, e-commerce/bidding, stock returnsdynamical phenomenaepidemic, cascade, synchronization, opinion change
5 / 52
NotationG = (V,E) graph(network)
vi ∈ V vertex(node), n = |V|(i, j) ∈ E edge(link) from vi to v j, 2m =
∑i j Ai j
Ai j adjacency matrix
Ai j =
1 (i, j) ∈ E0 otherwise
ki degree of vikout
i =∑
j Ai j, kinj =
∑i Ai j
wi j ≥ 0 weight of (i, j)wout
i =∑
j wi j, winj =
∑i wi j, 2w =
∑i j wi j
cs ∈ C community, q = |C|6 / 52
What is community detection
Communities are subgraphswithin which connections are dense, andbetween which they are sparse.
The concept of communityis not rigorously defined, andincludes some degree of arbitrariness.
Finding an exact solution is NP-hard in most cases.→ Thus an approximation algorithm is needed.
7 / 52
In this talk, we focus onnon-overlapping communitiesnon-dynamical graphssparse graphs: O(m) = O(n)
8 / 52
Girvan-Newman algorithm [1]
hierarchical divisive algorithmiteratively remove an edge with the highestbetweeness, and recalculate betweenessO(m2n)(for shortest-path betweeness version)
Alternative definitions of betweeness...1 shortest-path betweeness...2 flow betweeness...3 random-walk betweeness
9 / 52
Ref. [1]Ref. [1]
10 / 52
ModularityMany algorithms assume modularity Q as a measureof goodness of a partition. [2]
Q =1
2w
∑i j
wi j −win
i woutj
2w
δ(ci, c j)
=
q∑s=1
(wss
w− win
s wouts
4w2
)2wss =
∑i j wi jδ(ci, cs)δ(c j, cs), ws =
∑i wi jδ(ci, cs)
ci: community to which vi belongs
1st term: weight of within-community edges2nd term: expectation value of it for randomized graph
11 / 52
Greedy modularity optimization
hierarchical agglomerative algorithmiteratively merge communities to produce thelargest possible increase of modularityO((m + n)n) [2, Newman]
O(md log n), d = (depth of dendrogram) ∼ log nwith use of max-heap [4, Caluset-Newman-Moore]
Improvement of merging strategy [20].
12 / 52
Louvain method [25]
...1 assign its own community to each node
...2 iterate until no change happen...1 locally optimize each order in sequential
order until no change happen...2 replace communities by supernodes
Alternatively start with randomly assigned q < Ncommunities, and try several initial conditions.
A new random sequential order can be used eachtime.
13 / 52
Ref. [25]
14 / 52
Resolution limit of modularityModularity optimization may fail to identifycommunities smaller than an intrinsic sale
√m
measured by the number of links. [11]
mK
mK
mK
mK
mK
mKmK
mK
mKmK
A
pK
mK mK
pK
B
Ref. [11]
15 / 52
Infomap/map equation [22, 36]
Compress information of a random-walk trajectoryby Huffman code.
pi: visit frequency with teleportation ratio τqout
i : exit probability for cs
qexits = τ
n − ns
n
∑i
pi + (1 − τ)∑
i jci=cs, c j,cs
piAi j
16 / 52
✵✵✵✵
✵✵✶
✶✶
✶✵✵
✵✶
✶✵✶
✶✶✵✵✶✶
✵✵
✶✶✶
✶✵✶✵ ✶✵✵
✵✶✵
✵✶
✶✵
✵✵✶
✶✶✵✶✶
✵✵✵✵
✵✶✵
✶✶✵✶
✶✵
✵✵✶✶✶✶
✶✶✵✵
✵✶✵✶✶✵✶✶✶
✵✵✶✶✶✶✶✵✶✵✵✵✶✵✵✶✶✶✵✶✶✶✵✶✵✶✵✶✶✶✵✶✶✵✶✵✵✵✶✶✶✵✶✶✶✵✵✵✵✵✶✵✵✵✵✶✶✵✶✶✵✶✶✵✵✵✵✶✵✵✵✶✵✵✵✵✶✶✶✵✵✶✵✵✶✵✵✵
▲✭▼✮ ❂ �✳✁✸ ✰ ✷✳✾✼ ❂ ✸✳�✾ ❜✐ts
�✳✺ ✂
✁✳� ✂
✁✳✺ ✂
✷✳� ✂
Ref. [22]
17 / 52
Minimize the description length by Louvain method:
L =
∑s
qexits
H +∑
s
∑ici=cs
pi + qexits
Hs.
H is the entropy of a community-index codebook
H = −∑
s
qexits∑
r qexitr
log(
qs∑r qexit
r
).
Hs is the entropy of a within-community codebook
Hs = −qs
qexitr +
∑i
ci=cspi
log(
qs
qexitr +
∑i,ci=cs
pi
)
−∑
i
pi
qexiti +
∑i
ci=csp j
log
pi
qexiti +
∑i
ci=csp j
.18 / 52
Comparative EvaluationSynthetic graph for benchmark
degree distributioncommunity size distributiondensity radio of within/inter-community edgeshierarchical community structure
Similarity between clustering X and Y is measuredby normalized mutual information.
I(X,Y) =2∑
r,sNrsN log NrsN
Nr∗N∗s∑r
Nr∗N log Nr∗
N +∑
sN∗sN log N∗s
N
Nrs: #(node) assigned to a community r by X, and s by YN∗s =
∑r Nrs, Nr∗ =
∑s Nrs
19 / 52
Ref. [29]Ref. [26]
20 / 52
2. Synchronization
21 / 52
Kuramoto Model
θi = ωi +Kn
∑j
sin(θ j − θi)
ωi is distributed according to g(ω) with zero mean.Assume g(ω) is unimodal and g(ω) = g(−ω).
A mean-field order parameter defined by
reiϕ =1n
∑j eıθ j
yieldsθi = ωi + Kr sin(ϕ − θi).
22 / 52
The solutions exhibit two types of long-termbehavior. |ωi| ≤ Kr locked
|ωi| > Kr drift
Demanding the drifting oscillators form a stationarydistribution leads to a self-consistent equation:
r = Kr∫ π/2
−π/2cos2 θg(Kr sinθ).
A non-trivial solution (r > 0) is admitted beyond
Kc =2πg(0)
.
23 / 52
0
1
rinf
Kc K
Ref. [5]
24 / 52
Synchronization on network [28, review]
θi = ωi + K∑
j
Ai j sin(θ j − θi)
alternative choices of the coupling constant:
Ai j →Ai j
N,
Ai j
ki,
Ai j
⟨k⟩ .
For simplicity we focus on undirected & unweightedgraphs, and ki ≫ 1.
25 / 52
A local order parameter defined by
rieıϕi =∑
j Ai j⟨eıθ j⟩t
yields
θi = ωi − Kri sin(θi − ϕi) − Khi(t)
hu(t) = ℑ{e−ıθi
∑j ai j
(⟨eıθ j⟩t − eıθ j
)}.
hi is ignorable. Look for stationary solutions.
ri =∑|ω j|≤σr j
Ai jeı(θ j−ϕ j) +∑|ω j|>σr j
Ai j⟨eı(θ j−ϕ j)⟩t.
26 / 52
Suppose (ri, ϕi) is statistically independent of ωi.Ignore the 2nd term to get
ri =∑|ω j|≤σr j
Ai j cos(ϕ j − ϕi)
√1 −
(ω j
Kr j
)2
.
A critical coupling is obtained when cos(ϕ j − ϕi) = 1.Continuum approximation leads to
ri = K∑
j
Ai jr j
∫ 1
−1dxg(Kr jx)
√1 − x2
→ Kc =2πg(0)
1(largest eigenvalue of A)
.
27 / 52
Dynamical clustering [9]Suppose oscillators are identical ∀i, ωi = ω.→ Then full synchronization ∀i, θi = θ is possible.
A dense (sparse) subgraph synchronizerapidly (slowly).
start from random initial conditions Icalculate a local order parameter
ρi j(t) = ⟨cos(θi(t) − θ j(t))⟩Iand dynamical connectivity matrix
Dt(T)i j =
1 ρi j(t) > T (T: threshold)0 otherwise
.
28 / 52
Dt(T)i j gives a community structure.
Ref. [9]
29 / 52
Consider the linearized model
θi = −K∑
j
Li jθ j, Li j = kiδi j − Ai j
whose solution in terms of normal modes is
φi(t) =∑
j
Bi jθi(t) = φi(0)e−λit.
Bi j: matrix of eigenvectors of Li j
Suppose the adjacency matrix is symmetric.Eigenvalues of the Laplacian matrix Li j are
0 = λ1 ≤ λ2 · · · ≤ λn.
30 / 52
100time
10
100
i
15-2
0.1 11/λ
i
10
100
i
15-2
100time
10
100i
13-4
0.1 11/λ
i
10
100
i
13-4
Ref. [9]
Plateaus indicate stable community structures.31 / 52
Other dynamical clustering algorithmsopinion changing rate
xi = ωi +K∑
j bα(t)i j
∑j
bα(t)i j Ai jβ sin(x j − xi)e−β|x j−xi|
Rossler oscillator
xi = F(xi) −K∑
j bα(t)i j
∑j
bα(t)i j Li jH(xi − x j)
F = (−y − z, x + ay, b + (x − c)), H = (x, 0, 0)
α(t) = 0 or α(0) = 0, α ≤ 0
fully synchronized state + small disorder→ split into communities (as time goes)
32 / 52
3. Spinglass
33 / 52
Community detection by spinglassspin σi = community a node belongs to, ci
ground state = optimal partition
Start with an initial random condition.Minimize the energy of spinglass Potts model
H({σ}) = −∑
i j
(ai jAi j − bi j(1 − Ai j)
)δ(σi, σ j)
by usingsimulated annealing, orLouvain-like method [26].
34 / 52
Reichardt-Bornholdt model [10]
HRB = −∑
i j
(Ai jwi j − γpi j
)δ(σi, σ j), pi j =
wiw j
2w
pi j is an expectation value of wi j for a “null model”(config model in the above).
Q = −HRB,γ=1
m
Use Erdos-Reyni model as null model.
HRB−ER = −∑
i j
(Ai jwi j − γpi j
)δ(σi, σ j), pi j = p⟨w⟩
35 / 52
Arenas-Fernandes-Gomez model [21]
HAFR = −∑
i j
(Ai jwi j + rδi j − pi j(r)
)δ(σi, σ j)
pi j(r) =(wi + r)(w j + r)
2w + nr
A self-loop with weight r is added to each node.
Ronhovde-Nussinov model [26]
HRN = −∑
i j
(Ai jwi j − γ(1 − Ai j)
)δ(σi, σ j)
No global parameter is included.The weights of missing links are supposed to be γ.
36 / 52
Label propagation algorithm (LPA)
HLPA = −∑
i j
Ai jδ(σi, σ j)
Finding local minima by asynchronous localoptimization is equivalent to LPA. [27]
Ref. [27]
The global minimum is ferromagnetic.
37 / 52
Minimizing HamiltonianH is costly to evaluate.
∆HRB(σi = σr → σs)
=∑j,i
(wi j − γ
wiw j
2w
)δ(r, σ j) −
∑j,i
(wi j − γ
wiw j
2w
)δ(s, σ j)
=∑j,i
wi j(δ(r, σ j) − δ(s, σ j)) −γwi
2w(wr − wi − ws)
To calculate ∆H only the followings are necessary:states of neighborssome global bookkeeping (wr in the above)
38 / 52
...1 initialize: start from a state wherein each nodeforms its own community
...2 optimization: iterate until convergence...1 iterate until convergence
...1 sequentially pickup each node
...2 calculate the energy change as if it weremoved to neighboring community
...3 assign the node to the community with thelowest energy
...2 (node level) replace communities by supernode(super node level) return to node-level
An another way is to start with randomly assigned qcommunities, and try several initial conditions.
Optionally use a new random sequential order eachtime.
39 / 52
MultiresolutionFine structure is detected at large γCoarse-grained structure is detected at small γ
How to select the best resolution(s)?
1 10 100r - r
asymp
1
10
100
mod
ules
H 13-4
(I)
(II)
r = 0
Ref. [21] Ref. [21]
Plateaus indicate stable community structures.40 / 52
10-1 100 101
0
1
2
3
4
RBCMAPM
V(b)
APM
Hierarchy Level 2: APM (t = 1 trial) APM (t = 4) RBCM (t = 1) RBCM (t = 4)
RBCM
(a)
10-1 100 101
0
1
2
3
4Hierarchy Level 3:
APM (t = 1 trial) APM (t = 4) RBCM (t = 1) RBCM (t = 4)
V
Ref. [26]
41 / 52
Determine resolution by replicas [32]
Calculate S(γ) for each γ.generate r replicas by reordering the nodesoptimize replicas independentlyaverage I(A,B) over all pairs of replicas
S(γ) =2
r(r − 1)
∑(A,B)
I(A,B)
Select γ of the strongest correlation, or on plateaus.Another way is to minimize F =
∑replica H(γ) − TS(γ) directly.
42 / 52
IN
I q
V H q
10-1 100 1010.0
0.2
0.4
0.6
0.8
1.0
(iib)
(ib)
(iia)(ia)
(b)
I N
(a)
10-1 100 1010
1
2
3
4
V0
1
2
3
4
5
6
I
0
1
2
3
4
5
H
0
10
20
30
40
50
60
70
q
0
10
20
30
40
50
q
Ref. [32]
43 / 52
Resolution limit
A model which includes a global parameter tends tohave resolution limit.
HRB has resolution limit√
m/γHRN is “resolution-limit-free”
.Definition..
......
Let C = {Ci} be a H-optimal partition.H is resolution-limit-free if for each subgraphinduced by D ⊂ C, D is also H-optimal. [38]
44 / 52
4. Summary
45 / 52
Summary
We have reviewed two approaches to communitydetection
synchronizationspinglass
and the problem of resolution limit.
46 / 52
Which algorithm should we use?Any single algorithm is not preferred in all cases. [37]
Use multi-resolutional algorithms for graphs withheterogeneous/hierarchical community structure.
Other algorithms [35, review]
label propagationBayesian inferencespectral methodsMarkov clustering (MCL)clique percolation
47 / 52
Related topics
overlapping communitiesdynamical/adaptive graphmultigraphparallel computation
48 / 52
fin.
49 / 52
References I[1] M. E. J. Newman, M. Girvan, “Finding and evaluating community structure in networks,” Phys. Rev. E 69, 026113
(2004) [cond-mat/0308127].[2] M. E. J. Newman, “Finding and evaluating community structure in networks,” Phys. Rev. E 69, 066133 (2004)
[cond-mat/0309508].[3] J. Reichardt, S. Bornholdt, “Detecting fuzzy community structures in complex networks with a Potts model,” Phys. Rev.
Lett. 93, 218701 (2004) [cond-mat/0402349].[4] A. Clauset, M. E. J. Newman, C. Moore, “Finding community structure in very large networks,” Phys. Rev. E 70, 066111
(2004) [cond-mat/0408187].[5] A. Pluchino, V. Latora, A. RapisardaChanging, “Opinions in a Changing World: a New Perspective in Sociophysics,” Int.
J. Mod. Phys. C 16 515 (2005) [cond-mat/0410217].[6] L. Danon, J. Duch, A. Diaz-Guilera, A. Arenas, “Comparing community structure identification,” J. Stat. Mech. P09008
(2005) [cond-mat/0505245].[7] S. H. Yook, H. M. Ortmanns, “Synchronization of Rossler Oscillators on Scale-free Topologies,” Physica A 371, 781
(2006) [cond-mat/0507422].[8] S. W. Son, H. Jeong, J. D. Noh, “Random field Ising model and community structure in complex networks,” Eur. Phys. J.
B 50, 431 (2006) [cond-math/0502672].[9] A. Arenas, A. D. Guilera, C. J. P. Vicente, “Synchronization reveals topological scales in complex networks,” Phys. Rev.
Lett. 96, 114102 (2006) [cond-mat/0511730].[10] J. Reichardt, S. Bornholdt, “Statistical Mechanics of Community Detection,” Phys. Rev. E 74, 016110 (2006)
[cond-math/0603718].[11] S. Fortunato, M. Barthelemy, “Resolution limit in community detection,” Proc. Natl. Acad. Sci. 104, 36 (2007)
[physics/0607100].[12] S. Boccaletti, M. Ivanchenko, V. Latora, A. Pluchino, A. Rapisarda, “Detection of Complex Networks Modularity by
Dynamical Clustering,” Phys. Rev. E 75, 045102 (2007) [physics/0607179].[13] A. Pluchino, S. Boccaletti, V. Latora, A. Rapisarda, “Opinion dynamics and synchronization in a network of scientific
collaborations,” Physica A 372, 316 (2006) [physics/0607210].
50 / 52
References II[14] J. G. ardenes, Y. Moreno, A. Arenas, “Paths to Synchronization on Complex Networks,” Phys. Rev. Lett. 98, 034101
(2007) [cond-mat/0608314].[15] A. Arenas, A. D. Guilera, C. J. P. Vicente, “Synchronization processes in complex networks,” Physica D 224, 27 (2006)
[nlin/0610057].[16] J. M. Kumpula, J. Saramaki, K. Kaski, J. Kertesz, “Limited resolution in complex network community detection with
Potts model approach,” Eur. Phys. J. B 56, 41 (2007) [cond-mat/0610370].[17] A. Arenas, A. D. Guilera, “Synchronization and modularity in complex networks,” Eur. Phys. J. ST 143, 19 (2007)
[cond-mat/0610726].[18] M. Mosvall, C. T. Bergstrom, “An information-theoretic framework for resolving community structure in complex
networks,” Proc. Natl. Acad. Sci. 104, 7327 (2007) [physics/0612035].[19] C. Zhou, J. Kurths, “Hierarchical synchronization in complex networks with heterogeneous degress,” Chaos 16, 015104
(2006).[20] K. Wakita, T. Tsurumi, “Finding Community Structure in Mega-scale Social Networks,” [cs/0702048].[21] A. Arenas, A. Fernandez, S. Gomez, “Analysis of the structure of complex networks at different resolution levels,” New
J. Phys. 10, 053039 (2008) [physics/0703218].[22] M. Rosvall, C. T. Bergstrom, “Maps of random walks on complex networks reveal community structure,” Proc. Natl.
Acad. Sci. 105, 1118 (2008) [0707.0609].[23] U. N. Raghavan, R. Albert, S. Kumara, “Near linear time algorithm to detect community structures in large-scale
networks,” Phys. Rev. E 76, 036106 (2007) [0709.2938].[24] A. Pluchino, V. Latora, A. Rapisarda, S. Boccaletti, “Modules identification by a Dynamical Clustering algorithm based
on chaotic Rossler oscillators,” API Conf. Proc. 965, 323 (2007) [0711.1778].[25] V. D. Blondel, J. L. Guillaume, R. Lambiotte, E. Lefebvre, “Fast unfolding of communities in large networks,” J. Stat.
Mech. P10008 (2008) [0803.0476].[26] P. Ronhovde, Z. Nussinov, “Local resolution-limit-free Potts model for community detection,” Phys. Rev. E 81, 046114
(2010) [0803.2548].[27] G. Tibely, J. Kertesz, “Note on the equivalence of the label propagation method of community detection and a Potts
model approach,” Physica A 387, 4982 (2008) [0803.2804].
51 / 52
References III[28] A. Arenas, A. D. Guilera, J. Kurths, Y. Moreno, C. Zhou, “Synchronization in complex networks,” Phys. Rep. 469, 93
(2008) [0805.2976].[29] A. Lancichinetti, S. Fortunato, F. Radicchi, “Benchmark graphs for testing community detection algorithms,” Phys. Rev.
E 78, 046110 (2008) [0805.4770].[30] A. Pluchino, A. Rapisarda, V. Latora, “Communities recognition in the Chesapeake Bay ecosystem by dynamical
clustering algorithms based on different oscillators systems,” Proc. Intl. Workshop on Ecological Complex Systems:Stochastic Dynamics and Patterns, 22 (2007) [0806.4276].
[31] I. X. Y. Leung, P Hui, P Lio’, J Crowcroft, “Towards real-time community detection in large networks,” Phys. Rev. E 79,066107 (2009) [0808.2633].
[32] P. Ronhovde, Z. Nussinov, “Multiresolution community detection for megascale networks by information-based replicacorrelations,” Phys. Rev. E 80, 016109 (2009) [0812.1072].
[33] M. J. Barber, J. W. Clark, “Detecting network communities by propagating labels under constraints,” Phys. Rev. E 80,026129 (2009) [0903.3138].
[34] A. Lancichinetti, S. Fortunato, “Benchmarks for testing community detection algorithms on directed and weightedgraphs with overlapping communities”, Phys. Rev. E 80, 016118 (2009) [0904.3940].
[35] S. Fortunato, “Community detection in graphs,” Phys. Rep. 486, 74 (2010) [0906.0612].[36] M. Rosvall, D. Axelsson, C. T. Bergstrom, “The map equation”, Eur. J. Phys. ST 178, 13 (2009) [0906.1405].[37] A. Lancichinetti, S. Fortunato, “Community detection algorithms: a comparative analysis,” Phys. Rev. E 80, 056117
(2009) [0908.1062].[38] V. A. Traag, P. V. Dooren, Y. Nesterov, “Narrow scope for resolution-limit-free community detection”, Phys. Rev. E 84,
016114 (2011) [1104.3083].[39] L. Subelj, M. Bajec, “Unfolding communities in large complex networks: Combining defensive and offensive label
propagation for core extraction,” Phys. Rev. E 83, 036103 (2011) [1103.2593].[40] G Cordasco, L. Gargano, “Community Detection via Semi-Synchronous Label Propagation Algorithms,” Proceedings of
The International Workshop on Business Applications of Social Network Analysis (2010) [1103.4550].[41] G. Orman, V. Labatut, H. Cherifi, “Qualitative Comparison of Community Detection Algorithms,” Communcations in
Computer and Information Science 167, 265 (2011) [1207.3603].
Revision: a863de0 (2012-09-13)
52 / 52