Physics Inspired Approachesto Community Detection

2012-09

1 / 52

Abstract

Community structure is one of the most relevantfeatures of graphs in sociology, biology, computerscience and so on.

In this talk, we review the following methods forcommunity detection:

synchronizationspinglass

2 / 52

Outline

...1 Introduction

...2 Synchronization

...3 Spinglass

...4 Summary

3 / 52

1. Introduction

4 / 52

Communities in real-world networks

biological networkprotein interaction, gene regulatory, metabolic, food chainsocial networkSNS, collaborators, phone/email, organizationtechnical systemweb graph, Internet, power gridother networkcitation, e-commerce/bidding, stock returnsdynamical phenomenaepidemic, cascade, synchronization, opinion change

5 / 52

NotationG = (V,E) graph(network)

vi ∈ V vertex(node), n = |V|(i, j) ∈ E edge(link) from vi to v j, 2m =

∑i j Ai j

Ai j adjacency matrix

Ai j =

1 (i, j) ∈ E0 otherwise

ki degree of vikout

i =∑

j Ai j, kinj =

∑i Ai j

wi j ≥ 0 weight of (i, j)wout

i =∑

j wi j, winj =

∑i wi j, 2w =

∑i j wi j

cs ∈ C community, q = |C|6 / 52

What is community detection

Communities are subgraphswithin which connections are dense, andbetween which they are sparse.

The concept of communityis not rigorously defined, andincludes some degree of arbitrariness.

Finding an exact solution is NP-hard in most cases.→ Thus an approximation algorithm is needed.

7 / 52

In this talk, we focus onnon-overlapping communitiesnon-dynamical graphssparse graphs: O(m) = O(n)

8 / 52

Girvan-Newman algorithm [1]

hierarchical divisive algorithmiteratively remove an edge with the highestbetweeness, and recalculate betweenessO(m2n)(for shortest-path betweeness version)

Alternative definitions of betweeness...1 shortest-path betweeness...2 flow betweeness...3 random-walk betweeness

9 / 52

Ref. [1]Ref. [1]

10 / 52

ModularityMany algorithms assume modularity Q as a measureof goodness of a partition. [2]

Q =1

2w

∑i j

wi j −win

i woutj

2w

δ(ci, c j)

=

q∑s=1

(wss

w− win

s wouts

4w2

)2wss =

∑i j wi jδ(ci, cs)δ(c j, cs), ws =

∑i wi jδ(ci, cs)

ci: community to which vi belongs

1st term: weight of within-community edges2nd term: expectation value of it for randomized graph

11 / 52

Greedy modularity optimization

hierarchical agglomerative algorithmiteratively merge communities to produce thelargest possible increase of modularityO((m + n)n) [2, Newman]

O(md log n), d = (depth of dendrogram) ∼ log nwith use of max-heap [4, Caluset-Newman-Moore]

Improvement of merging strategy [20].

12 / 52

Louvain method [25]

...1 assign its own community to each node

...2 iterate until no change happen...1 locally optimize each order in sequential

order until no change happen...2 replace communities by supernodes

Alternatively start with randomly assigned q < Ncommunities, and try several initial conditions.

A new random sequential order can be used eachtime.

13 / 52

Ref. [25]

14 / 52

Resolution limit of modularityModularity optimization may fail to identifycommunities smaller than an intrinsic sale

√m

measured by the number of links. [11]

mK

mK

mK

mK

mK

mKmK

mK

mKmK

A

pK

mK mK

pK

B

Ref. [11]

15 / 52

Infomap/map equation [22, 36]

Compress information of a random-walk trajectoryby Huffman code.

pi: visit frequency with teleportation ratio τqout

i : exit probability for cs

qexits = τ

n − ns

n

∑i

pi + (1 − τ)∑

i jci=cs, c j,cs

piAi j

16 / 52

✵✵✵✵

✵✵✶

✶✶

✶✵✵

✵✶

✶✵✶

✶✶✵✵✶✶

✵✵

✶✶✶

✶✵✶✵ ✶✵✵

✵✶✵

✵✶

✶✵

✵✵✶

✶✶✵✶✶

✵✵✵✵

✵✶✵

✶✶✵✶

✶✵

✵✵✶✶✶✶

✶✶✵✵

✵✶✵✶✶✵✶✶✶

✵✵✶✶✶✶✶✵✶✵✵✵✶✵✵✶✶✶✵✶✶✶✵✶✵✶✵✶✶✶✵✶✶✵✶✵✵✵✶✶✶✵✶✶✶✵✵✵✵✵✶✵✵✵✵✶✶✵✶✶✵✶✶✵✵✵✵✶✵✵✵✶✵✵✵✵✶✶✶✵✵✶✵✵✶✵✵✵

▲✭▼✮ ❂ �✳✁✸ ✰ ✷✳✾✼ ❂ ✸✳�✾ ❜✐ts

�✳✺ ✂

✁✳� ✂

✁✳✺ ✂

✷✳� ✂

Ref. [22]

17 / 52

Minimize the description length by Louvain method:

L =

∑s

qexits

H +∑

s

∑ici=cs

pi + qexits

Hs.

H is the entropy of a community-index codebook

H = −∑

s

qexits∑

r qexitr

log(

qs∑r qexit

r

).

Hs is the entropy of a within-community codebook

Hs = −qs

qexitr +

∑i

ci=cspi

log(

qs

qexitr +

∑i,ci=cs

pi

)

−∑

i

pi

qexiti +

∑i

ci=csp j

log

pi

qexiti +

∑i

ci=csp j

.18 / 52

Comparative EvaluationSynthetic graph for benchmark

degree distributioncommunity size distributiondensity radio of within/inter-community edgeshierarchical community structure

Similarity between clustering X and Y is measuredby normalized mutual information.

I(X,Y) =2∑

r,sNrsN log NrsN

Nr∗N∗s∑r

Nr∗N log Nr∗

N +∑

sN∗sN log N∗s

N

Nrs: #(node) assigned to a community r by X, and s by YN∗s =

∑r Nrs, Nr∗ =

∑s Nrs

19 / 52

Ref. [29]Ref. [26]

20 / 52

2. Synchronization

21 / 52

Kuramoto Model

θi = ωi +Kn

∑j

sin(θ j − θi)

ωi is distributed according to g(ω) with zero mean.Assume g(ω) is unimodal and g(ω) = g(−ω).

A mean-field order parameter defined by

reiϕ =1n

∑j eıθ j

yieldsθi = ωi + Kr sin(ϕ − θi).

22 / 52

The solutions exhibit two types of long-termbehavior. |ωi| ≤ Kr locked

|ωi| > Kr drift

Demanding the drifting oscillators form a stationarydistribution leads to a self-consistent equation:

r = Kr∫ π/2

−π/2cos2 θg(Kr sinθ).

A non-trivial solution (r > 0) is admitted beyond

Kc =2πg(0)

.

23 / 52

0

1

rinf

Kc K

Ref. [5]

24 / 52

Synchronization on network [28, review]

θi = ωi + K∑

j

Ai j sin(θ j − θi)

alternative choices of the coupling constant:

Ai j →Ai j

N,

Ai j

ki,

Ai j

⟨k⟩ .

For simplicity we focus on undirected & unweightedgraphs, and ki ≫ 1.

25 / 52

A local order parameter defined by

rieıϕi =∑

j Ai j⟨eıθ j⟩t

yields

θi = ωi − Kri sin(θi − ϕi) − Khi(t)

hu(t) = ℑ{e−ıθi

∑j ai j

(⟨eıθ j⟩t − eıθ j

)}.

hi is ignorable. Look for stationary solutions.

ri =∑|ω j|≤σr j

Ai jeı(θ j−ϕ j) +∑|ω j|>σr j

Ai j⟨eı(θ j−ϕ j)⟩t.

26 / 52

Suppose (ri, ϕi) is statistically independent of ωi.Ignore the 2nd term to get

ri =∑|ω j|≤σr j

Ai j cos(ϕ j − ϕi)

√1 −

(ω j

Kr j

)2

.

A critical coupling is obtained when cos(ϕ j − ϕi) = 1.Continuum approximation leads to

ri = K∑

j

Ai jr j

∫ 1

−1dxg(Kr jx)

√1 − x2

→ Kc =2πg(0)

1(largest eigenvalue of A)

.

27 / 52

Dynamical clustering [9]Suppose oscillators are identical ∀i, ωi = ω.→ Then full synchronization ∀i, θi = θ is possible.

A dense (sparse) subgraph synchronizerapidly (slowly).

start from random initial conditions Icalculate a local order parameter

ρi j(t) = ⟨cos(θi(t) − θ j(t))⟩Iand dynamical connectivity matrix

Dt(T)i j =

1 ρi j(t) > T (T: threshold)0 otherwise

.

28 / 52

Dt(T)i j gives a community structure.

Ref. [9]

29 / 52

Consider the linearized model

θi = −K∑

j

Li jθ j, Li j = kiδi j − Ai j

whose solution in terms of normal modes is

φi(t) =∑

j

Bi jθi(t) = φi(0)e−λit.

Bi j: matrix of eigenvectors of Li j

Suppose the adjacency matrix is symmetric.Eigenvalues of the Laplacian matrix Li j are

0 = λ1 ≤ λ2 · · · ≤ λn.

30 / 52

100time

10

100

i

15-2

0.1 11/λ

i

10

100

i

15-2

100time

10

100i

13-4

0.1 11/λ

i

10

100

i

13-4

Ref. [9]

Plateaus indicate stable community structures.31 / 52

Other dynamical clustering algorithmsopinion changing rate

xi = ωi +K∑

j bα(t)i j

∑j

bα(t)i j Ai jβ sin(x j − xi)e−β|x j−xi|

Rossler oscillator

xi = F(xi) −K∑

j bα(t)i j

∑j

bα(t)i j Li jH(xi − x j)

F = (−y − z, x + ay, b + (x − c)), H = (x, 0, 0)

α(t) = 0 or α(0) = 0, α ≤ 0

fully synchronized state + small disorder→ split into communities (as time goes)

32 / 52

3. Spinglass

33 / 52

Community detection by spinglassspin σi = community a node belongs to, ci

ground state = optimal partition

Start with an initial random condition.Minimize the energy of spinglass Potts model

H({σ}) = −∑

i j

(ai jAi j − bi j(1 − Ai j)

)δ(σi, σ j)

by usingsimulated annealing, orLouvain-like method [26].

34 / 52

Reichardt-Bornholdt model [10]

HRB = −∑

i j

(Ai jwi j − γpi j

)δ(σi, σ j), pi j =

wiw j

2w

pi j is an expectation value of wi j for a “null model”(config model in the above).

Q = −HRB,γ=1

m

Use Erdos-Reyni model as null model.

HRB−ER = −∑

i j

(Ai jwi j − γpi j

)δ(σi, σ j), pi j = p⟨w⟩

35 / 52

Arenas-Fernandes-Gomez model [21]

HAFR = −∑

i j

(Ai jwi j + rδi j − pi j(r)

)δ(σi, σ j)

pi j(r) =(wi + r)(w j + r)

2w + nr

A self-loop with weight r is added to each node.

Ronhovde-Nussinov model [26]

HRN = −∑

i j

(Ai jwi j − γ(1 − Ai j)

)δ(σi, σ j)

No global parameter is included.The weights of missing links are supposed to be γ.

36 / 52

Label propagation algorithm (LPA)

HLPA = −∑

i j

Ai jδ(σi, σ j)

Finding local minima by asynchronous localoptimization is equivalent to LPA. [27]

Ref. [27]

The global minimum is ferromagnetic.

37 / 52

Minimizing HamiltonianH is costly to evaluate.

∆HRB(σi = σr → σs)

=∑j,i

(wi j − γ

wiw j

2w

)δ(r, σ j) −

∑j,i

(wi j − γ

wiw j

2w

)δ(s, σ j)

=∑j,i

wi j(δ(r, σ j) − δ(s, σ j)) −γwi

2w(wr − wi − ws)

To calculate ∆H only the followings are necessary:states of neighborssome global bookkeeping (wr in the above)

38 / 52

...1 initialize: start from a state wherein each nodeforms its own community

...2 optimization: iterate until convergence...1 iterate until convergence

...1 sequentially pickup each node

...2 calculate the energy change as if it weremoved to neighboring community

...3 assign the node to the community with thelowest energy

...2 (node level) replace communities by supernode(super node level) return to node-level

An another way is to start with randomly assigned qcommunities, and try several initial conditions.

Optionally use a new random sequential order eachtime.

39 / 52

MultiresolutionFine structure is detected at large γCoarse-grained structure is detected at small γ

How to select the best resolution(s)?

1 10 100r - r

asymp

1

10

100

mod

ules

H 13-4

(I)

(II)

r = 0

Ref. [21] Ref. [21]

Plateaus indicate stable community structures.40 / 52

10-1 100 101

0

1

2

3

4

RBCMAPM

V(b)

APM

Hierarchy Level 2: APM (t = 1 trial) APM (t = 4) RBCM (t = 1) RBCM (t = 4)

RBCM

(a)

10-1 100 101

0

1

2

3

4Hierarchy Level 3:

APM (t = 1 trial) APM (t = 4) RBCM (t = 1) RBCM (t = 4)

V

Ref. [26]

41 / 52

Determine resolution by replicas [32]

Calculate S(γ) for each γ.generate r replicas by reordering the nodesoptimize replicas independentlyaverage I(A,B) over all pairs of replicas

S(γ) =2

r(r − 1)

∑(A,B)

I(A,B)

Select γ of the strongest correlation, or on plateaus.Another way is to minimize F =

∑replica H(γ) − TS(γ) directly.

42 / 52

IN

I q

V H q

10-1 100 1010.0

0.2

0.4

0.6

0.8

1.0

(iib)

(ib)

(iia)(ia)

(b)

I N

(a)

10-1 100 1010

1

2

3

4

V0

1

2

3

4

5

6

I

0

1

2

3

4

5

H

0

10

20

30

40

50

60

70

q

0

10

20

30

40

50

q

Ref. [32]

43 / 52

Resolution limit

A model which includes a global parameter tends tohave resolution limit.

HRB has resolution limit√

m/γHRN is “resolution-limit-free”

.Definition..

......

Let C = {Ci} be a H-optimal partition.H is resolution-limit-free if for each subgraphinduced by D ⊂ C, D is also H-optimal. [38]

44 / 52

4. Summary

45 / 52

Summary

We have reviewed two approaches to communitydetection

synchronizationspinglass

and the problem of resolution limit.

46 / 52

Which algorithm should we use?Any single algorithm is not preferred in all cases. [37]

Use multi-resolutional algorithms for graphs withheterogeneous/hierarchical community structure.

Other algorithms [35, review]

label propagationBayesian inferencespectral methodsMarkov clustering (MCL)clique percolation

47 / 52

Related topics

overlapping communitiesdynamical/adaptive graphmultigraphparallel computation

48 / 52

fin.

49 / 52

References I[1] M. E. J. Newman, M. Girvan, “Finding and evaluating community structure in networks,” Phys. Rev. E 69, 026113

(2004) [cond-mat/0308127].[2] M. E. J. Newman, “Finding and evaluating community structure in networks,” Phys. Rev. E 69, 066133 (2004)

[cond-mat/0309508].[3] J. Reichardt, S. Bornholdt, “Detecting fuzzy community structures in complex networks with a Potts model,” Phys. Rev.

Lett. 93, 218701 (2004) [cond-mat/0402349].[4] A. Clauset, M. E. J. Newman, C. Moore, “Finding community structure in very large networks,” Phys. Rev. E 70, 066111

(2004) [cond-mat/0408187].[5] A. Pluchino, V. Latora, A. RapisardaChanging, “Opinions in a Changing World: a New Perspective in Sociophysics,” Int.

J. Mod. Phys. C 16 515 (2005) [cond-mat/0410217].[6] L. Danon, J. Duch, A. Diaz-Guilera, A. Arenas, “Comparing community structure identification,” J. Stat. Mech. P09008

(2005) [cond-mat/0505245].[7] S. H. Yook, H. M. Ortmanns, “Synchronization of Rossler Oscillators on Scale-free Topologies,” Physica A 371, 781

(2006) [cond-mat/0507422].[8] S. W. Son, H. Jeong, J. D. Noh, “Random field Ising model and community structure in complex networks,” Eur. Phys. J.

B 50, 431 (2006) [cond-math/0502672].[9] A. Arenas, A. D. Guilera, C. J. P. Vicente, “Synchronization reveals topological scales in complex networks,” Phys. Rev.

Lett. 96, 114102 (2006) [cond-mat/0511730].[10] J. Reichardt, S. Bornholdt, “Statistical Mechanics of Community Detection,” Phys. Rev. E 74, 016110 (2006)

[cond-math/0603718].[11] S. Fortunato, M. Barthelemy, “Resolution limit in community detection,” Proc. Natl. Acad. Sci. 104, 36 (2007)

[physics/0607100].[12] S. Boccaletti, M. Ivanchenko, V. Latora, A. Pluchino, A. Rapisarda, “Detection of Complex Networks Modularity by

Dynamical Clustering,” Phys. Rev. E 75, 045102 (2007) [physics/0607179].[13] A. Pluchino, S. Boccaletti, V. Latora, A. Rapisarda, “Opinion dynamics and synchronization in a network of scientific

collaborations,” Physica A 372, 316 (2006) [physics/0607210].

50 / 52

References II[14] J. G. ardenes, Y. Moreno, A. Arenas, “Paths to Synchronization on Complex Networks,” Phys. Rev. Lett. 98, 034101

(2007) [cond-mat/0608314].[15] A. Arenas, A. D. Guilera, C. J. P. Vicente, “Synchronization processes in complex networks,” Physica D 224, 27 (2006)

[nlin/0610057].[16] J. M. Kumpula, J. Saramaki, K. Kaski, J. Kertesz, “Limited resolution in complex network community detection with

Potts model approach,” Eur. Phys. J. B 56, 41 (2007) [cond-mat/0610370].[17] A. Arenas, A. D. Guilera, “Synchronization and modularity in complex networks,” Eur. Phys. J. ST 143, 19 (2007)

[cond-mat/0610726].[18] M. Mosvall, C. T. Bergstrom, “An information-theoretic framework for resolving community structure in complex

networks,” Proc. Natl. Acad. Sci. 104, 7327 (2007) [physics/0612035].[19] C. Zhou, J. Kurths, “Hierarchical synchronization in complex networks with heterogeneous degress,” Chaos 16, 015104

(2006).[20] K. Wakita, T. Tsurumi, “Finding Community Structure in Mega-scale Social Networks,” [cs/0702048].[21] A. Arenas, A. Fernandez, S. Gomez, “Analysis of the structure of complex networks at different resolution levels,” New

J. Phys. 10, 053039 (2008) [physics/0703218].[22] M. Rosvall, C. T. Bergstrom, “Maps of random walks on complex networks reveal community structure,” Proc. Natl.

Acad. Sci. 105, 1118 (2008) [0707.0609].[23] U. N. Raghavan, R. Albert, S. Kumara, “Near linear time algorithm to detect community structures in large-scale

networks,” Phys. Rev. E 76, 036106 (2007) [0709.2938].[24] A. Pluchino, V. Latora, A. Rapisarda, S. Boccaletti, “Modules identification by a Dynamical Clustering algorithm based

on chaotic Rossler oscillators,” API Conf. Proc. 965, 323 (2007) [0711.1778].[25] V. D. Blondel, J. L. Guillaume, R. Lambiotte, E. Lefebvre, “Fast unfolding of communities in large networks,” J. Stat.

Mech. P10008 (2008) [0803.0476].[26] P. Ronhovde, Z. Nussinov, “Local resolution-limit-free Potts model for community detection,” Phys. Rev. E 81, 046114

(2010) [0803.2548].[27] G. Tibely, J. Kertesz, “Note on the equivalence of the label propagation method of community detection and a Potts

model approach,” Physica A 387, 4982 (2008) [0803.2804].

51 / 52

References III[28] A. Arenas, A. D. Guilera, J. Kurths, Y. Moreno, C. Zhou, “Synchronization in complex networks,” Phys. Rep. 469, 93

(2008) [0805.2976].[29] A. Lancichinetti, S. Fortunato, F. Radicchi, “Benchmark graphs for testing community detection algorithms,” Phys. Rev.

E 78, 046110 (2008) [0805.4770].[30] A. Pluchino, A. Rapisarda, V. Latora, “Communities recognition in the Chesapeake Bay ecosystem by dynamical

clustering algorithms based on different oscillators systems,” Proc. Intl. Workshop on Ecological Complex Systems:Stochastic Dynamics and Patterns, 22 (2007) [0806.4276].

[31] I. X. Y. Leung, P Hui, P Lio’, J Crowcroft, “Towards real-time community detection in large networks,” Phys. Rev. E 79,066107 (2009) [0808.2633].

[32] P. Ronhovde, Z. Nussinov, “Multiresolution community detection for megascale networks by information-based replicacorrelations,” Phys. Rev. E 80, 016109 (2009) [0812.1072].

[33] M. J. Barber, J. W. Clark, “Detecting network communities by propagating labels under constraints,” Phys. Rev. E 80,026129 (2009) [0903.3138].

[34] A. Lancichinetti, S. Fortunato, “Benchmarks for testing community detection algorithms on directed and weightedgraphs with overlapping communities”, Phys. Rev. E 80, 016118 (2009) [0904.3940].

[35] S. Fortunato, “Community detection in graphs,” Phys. Rep. 486, 74 (2010) [0906.0612].[36] M. Rosvall, D. Axelsson, C. T. Bergstrom, “The map equation”, Eur. J. Phys. ST 178, 13 (2009) [0906.1405].[37] A. Lancichinetti, S. Fortunato, “Community detection algorithms: a comparative analysis,” Phys. Rev. E 80, 056117

(2009) [0908.1062].[38] V. A. Traag, P. V. Dooren, Y. Nesterov, “Narrow scope for resolution-limit-free community detection”, Phys. Rev. E 84,

016114 (2011) [1104.3083].[39] L. Subelj, M. Bajec, “Unfolding communities in large complex networks: Combining defensive and offensive label

propagation for core extraction,” Phys. Rev. E 83, 036103 (2011) [1103.2593].[40] G Cordasco, L. Gargano, “Community Detection via Semi-Synchronous Label Propagation Algorithms,” Proceedings of

The International Workshop on Business Applications of Social Network Analysis (2010) [1103.4550].[41] G. Orman, V. Labatut, H. Cherifi, “Qualitative Comparison of Community Detection Algorithms,” Communcations in

Computer and Information Science 167, 265 (2011) [1207.3603].

Revision: a863de0 (2012-09-13)

52 / 52