6 IEEE CIRCUITS AND SYSTEMS MAGAZINE 1531-636X/03/$17.002003 IEEE FIRST QUARTER 2003

In the past few years, the discovery ofsmall-world and scale-free propertiesof many natural and artificial complexnetworks has stimulated a great dealof interest in studying the underlyingorganizing principles of various com-plex networks, which has led to dra-matic advances in this emerging andactive field of research. The presentarticle reviews some basic concepts,important progress, and significantresults in the current studies of vari-ous complex networks, with emphasison the relationship between thetopology and the dynamics of suchcomplex networks. Some fundamentalproperties and typical complex net-work models are described; and, as anexample, epidemic dynamics are ana-lyzed and discussed in some detail.Finally, the important issue of robust-ness versus fragility of dynamicalsynchronization in complex networksis introduced and discussed.

Index termscomplex network, small-world network, scale-free network,synchronization, robustness

Xiao Fan Wang and Guanrong Chen

Complex Networks:

Small-World, Scale-Free and Beyond

Abstract

D

IGIT

AL

VIS

ION

, LT

D.

Feature

Introduction

Complex networks are currently being studied acrossmany fields of science [1-3]. Undoubtedly, manysystems in nature can be described by models of

complex networks, which are structures consisting ofnodes or vertices connected by links or edges. Examplesare numerous. The Internet is a network of routers ordomains. The World Wide Web (WWW) is a network ofwebsites (Fig. 1). The brain is a network of neu-rons. An organization is a network of people.The global economy is a network of nationaleconomies, which are themselves networks ofmarkets; and markets are themselves networksof interacting producers and consumers. Foodwebs and metabolic pathways can all be repre-sented by networks, as can the relationshipsamong words in a language, topics in a conver-sation, and even strategies for solving a mathe-matical problem. Moreover, diseases aretransmitted through social networks; and com-puter viruses occasionally spread through theInternet. Energy is distributed through trans-portation networks, both in living organisms,man-made infrastructures, and in many physicalsystems such as the power grids. Figures 2-4 areartistic drawings that help visualize the com-plexities of some typical real-world networks.

The ubiquity of complex networks in sci-ence and technology has naturally led to a setof common and important research problemsconcerning how the network structure facili-tates and constrains the network dynamicalbehaviors, which have largely been neglectedin the studies of traditional disciplines. Forexample, how do social networks mediate thetransmission of a disease? How do cascadingfailures propagate throughout a large powertransmission grid or a global financial network?What is the most efficient and robust architecture for aparticular organization or an artifact under a changingand uncertain environment? Problems of this kind areconfronting us everyday, problems which demandanswers and solutions.

For over a century, modeling of physical as well asnon-physical systems and processes has been performedunder an implicit assumption that the interaction pat-terns among the individuals of the underlying system orprocess can be embedded onto a regular and perhapsuniversal structure such as a Euclidean lattice. In late1950s, two mathematicians, Erds and Rnyi (ER), made

a breakthrough in the classical mathematical graph theo-ry. They described a network with complex topology by arandom graph [4]. Their work had laid a foundation of therandom network theory, followed by intensive studies inthe next 40 years and even today. Although intuitionclearly indicates that many real-life complex networks areneither completely regular nor completely random, theER random graph model was the only sensible and rigor-

ous approach that dominated scientists thinking aboutcomplex networks for nearly half of a century, due essen-tially to the absence of super-computational power anddetailed topological information about very large-scalereal-world networks.

In the past few years, the computerization of dataacquisition and the availability of high computing powerhave led to the emergence of huge databases on variousreal networks of complex topology. The public access tothe huge amount of real data has in turn stimulated greatinterest in trying to uncover the generic properties of dif-ferent kinds of complex networks. In this endeavor, two

7

Xiao Fan Wang is with the Department of Automation, Shanghai Jiao Tong University, Shanghai 200030, P. R. China. Email: xfwang@sjtu.edu.cn. Guanrong (Ron) Chen is with the Department of Electronic Engineering and director of the Centre for Chaos Control and Synchronization, CityUniversity of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong SAR, P. R. China. Email: gchen@ee.cityu.edu.hk.

FIRST QUARTER 2003 IEEE CIRCUITS AND SYSTEMS MAGAZINE

Internet

WWW

Home Page

Domain 3

Domain 2

Router

Domain 1

Figure 1. Network structures of the Internet and the WWW. On the Inter-net, nodes are routers (or domains) connected by physical links such asoptical fibers. The nodes of the WWW are webpages connected by direct-ed hyperlinks.

significant recent discoveries are the small-world effectand the scale-free feature of most complex networks.

In 1998, in order to describe the transition from a regu-lar lattice to a random graph, Watts and Strogatz (WS)introduced the concept of small-world network [5]. It isnotable that the small-world phenomenon is indeed verycommon. An interesting experience is that, oftentimes,soon after meeting a stranger, one is surprised to find thatthey have a common friend in between; so they both cheer:What a small world! An even more interesting popularmanifestation of the small-world effect is the so-calledsix degrees of separation principle, suggested by a socialpsychologist, Milgram, in the late 1960s [6]. Although thispoint remains controversial, the small-world pattern hasbeen shown to be ubiquitous in many real networks. Aprominent common feature of the ER random graph andthe WS small-world model is that the connectivity distri-bution of a network peaks at an average value and decaysexponentially. Such networks are called exponential net-works or homogeneous networks, because each nodehas about the same number of link connections.

Another significant recent discovery in the field of com-plex networks is the observation that many large-scalecomplex networks are scale-free, that is, their connectivitydistributions are in a power-law form that is independentof the network scale [7, 8]. Differing from an exponentialnetwork, a scale-free network is inhomogeneous in nature:most nodes have very few link connections and yet a fewnodes have many connections.

The discovery of the small-world effect and scale-free

feature of complex networks has led to dramaticadvances in the field of complex networks theory in thepast few years. The main purpose of this article is to pro-vide some introduction and insights into this emergingnew discipline of complex networks, with emphasis onthe relationship between the topology and dynamicalbehaviors of such complex networks.

Some Basic Concepts

Although many quantities and measures of complex net-works have been proposed and investigated in the lastdecades, three spectacular conceptsthe average pathlength, clustering coefficient, and degree distributionplay a key role in the recent study and development ofcomplex networks theory. In fact, the original attempt ofWatts and Strogatz in their work on small-world networks[5] was to construct a network model with small averagepath length as a random graph and relatively large clus-tering coefficient as a regular lattice, which evolved tobecome a new network model as it stands today. On theother hand, the discovery of scale-free networks wasbased on the observation that the degree distributions ofmany real networks have a power-law form, albeit power-law distributions have been investigated for a long time inphysics for many other systems and processes. This sec-tion provides a brief review of these important concepts.

Average Path LengthIn a network, the distance dij between two nodes, labeledi and j respectively, is defined as the number of edges

8 IEEE CIRCUITS AND SYSTEMS MAGAZINE FIRST QUARTER 2003

Large-Scale Organization

FunctionalModules

Regulatory Motifs

Genes

Information Storage Processing Execution

mRNA

Proteins Metabolites

ATP

Metabolic Pathways

Org

anis

m S

peci

ficity

Universality

Leu3

LEU1 BAT1 ILV2

ATP ATP ATPADP ADP ADP

UMP UDP UTP CTP

Mg2+ Mg2+Mg2+

Figure 2. [Courtesy of SCIENCE] A simple complexity pyra-mid composed of various molecular components of cell-

genes, RNAs, proteins, and metabolites [47]. The bottomof the pyramid shows the traditional representation of

the cells functional organization (level 1). There is aremarkable integration of various layers at both the

regulatory and the structural levels. Insights intothe logic of cellular organization can be gained

when one views the cell as an individual com-plex network in which the components are

connected by functional links. At the low-est level, these components form genet-

ic-regulatory motifs or metabolic path-ways (level 2), which in turn are the

building blocks of functional mod-ules (level 3). Finally, these mod-

ules are nested, generating ascale-free hierarchical archi-

tecture (level 4).

along the shortest path connecting them. The diameter Dof a network, therefore, is defined to be the maximal dis-tance among all distances between any pair of nodes inthe network. The average path length L of the network,then, is defined as the mean dis-tance between two nodes, aver-aged over all pairs of nodes.Here, L determines the effectivesize of a network, the mosttypical separation of one pair ofnodes therein. In a friendshipnetwork, for instance, L is theaverage number of friends exist-ing in the shortest chain con-necting two persons in thenetwork. It was an interestingdiscovery that the average pathlength of most real complex net-works is relatively small, even inthose cases where these kindsof networks have many feweredges than a typical globallycoupled network with a equalnumber of nodes. This small-ness inferred the small-worldeffect, hence the name of small-world networks.

Clustering CoefficientIn your friendship network, it isquite possible that your friendsfriend is also your direct friend;or, to put it another way, two ofyour friends are quite possiblyfriends of each other. This prop-erty refers to the clustering of thenetwork. More precisely, one candefine a clustering coefficient C asthe average fraction of pairs ofneighbors of a node that are alsoneighbors of each other. Supposethat a node i in the network has kiedges and they connect this nodeto ki other nodes. These nodesare all neighbors of node i. Clear-ly, at most ki (ki 1)/2 edges canexist among them, and thisoccurs when every neighbor ofnode i connected to every otherneighbor of node i. The clusteringcoefficient C i of node i is thendefined as the ratio between thenumber E i of edges that actually

exist among these ki nodes and the total possible numberki(ki 1)/2, namely, Ci = 2Ei/(ki(ki 1)). The clusteringcoefficient C of the whole network is the average of Ci overall i. Clearly, C 1; and C = 1 if and only if the network is

9FIRST QUARTER 2003 IEEE CIRCUITS AND SYSTEMS MAGAZINE

(a) Food Web (c) Social Network

(b) Metabolic Network (d) Java Network

Figure 3. Wiring diagrams for several complex networks. (a) Food web of the Little RockLake shows who eats whom in the lake. The nodes are functionally distinct trophicspecies. (b) The metabolic network of the yeast cell is built up of nodesthe substratesthat are connected to one another through links, which are the actual metabolic reactions.(c) A social network that visualizes the relationship among different groups of people inCanberra, Australia. (d) The software architecture for a large component of the JavaDevelopment Kit 1.2. The nodes represent different classes and a link is set if there issome relationship (use, inheritance, or composition) between two corresponding classes.

(a) (b)

Figure 4. [Courtesy of Richard V. Sole] Wiring diagrams of a digital circuit (a), and an oldtelevision circuit (b). The dots correspond to components, and the lines, wiring. Concentricrings indicate a hierarchy due to the nested modular structure of the circuits.

10 IEEE CIRCUITS AND SYSTEMS MAGAZINE FIRST QUARTER 2003

Network Size Clustering coefficient Average path length Degree exponent

Internet, domain level [13] 32711 0.24 3.56 2.1

Internet, router level [13] 228298 0.03 9.51 2.1

WWW [14] 153127 0.11 3.1 in = 2.1 out = 2.45E-mail [15] 56969 0.03 4.95 1.81

Software [16] 1376 0.06 6.39 2.5

Electronic circuits [17] 329 0.34 3.17 2.5

Language [18] 460902 0.437 2.67 2.7

Movie actors [5, 7] 225226 0.79 3.65 2.3

Math. co-authorship [19] 70975 0.59 9.50 2.5

Food web [20, 21] 154 0.15 3.40 1.13

Metabolic system [22] 778 3.2 in = out = 2.2

Table 1. Small-world pattern and scale-free property of several real networks. Each network has the number of nodes N , the clustering coeffi-cient C , the average path length L and the degree exponent of the power-law degree distribution. The WWW and metabolic networkare described by directed graphs.

is globally coupled, which means that every node in thenetwork connects to every other node. In a completelyrandom network consisting of N nodes, C 1/N , which isvery small as compared to most real networks. It has beenfound that most large-scale real networks have a tendencytoward clustering, in the sense that their clustering coeffi-cients are much greater than O(1/N), although they arestill significantly less than one (namely, far away frombeing globally connected). This, in turn, means that mostreal complex networks are not completely random. There-fore they should not be treated as completely random andfully coupled lattices alike.

Degree DistributionThe simplest and perhaps also the most important char-acteristic of a single node is its degree. The degree ki of anode i is usually defined to be the total number of its con-nections. Thus, the larger the degree, the more impor-tant the node is in a network. The average of ki over all iis called the average degree of the network, and is denot-ed by < k >. The spread of node degrees over a networkis characterized by a distribution function P(k), which isthe probability that a randomly selected node has exact-ly k edges. A regular lattice has a simple degree sequencebecause all the nodes have the same number of edges;and so a plot of the degree distribution contains a singlesharp spike (delta distribution). Any randomness in thenetwork will broaden the shape of this peak. In the limit-ing case of a completely random network, the degreesequence obeys the familiar Poisson distribution; and theshape of the Poisson distribution falls off exponentially,

away from the peak value < k >. Because of this expo-nential decline, the probability of finding a node with kedges becomes negligibly small for k >> < k >.In the pastfew years, many empirical results showed that for mostlarge-scale real networks the degree distribution deviatessignificantly from the Poisson distribution. In particular, fora number of networks, the degree distribution can be bet-ter described by a power law of the form P(k) k . Thispower-law distribution falls off more gradually than anexponential one, allowing for a few nodes of very largedegree to exist. Because these power-laws are free of anycharacteristic scale, such a network with a power-lawdegree distribution is called a scale-free network. Somestriking differences between an exponential network and ascale-free network can be seen by comparing a U.S.roadmap with an airline routing map, shown in Fig. 5.

The small-world and scale-free features are common tomany real-world complex networks. Table 1 shows someexamples that might interest the circuits and systemscommunity (for example, the discovery of the scale-freefeature of the Internet has motivated the development ofa new brand of Internet topology generators [9-12]).

Complex Network Models

Measuring some basic properties of a complex network,such as the average path length L, the clustering coeffi-cient C , and the degree distribution P(k), is the first steptoward understanding its structure. The next step, then,is to develop a mathematical model with a topology ofsimilar statistical properties, thereby obtaining a plat-form on which mathematical analysis is possible.

11FIRST QUARTER 2003 IEEE CIRCUITS AND SYSTEMS MAGAZINE

Regular Coupled NetworksIntuitively, a globally coupled network has the smallestaverage path length and the largest clustering coefficient.Although the globally coupled network model captures thesmall-world and large-clustering properties of many realnetworks, it is easy to notice its limitations: a globally cou-pled network with N nodes has N(N 1)/2 edges, whilemost large-scale real networks appear to be sparse, that is,most real networks are not fully connected and their num-ber of edges is generally of order N rather than N2.

A widely studied, sparse, and regular network model isthe nearest-neighbor coupled network (a lattice), whichis a regular graph in which every node is joined only by a

few of its neighbors. The term lattice here may suggesta two-dimensional square grid, but actually it can havevarious geometries. A minimal lattice is a simple one-dimensional structure, like a row of people holdinghands. A nearest-neighbor lattice with a periodic bound-ary condition consists of N nodes arranged in a ring,where each node i is adjacent to its neighboring nodes,i = 1, 2, , K/2, with K being an even integer. For a largeK , such a network is highly clustered; in fact, the cluster-ing coefficient of the nearest-neighbor coupled network isapproximately C = 3/4.

However, the nearest-neighbor coupled network is nota small-world network. On the contrary, its average path

Poisson Distribution Power-Law Distribution

Exponential Network Scale-Free Network

k 1 10 100 1000

0.0001

0.001

0.01

1

0.1

P(k

)

P(k

)

k

Figure 5. [Courtesy of A.-L. Barabsi] Differences between an exponential networka U.S. roadmap and a scale-free networkan air-line routing map. On the roadmap, the nodes are cities that are connected by highways. This is a fairly uniform network: each majorcity has at least one link to the highway system, and there are no cities served by hundreds of highways. The airline routing map dif-fers drastically from the roadmap. The nodes of this network are airports connected by direct flights among them. There are a fewhubs on the airline routing map, including Chicago, Dallas, Denver, Atlanta, and New York, from which flights depart to almost allother U.S. airports. The vast majority of airports are tiny, appearing as nodes with one or a few links connecting them to one or sev-eral hubs.

length is quite large and tends to infinity as N . Thismay help explain why it is difficult to achieve any dynam-ical process (e.g., synchronization) that requires globalcoordination in such a locally coupled network. Doesthere exist a regular network that is sparse and clustered,but has a small average path length? The answer is Yes. Asimple example is a star-shaped coupled network, inwhich there is a center node and each of the other N 1nodes only connect to this center but not among them-selves. For this kind of network, the average path lengthtends to 2 and its clustering coefficient tends to 1, asN . The star-shaped network model captures thesparse, clustering, small-world, as well as some otherinteresting properties of many real-world networks. There-fore, in this sense, it is better than the regular lattice as amodel of many well-known real networks. Clearly, though,most real networks do not have a precise star shape.

Random GraphsAt the opposite end of the spectrum from a completelyregular network is a network with a completely randomgraph, which was studied first by Erds and Rnyi (ER)about 40 years ago [4].

Try to imagine that you have a large number (N >> 1)of buttons scattered on the floor. With the same probabil-ity p, you tie every pair of buttons with a thread. The result

is a physical example of an ER random graph with N nodesand about pN(N 1)/2 edges (Fig. 6). The main goal ofthe random graph theory is to determine at what connec-tion probability p a particular property of a graph willmost likely arise. A remarkable discovery of this type wasthat important properties of random graphs can appearquite suddenly. For example, if you lift up a button, howmany other buttons will you pick up thereby? ER showedthat, if the probability p is greater than a certain thresholdpc (ln N)/N , then almost every random graph is con-nected, which means that you will pick up all the buttonson the floor by randomly lifting up just one button.

The average degree of the random graph is< k >= p(N 1) = pN . Let Lrand be the average pathlength of a random network. Intuitively, about < k >Lrand

nodes of the random network are at a distance Lrand orvery close to it. Hence, N < k >Lrand , which means thatLrand ln N/ < k >. This logarithmic increase in averagepath length with the size of the network is a typical small-world effect. Because ln N increases slowly with N , itallows the average path length to be quite small even in afairly large network. On the other hand, in a completelyrandom network, for example in your friendship network(say it is completely random), the probability that two ofyour friends are friends themselves is no greater than theprobability that two randomly chosen persons from yournetwork happen to be friends. Hence, the clustering coef-ficient of the ER model is C = p =< k > /N

WS Small-World Model Algorithm1) Start with order: Begin with a nearest-neigh-

bor coupled network consisting of N nodesarranged in a ring, where each node i is adjacent toits neighbor nodes, i = 1, 2, , K/2, with K beingeven.

2) Randomization: Randomly rewire each edgeof the network with probability p; varying p in sucha way that the transition between order (p = 0) andrandomness (p = 1) can be closely monitored.

Rewiring within this context means shifting one end ofthe connection to a new node chosen at random from thewhole network, with the constraints that any two differentnodes cannot have more than one connection betweenthem, and no node can have a connection with itself. Thisprocess introduces pN K/2 long-range edges, which con-nect nodes that otherwise would be part of different neigh-borhoods. Both the behaviors of the clustering coefficientC (p) and of the average path length L(p) in the WS small-world model can be considered as a function of therewiring probability p. A regular ring lattice (p = 0) is high-ly clustered (C (0) = 3/4) but has a large average pathlength (L(0) = N2K >> 1). It is found that, for a small prob-ability of rewiring, when the local properties of the networkare still nearly the same as those for the original regularnetwork, and when the clustering coefficient does not dif-fer subsequently from its initial value (C (p) C (0)), theaverage path length drops rapidly and is in the same orderas the one for random networks (L(p) >> L(0)) (Fig. 8).This result is actually quite natural. On the one hand, it issufficient to make several random rewirings to decrease

the average path length significantly. On theother hand, several rewired links cannotcrucially change the local clustering proper-ty of the network.

The small-world model can also beviewed as a homogeneous network, inwhich all nodes have approximately thesame number of edges. In this regard, theWS small-world network model is similar tothe ER random graph model. The work onWS small-world networks has started anavalanche of research on new models ofcomplex networks, including some variantsof the WS model. A typical variant was theone proposed by Newman and Watts [23],referred to as the NW small-world modellately. In the NW model, one does not breakany connection between any two nearestneighbors, but instead, adds with probabil-ity p a connection between a pair of nodes.Likewise, here one does not allow a node to

be coupled to another node more than once, or to couplewith itself. With p = 0, the NW model reduces to the origi-nal nearest-neighbor coupled network, and if p = 1 itbecomes a globally coupled network. The NW model issomewhat easier to analyze than the original WS modelbecause it does not lead to the formation of isolated clus-ters, whereas this can indeed happen in the WS model. Forsufficiently small p and sufficiently large N , the NW modelis essentially equivalent to the WS model. Today, thesetwo models are together commonly termed small-worldmodels for brevity.

The small-world models have their roots in social net-works, where most people are friends with their immedi-ate neighbors, for example neighbors on the same street

13FIRST QUARTER 2003 IEEE CIRCUITS AND SYSTEMS MAGAZINE

Rewiring of Links

P=0 0

or colleagues in the same office. On the other hand, manypeople have a few friends who are far away in distance,such as friends in other countries, which are representedby the long-range edges created by the rewiring proce-dure in the WS model, or by the connection-adding pro-cedure in the NW model.

Scale-Free ModelsA common feature of the ER random graph and the WSsmall-world models is that the connectivity distribution ofthe network is homogenous, with peak at an average valueand decay exponentially. Such networks are called expo-nential networks. A significant recent discovery in the fieldof complex networks is the observation that a number oflarge-scale complex networks, including the Internet,WWW, and metabolic networks, are scale-free and theirconnectivity distributions have a power-law form.

To explain the origin of power-law degree distribution,Barabsi and Albert (BA) proposed another networkmodel [7,8]. They argued that many existing models failto take into account two important attributes of most realnetworks. First, real networks are open and they aredynamically formed by continuous addition of new nodesto the network; but the other models are static in thesense that although edges can be added or rearranged,the number of nodes is fixed throughout the formingprocess. For example, the WWW is continually sproutingnew webpages, and the research literature constantlygrows since new papers are continuously being pub-lished. Second, both the random graph and small-worldmodels assume uniform probabilities when creating newedges, but this is not realistic either. Intuitively, webpages

that already have many links (such as the homepage ofYahoo or CNN) are more likely to acquire even more links;a new manuscript is more likely to cite a well-known andthus much-often-cited paper than many other less-knownones. This is the so-called rich get richer phenomenon,for which the other models do not account.

The BA model suggests that two main ingredients ofself-organization of a network in a scale-free structure aregrowth and preferential attachment. These point to thefacts that most networks continuously grow by the addi-tion of new nodes, and new nodes are preferentiallyattached to existing nodes with large numbers of connec-tions (again, rich get richer). The generation scheme ofa BA scale-free model is as follows:

BA Scale-Free Model Algorithm1) Growth: Start with a small number (m0) of

nodes; at every time step, a new node is introducedand is connected to m m0 already-existing nodes.

2) Preferential Attachment: The probability ithat a new node will be connected to node i (one ofthe m already-existing nodes) depends on thedegree ki of node i, in such a way that i = ki/

j kj.

After t time steps, this algorithm results in a networkwith N = t + m0 nodes and mt edges (Fig. 9). Growingaccording to these two rules, the network evolves into ascale-invariant state: The shape of the degree distributiondoes not change over time, namely, does not change dueto further increase of the network scale. The correspon-ding degree distribution is described by a power law withexponent 3, that is, the probability of finding a nodewith k edges is proportional to k3.

Numerical results have indicated that, in comparisonwith a random graph having the same size and the sameaverage degree, the average path length of the scale-freemodel is somewhat smaller, and yet the clustering coeffi-cient is much higher. This implies that the existence of afew big nodes with very large degrees (i.e., with a verylarge number of connections) plays a key role in bringingthe other nodes of the network close to each other. How-ever, there is today no analytical prediction formula forthe average path length and the clustering coefficient forthe scale-free model. The BA model is a minimal modelthat captures the mechanisms responsible for the power-law degree distribution. This model has some evident lim-itations when compared with some real-world networks.This observation has in effect spurred more research onevolving networks, with the intention to overcome limita-tions such as those of the BA model. A summary of thesemodels is given by Albert and Barabsi [2].

Recently, Milo et al. [24] defined the so-called net-work motifs as patterns of interconnections occurring in

14 IEEE CIRCUITS AND SYSTEMS MAGAZINE FIRST QUARTER 2003

Figure 9. A scale-free network of 130 nodes, generated bythe BA scale-free model. The five biggest nodes are shown inred, and they are in contact with 60% of other nodes (green).

complex networks at numbers that are significantly high-er than those in completely random networks. Suchmotifs have been found in networks ranging from bio-chemistry, neurobiology, ecology, to engineering. Thisresearch may uncover the basic building blocks pertain-ing to each class of networks.

Achilles Heel of Complex Networks

An interesting phenomenon of complex networks is theirAchilles heelrobustness versus fragility. For illustra-tion, let us start from a large and connected network. Ateach time step, remove a node (Fig. 10). The disappear-ance of the node implies the removal of all of its connec-tions, disrupting some of the paths among the remainingnodes. If there were multiple paths between two nodes iand j, the disruption of one of them may mean that thedistance dij between them will increase, which, in turn,may cause the increase of the average path length L ofthe entire network. In a more severe case, when initiallythere was a single path between i and j, the disruption ofthis particular path means that the two nodes becomedisconnected. The connectivity of a network is robust (orerror tolerant) if it contains a giant cluster comprisingmany nodes, even after a removal of a fraction of nodes.

The predecessor of the Internetthe ARPANETwas

created by the US Department of Defense, by itsAdvanced Research Projects Agency (ARPA), in the late1960s. The goal of the ARPANET was to enable continuoussupply of communications services, even in the case thatsome subnetworks and gateways were failing. Today, theInternet has grown to be a huge network and has playeda crucial role in virtually all aspects of the world. One maywonder if we can continue to maintain the functionality ofthe network under inevitable failures or frequent attacksfrom computer hackers. The good news is that by ran-domly removing certain portions of domains from theInternet, we have found that, even if more than 80% of thenodes fail, it might not cause the Internet to collapse.However, the bad news is that if a hacker targeted somekey nodes with very high connections, then he couldachieve the same effect by purposefully removing a verysmall fraction of the nodes (Fig. 11). It has been shownthat such error tolerance and attack vulnerability aregeneric properties of scale-free networks (Fig. 12) [25-28].These properties are rooted in the extremely inhomoge-neous nature of degree distributions in scale-free net-works. This attack vulnerability property is called anAchilles heel of complex networks, because the mytho-logical warrior Achilles had been magically protected inall but one small part of his bodyhis heel.

15FIRST QUARTER 2003 IEEE CIRCUITS AND SYSTEMS MAGAZINE

A A

AA

B B

BB

C C

CC

D D

DD

Remove r1

Remove r2Remove r2

Remove r1

r1

r1

r2r2

Figure 10. Illustration of the effects of node removal on aninitially connected network. In the unperturbed state,dAB = dC D = 2. After the removal of node r1 from the originalnetwork, dAB = 8. After the removal of node r2 from the orig-inal network, dC D = 7. After the removal of nodes r1 and r2,the network breaks into three isolated clusters anddAB = dC D = .

Internet1.0

0.8

0.6

0.4

0.2

0.0

S

0.0 0.2 0.4 0.6 0.8 1.0

Figure 11. The relative size S of the largest cluster in theInternet, when a fraction f of the domains are removed [25]. , random node removal; , preferential removal of themost connected nodes.

Robustto RandomFailures Fragile

to IntentionalAttacks

Figure 12. The robust, yet fragile feature of complex net-works.

Epidemic Dynamics in Complex Networks

For one specific example, the AIDS propagation networkis quite typical. When AIDS first emerged as a diseaseabout twenty years ago, few people could have predictedhow the epidemic would evolve, and even fewer couldhave been able to describe with certainty the best way offighting it. Unfortunately, according to estimates from theJoint United Nations Programme on HIV/AIDS (UNAIDS)and the World Health Organization (WHO), 21.8 millionpeople around the world had died of AIDS up to the endof 2000 and 36.1 million people were living with thehuman immunodeficiency virus (HIV) by the same time.

As another example, in spite of technological progressand great investments to ensure a secure supply of elec-tric energy, blackouts of the electric transmission grid arenot uncommon. Cascading failures in large-scale electricpower transmission systems arean important cause of the cata-strophic blackouts. Most wellknown is the cascading series offailures in power lines in August1996, leading to blackouts in 11US states and two Canadianprovinces. This incident leftabout 7 million customers with-out power for up to 16 hours, andcost billions of dollars in totaldamage. There is an urgent needfor developing innovativemethodologies and conceptualbreakthroughs for analysis, plan-ning, operation, and protection ofthe complex and dynamical elec-tric power networks. In yet anoth-er example, the ILOVEYOU computer virus spread overthe Internet in May 2000 and caused a loss of nearly 7 bil-lion dollars in facility damage and computer down-time.

How do diseases, jokes, and fashions spread out overthe social networks? How do cascading failures propagatethrough large-scale power grids? How do computer virus-es spread out through the Internet? Serious issues likethese are attracting much attention these days. Clearly,the topology of a network has great influence on the over-all behavior of an epidemic spreading in the network.Recently, some researchers have started to study suchspreading phenomena, for example on small-world andscale-free networks [29-34].

A notable attempt of Pastor-Satorras and Vespignani[31-32] was to study both analytically and numerically alarge-scale dynamical model on the spreading of epi-demics in complex networks. The standard susceptible-infected-susceptible (SIS) epidemiological model wasused for investigation. Each node of the network repre-

sents an individual, and each link is a connection alongwhich the infection can spread from one individual tosome others. It is natural to assume that each individualcan only exist in one of two discrete statessusceptibleand infected. At every time step, each susceptible node isinfected with probability if it is connected to at leastone infected node. At the same time, infected nodes arecured and become again susceptible with probability .They together define an effective spreading rate, = /.The updating can be performed with both parallel andsequential dynamics. The main prediction of the SISmodel in homogeneous networks (including lattices, ran-dom graphs, and small-world models) is the presence ofa nonzero epidemic threshold, c > 0. If c, the infec-tion spreads and becomes persistent in time; yet if < c,the infection dies out exponentially fast (Fig. 13 (a)).

It was found [31-32] that, while for exponential net-works the epidemic threshold is a positive constant, for alarge class of scale-free networks the critical spreadingrate tends to zero (Fig. 13(b)). In other words, scale-freenetworks are prone to the spreading and the persistenceof infections, regardless of the spreading rate of the epi-demic agents. It implies that computer viruses can spreadfar and wide on the Internet by infecting only a tiny frac-tion of the huge network. Fortunately, this is balanced byexponentially small prevalence and by the fact that it istrue only for a range of very small spreading rates(

waves, spiral waves, and spatiotemporal chaos. Also, thesenetworks are important in modeling many large-scale real-world systems.

In the past decade, special attention has been focusedon the synchronization of chaotic dynamical systems. Forthe same reason, many scientists have started to considerthe synchronization phenomenon in large-scale networksof coupled chaotic oscillators. These networks are usuallydescribed by systems of coupled ordinary differentialequations or maps, with completely regular topologicalstructures such as chains, grids, lattices, and globally cou-pled graphs. Two typical settings are the discrete-time cou-pled map lattice (CML) [35] and the continuous-time cellu-lar neural (or more generally, nonlinear) networks (CNN)[36]. The main advantage of these simple architectures isthat it allows one to focus on the complexity caused by thenonlinear dynamics of the nodes without worrying aboutadditional complexity in the network structure; and anoth-er appealing feature is the ease of their implementation byintegrated circuits.

The topology of a network, onthe other hand, often plays a crucialrole in determining its dynamicalbehaviors. For example, although astrong enough diffusive couplingwill result in synchronization withinan array of identical nodes [37], itcannot explain why many real-world complex networks exhibit astrong tendency toward synchro-nization even with a relatively weakcoupling. As an instance, it wasobserved that the apparently inde-pendent routing messages from dif-ferent routers in the Internet caneasily become synchronized, whilethe tendency for routers towardssynchronization may depend heavi-ly on the topology of the Internet[38]. One way to break up theunwanted synchronization is foreach router to add a (sufficientlylarge) component randomly to theperiod between two routing mes-sages. However, the tendency tosynchronization in the Internet is sostrong that changing one determin-istic protocol to correct the syn-chronization is likely to generateanother synchrony elsewhere at thesame time. This suggests that amore efficient solution requires abetter understanding of the nature

of the synchronization behavior in such complex net-works as the Internet.

Recently, synchronization in different small-world andscale-free dynamical network models has been carefullystudied [39-45]. These studies may shed new light on thesynchronization phenomenon in various real-world com-plex networks.

A Typical Dynamical Network ModelConsider a typical dynamical network consisting of Nidentical linearly and diffusively coupled nodes, witheach node being an n-dimensional dynamical system(e.g., a chaotic system). The state equations of this net-work are described by

xi = f(xi) + cN

j=1aijxj, i = 1, 2, , N. (1)

In this model, xi = (xi1, xi2, , xin)T n are the statevariables of node i, the constant c > 0 represents the cou-

17FIRST QUARTER 2003 IEEE CIRCUITS AND SYSTEMS MAGAZINE

100080

60

40

20

0

500

400

300

200

100

0

80060040020010.5p

(a) (b)

N0

2sw 2sw

Figure 14. The second-largest eigenvalue 2sw of the coupling matrix of the small-worldnetwork (1) [41]. (a) 2sw as a function of the adding probability p with the network sizeN = 500. (b) 2sw as a function of the network size with adding probability p = 0.1.

Synchronizability

Locally Coupled Small-World(a) (b)

pling strength, and nn is a constant 0 1 matrixlinking coupled variables. If there is a connection betweennode i and node j (i = j), then aij = aji = 1; otherwise,aij = aji = 0 (i = j). Moreover, aii = ki, where ki is thedegree of node i. The coupling matrix A = (aij) NNrepresents the coupling configuration of the network.

Dynamical network (1) is said to be (asymptotically)synchronized if

x1(t) = x2(t) = = xN (t) = s(t), as t , (2)

where s(t) n is a solution of an isolated node, i.e.,s(t) = f(s(t)). Here, s(t) can be an equilibrium point, aperiodic orbit, or a chaotic attractor, depending on theinterest of the study.

Consider the case that the network is connected inthe sense that there are no isolated clusters. Then, thecoupling matrix A = (aij)NN is a symmetric irreduciblematrix. In this case, it can be shown that 1 = 0 is thelargest eigenvalue of A with multiplicity 1 but all theother eigenvalues of A are strictly negative. Let 2 < 0be the second-largest eigenvalue of A. It has beenproved [40, 41] that the synchronization state (2) isexponentially stable if

c d/2

, (3)

where d < 0 is a constant determined by the dynamics ofan isolated node and the inner linking structural matrix .(In fact, d can be more precisely characterized by the Lya-punov exponents of the network [46].)

Given the dynamics of an isolated node and the innerlinking structural matrix , the synchronizability of thedynamical network (1) with respect to a specific couplingconfiguration A is said to be strong if the network can syn-chronize with a small value of the coupling strength c. Theabove result implies that the synchronizability of thedynamical network (1) can be characterized by the sec-ond-largest eigenvalue of its coupling matrix.

The second-largest eigenvalue of the coupling matrix of aglobally coupled network is N, which implies that for anygiven and fixed nonzero coupling strength c, a globally cou-pled network will synchronize as long as its size N is largeenough. On the other hand, the second-largest eigenvalue ofthe coupling matrix of a nearest-neighbor coupled networktends to zero as N , which implies that for any given andfixed nonzero coupling strength c, a nearest-neighbor couplednetwork cannot synchronize if its size N is sufficiently large.

Synchronization in Small-World NetworksConsider the dynamical network (1) with NW small-worldconnections [41]. Let 2sw be the second-largest eigenval-ue of the network coupling matrix. Figures 14 (a) and (b)

show the numerical values of 2sw as a function of theadding probability p and the network size N , respectively.It can be seen that, for any given coupling strength c > 0:(i) for any N > |d|/c, there exists a critical value p suchthat if p p 1 then the small-world network will syn-chronize; (ii) for any given p (0, 1], there exists a criticalvalue N such that if N > N then the small-world networkwill synchronize. These results imply that the ability toachieve synchronization in a large-size nearest-neighborcoupled network can be greatly enhanced by just adding atiny fraction of distant links, thereby making the networkbecome a small-world model. This reveals an advantage ofsmall-world networks for achieving synchronization, ifdesired (Fig. 15).

Synchronization in Scale-Free NetworksNow consider the dynamical network (1) with BA scale-free connections instead [42]. Figure 16 shows that thesecond-largest eigenvalue of the corresponding cou-pling matrix is very close to 1, which actually is thesecond-largest eigenvalue of the star-shaped couplednetwork. This implies that the synchronizability of a

18 IEEE CIRCUITS AND SYSTEMS MAGAZINE FIRST QUARTER 2003

2sf

0.90

0.85

0.95

1.00

200 400 600 800 1,000N

Figure 16. The second-largest eigenvalue of the couplingmatrix of the scale-free network (1), for m0 = m = 3 ();m0 = m = 5( ); and m0 = m = 7() [42].

Scale-Free Star-Shaped

Synchronizability

Hub

(a) (b)

Figure 17. Synchronizability of a scale-free network is aboutthe same as that of a star-shaped coupled network.

scale-free network is about the same as that of a star-shaped coupled network (Fig. 17). It may be due to theextremely inhomogeneous connectivity distribution ofsuch networks: a few hubs in a scale-free network playa similar (important) role as a single center in a star-shaped coupled network.

The robustness of synchronization in a scale-freedynamical network has also been investigated, againsteither random or specific removal of a small fraction f(0 < f

046119, Sept. 2001.[18] R. F. i Cancho and R. V. Sole, The small-world of human language,Proc. R. Soc. London, Ser. B, vol. 268, no. 1482, pp. 2261 - 2265, 2001[19] M. E. J. Newman, Scientific collaboration networks: I. Network con-struction and fundamental results, Phys. Rev. E, vol. 62, 016131, 2001.[20] J. M. Montoya and R. V. Sol, Small-world patterns in food webs, J.Theor. Biol. vol. 214, 405-412, 2002.[21] R. J. Williams, N. D. Martinez, E. L Berlow, J. A. Dunne and A-L.Barabasi, Two degrees of separation in complex food webs, Proc. Natl.Acad. Sci, vol 99, no. 20, 12913-12916, Oct. 2002.[22] H. Jeong, B. Tombor, R. Albert, Z. Oltvai, and A.-L. Barabsi, Thelarge-scale organization of metabolic networks, Nature, vol. 407, pp.651-653, Oct. 2000.[23] M. E. J. Newman and D. J. Watts, Renormalization group analysis ofthe small-world network model, Phys. Lett. A, vol. 263, pp. 341-346, 1999.[24] R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii and U.Alon, Network motifs: Simple building blocks of complex networks,Science, vol. 298, no. 5594, pp. 824-827, Oct. 2002.[25] R. Albert, H. Jeong and A-L. Barabsi, Attack and error tolerancein complex networks, Nature, vol. 406, pp. 387-482, July 2000.[26] D. S. Callway, M. E. J. Newman, S. H. Strogatz and D. J. Watts, Net-work robustness and fragility: Percolation on random graphs, Phys.Rev. Lett., vol. 85, no. 25, pp. 5468-5471, 2000.[27] R. Cohen, K. Erez, D. ben-Avraham and S. Havlin, Resilience of theInternet to random breakdowns, Phys. Rev. Lett., vol. 85, no. 21, pp.4626-4628, 2000.[28] R. Cohen, K. Erez, D. ben-Avraham and S. Havlin, Breakdown of theInternet under Intentional Attack, Phys. Rev. Lett., vol. 86, no. 16, pp.3682-3685, 2001.[29] Moore and M. E. J. Newman, Epidemics and percolation in small-world networks, Phys. Rev. E, vol. 61, pp. 5678-5682, 2000.[30] D. J. Watts, A simple model of fads and cascading failures, SantaFe Institute Working Paper, 00-12-062, 2000.[31] R. Pastor-Satorras and A. Vespignani, Epidemic spreading in scale-free networks, Phys. Rev. Lett., 2001, vol. 86, no. 14, 3200-3203, 2001.[32] R. Pastor-Satorras and A. Vespignani, Epidemic dynamics andendemic states in complex networks, Phys. Rev. E, vol. 63, 066117, 2001. [33] R. M. May and A. L. Lloyd, Infection dynamics on scale-free net-works, Phys. Rev. E, vol. 64, 066112, 2001.[34] Y. Moreno, R. Pastor-Satorras and A. Vespignani, Epidemic out-breaks in complex heterogeneous networks, Eur. Phys. J. B, vol 26, pp.521-529, 2002.[35] K. Kaneko (ed.), Coupled Map Lattices, World Scientific Pub. Co.,Singapore, 1992.[36] L. O. Chua, CNN: A Paradigm for Complexity, World Scientific, Sin-gapore, 1998.[37] C. W. Wu, and L. O. Chua, Synchronization in an array of linearlycoupled dynamical systems, IEEE Trans. Circuits Syst.-I, vol. 42, no. 8,pp. 430-447, Aug. 1995.[38] S. Floyd and V. Jacobson, The synchronization of periodic rout-ing messages, IEEE/ACM Trans. Networking, vol. 2, no. 2, pp. 122-136,April 1994.[39] P. M. Gade and C-K. Hu, Synchronous chaos in coupled map withsmall-world interactions, Phys. Rev. E, vol. 62, pp. 6409-6413, 2000.[40] L. F. Lago-Fernandez, R. Huerta, F. Corbacho and J. A. Siguenza,Fast response and temporal coherent oscillations in small-world net-works, Phys. Rev. Lett., vol. 84 no. 12, pp. 2758-2761, 2000.[41] X. F. Wang and G. Chen, Synchronization in small-world dynamical net-works, Int. J. Bifurcation and Chaos, vol. 12, no. 1, pp. 187-192, January 2002.[42] X. F. Wang and G. Chen, Synchronization in scale-free dynamicalnetworks: Robustness and fragility, IEEE Trans. Circuits Syst. I., vol. 49,no. 1, pp. 54-62, January 2002.[43] H. Hong, M. Y. Choi, and B. J. Kim, Synchronization on small-worldnetworks, Phys. Rev. E, vol. 65, 026139, 2002.[44] J. Jost and M. P. Joy, Spectral properties and synchronization incoupled map lattices, Phys. Rev. E, vol. 65, 016201, 2002.[45] M. Barahona and L. M. Pecora, Synchronization in small-worldsystems, Phys. Rev. Lett., vol. 89, no. 5, 054101, 2002.[46] X. Li and G. Chen, Synchronization and desynchronization ofcomplex dynamical networks: An engineering viewpoint, 2002,preprint.[47] Z. N. Oltvai and A-L Barabsi, Lifes complexity pyramid, Science,vol. 298, pp. 763-764, Oct. 2002.

Xiao Fan Wang (M00) received the B.Sc.degree from Suzhou University in 1986,the M.Sc. degree from Nanjing NormalUniversity in 1991, and the Ph.D. degreefrom Southeast University in 1996. FromOctober 1996 to August 1999 he was aPost-Doctorate Fellow and then an Asso-

ciate Professor in the Department of Automation, Nan-jing University of Scientific and Technology. FromSeptember 1999 to August 2000, he was a Research Asso-ciate in the City University of Hong Kong. From October2000 to January 2001, he was a Post-Doctorate ResearchFellow in the University of Bristol, UK. Currently he is aProfessor in the Department of Automation, ShanghaiJiao Tong University. His current research interestsinclude control and synchronization of complex dynami-cal systems and networks.

Guanrong Chen (M87, SM92, F96)received the M.Sc. degree in ComputerScience from Sun Yatsen (Zhongshan) Uni-versity and the Ph.D. degree in AppliedMathematics from Texas A&M University.Currently he is a Chair Professor and theDirector of the Centre for Chaos Control

and Synchronization at the City University of Hong Kong.He is a Fellow of the IEEE for his fundamental contributionsto the theory and applications of chaos control and bifur-cation analysis.

Prof. Chen has numerous publications since 1981 in thefields of nonlinear systems, in both dynamics and controls.Among his publications are the research monographs andedited books entitled Hopf Bifurcation Analysis: A Frequen-cy Domain Approach (World Scientific, 1996), From Chaos toOrder: Methodologies, Perspectives and Applications (WorldScientific, 1998), Controlling Chaos and Bifurcations in Engi-neering Systems (CRC Press, 1999), and Chaos in Circuitsand Systems (World Scientific, 2002).

Prof. Chen served and is serving as the Advisory Edi-tor, Features Editor, and Associate Editor for 7 interna-tional journals including the IEEE Transactions andMagazine on Circuits and Systems and the InternationalJournal of Bifurcation and Chaos. He received the 1998Harden-Simons Outstanding Prize for the Best JournalPaper Award from the American Society of EngineeringEducation, the 2001 M. Barry Carlton Best Annual Trans-actions Paper Award from the IEEE Aerospace and Elec-tronic Systems Society, and the 2002 Best Paper Awardfrom the Institute of Information Theory and Automation,Academy of Sciences of the Czech Republic. He is Hon-orary Professor of the Central Queensland University,Australia, as well as Honorary Guest-Chair Professors ofseveral universities in China.

20 IEEE CIRCUITS AND SYSTEMS MAGAZINE FIRST QUARTER 2003

Index: CCC: 0-7803-5957-7/00/$10.00 2000 IEEEccc: 0-7803-5957-7/00/$10.00 2000 IEEEcce: 0-7803-5957-7/00/$10.00 2000 IEEEindex: INDEX: ind: