Presentation ENIA 2011: Semisupervised Learning in Complex Networks

8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks

1/32

Thiago Christiano Silva

Liang Zhao

Roberto Alves Gueleri

Institute of Mathematics and Computer Science

University of So Paulo, So Carlos, So Paulo, Brazil


2/32

Summary Introduction

Complex Networks

Communities

Competitive Learning

Prior Related Work

Proposed Technique Description of the Technique

Mathematical Analysis of the Model

Time Complexity Analysis of the Model

Parameter Sensitivity Analysis

Computer Simulations Artificial Data Sets

Real-world Data Sets

Conclusions

Future Works

2


3/32

Complex Networks Interest shifting away from small to large-scale networks

Ubiquitous in Nature and everyday life

E.g.: WWW, Biological Neural Networks, Social Networks, Food

Webs, Metabolic Networks, Electrical Energy

Complex network representation unifies the structure,

dynamics and functions of a system which it represents

Inherent ability to describe the topological structure of the

original system

Presence of structures known as communities

- S. Fortunato, Community detection in graphs, Physics Reports 486 (3-5) (2010) 75174.

- M. Newman, The structure and function of complex networks, SIAM Review, vol. 45, no. 2, pp. 167256,

2003

- G. Palla, I. Dernyi, I. Farkas, and T. Vicsek, Uncovering the overlapping community structure of complex

networks in nature and society, Nature, vol. 435, pp. 814818, 2005. 3


4/32

Communitiesy A group of vertices with high concentrations of edges

within this group, and low concentrations between

different groups

4

- M. Girvan, M.E.J. Newman, Community structure in social and biological networks, Proc. Natl. Acad. Sci. USA 99 (12)

(2002) 78217826

- A. Lancichinetti, S. Fortunato, F. Radicchi, Benchmark graphs for testing community detection algorithms, Phys. Rev. E78 (4) (2008) 046110


5/32

Competitive Learningy Observed in nature and in many social systems sharing limited

resources

y Water, food, mates, territory, recognition, etc.

y Important field of Machine Learning

y Widely implemented in neural networks

y Several real-world applications

y Early works include:

y Self-organizing maps (SOM)

y Differential Competitive Learning

y Adaptive Resonance Theory (ART)

5

- T. Kohonen, The self-organizing map, Proceedings of the IEEE, vol. 78, no. 9, pp. 14641480, 1990.

- B. Kosko, Stochastic competitive learning, IEEE Trans. Neural Networks, vol. 2, no. 5, pp. 522529, 1991.

- S. Grossberg, Competitive learning: From interactive activation to adaptive resonance, Cognitive Science,

vol. 11, pp. 2363, 1987.


6/32

Prior Related Worky Originally proposed by Quiles et Al. in the unsupervised

learning approach

y Several particles walk in the network and compete with

each other to mark their own territory, while attempting

to reject intruder particles

y Each particle can perform:

y Random Walk

y Deterministic Walk

y Only a procedure of particle competition is introduced

without formal definition

y Only applied to community detection tasks

6

- M. G. Quiles, L. Zhao, R. L. Alonso, and R. A. F. Romero, Particle competition for complex network community

detection, Chaos, vol. 18, no. 3, p. 033107, 2008.


7/32

Contributions of the Proposed Techniquey A new type of competitive learning mechanism inspired

by the work in Quiles et Al.

y Unlike the original model, this is applied to the semi-

supervised learning approachy A particle-cooperative mechanism is introduced

y Here, the particle competition is formally represented by

a stochastic dynamical system

y We have applied the model for data classification

y The competitive walking process reaches dynamics

equilibrium when each class is dominated by a single or

team of particles

7


8/32

Description of the Techniquey

8


9/32

Notationy

9


10/32


11/32

Particles Movement Policyy

11

Active Term Exhausted

Term


12/32

Description of Each Movement Termy

12

RANDOM TERM

y Adventurous Behavior

y Does not take into account

the dominated vertices

PREFERENTIAL TERM

y Defensive Behavior

y Prefers visiting vertices with

high domination levels


13/32

13

Description of Each Movement Term

13


14/32

Stochastic Dynamical System

y

14

OBS.:One can see that the proposed dynamical system is Markovian, since it

only depends on the present state to completely define the immediate future state


15/32

Initial Conditionsy

15

y All particles start with the same energy level given by:

y All particles start in the active mode:


16/32

16


17/32

Mathematical Analysisy

17


18/32

18

2

4

3

113

14

176

5

7

8

9

15

16

18

19

20

10

11

12


19/32

Time Complexity Analysisy

19


20/32

20- L. Danon, A. Daz-Guilera, J. Duch, and A. Arenas, Comparing community structure identification, J. Stat. Mech., p.P09008, 2005.


21/32

Parameter Sensitivity Analysisy

21


22/32

Parameter Sensitivity Analysisy

22


23/32

23

Computer Simulations


24/32

2424

Computer Simulations

Particles are purposefully

inserted into the worstcase scenario at the

beginning (all in one

community)


25/32

2525


26/32

Real-world data sets (Chapelles Benchmark)

26

- O. Chapelle, B. Schlkopf, and A. Zien, Eds., Semi-supervised Learning, ser. Adaptive computation and machine

learning. Cambridge, MA, USA: MIT Press, 2006.


27/32

Some competing techniques

27


28/32

28

Only 10 pre-labeled samples

All parameters have been optimized using the MATLAB Genetic

Toolbox


29/32

29

Only 100 pre-labeled samples

All parameters have been optimized using the MATLAB Genetic

Toolbox


30/32

Large-scale multi-class data sety

30


31/32

Conclusionsy We have proposed a semi-supervised technique based on

competitive learning

y A rigorous definition has been provided using a nonlinear

stochastic dynamical system

y A mathematical analysis has been carried out

y A parameter sensitivity analysis has been conducted

y Computer simulations have been performed and

satisfactory results have been obtained

y More importantly, this work is an attempt to provide an

alternative way to the study of competitive learning

31


32/32

Future Worksy Study the overlapping characteristics of the vertices or

sub-graphs in the network

y Tackle the problem of data classification reliability

y Usually, some pre-labeled samples are mislabeled

y We will study this case and will attempt to avoid the

propagation of these false labelsy This is an important issue, since the task of labeling involves

human efforts: hence, susceptible for errors

32

Presentation ENIA 2011: Semisupervised Learning in Complex Networks

Documents

Transcript of Presentation ENIA 2011: Semisupervised Learning in Complex Networks