Neural Networks with Anticipation: Problems and Prospects
description
Transcript of Neural Networks with Anticipation: Problems and Prospects
NEURAL NETWORKS WITH
ANTICIPATION:
PROBLEMS AND
PROSPECTS
Alexander MAKARENKO
Institute for Applied System Analysis at
National Technical University of Ukraine (KPI)
INTRODUCTION
Nonlinear networks science
(problems and effects):
stability
bifurcations
chaos
sinchronisation
turbulence
chimera states
MODELS AND SYSTEMS
(RECENTLY):coupled oscillators
coupled maps
neural networks
cellular automata
o.d.e. system
………MAINLY of NEUTRAL or
WITH DELAY
ANTICIPATION
Neural network learning (Sutton, Barto, 1982)
Control theory (Pyragas, 2000?)
Neuroscience (1970-1980,… , 2009)
Traffic investigations and models (1980, …, 2008)
Biology (R. Rosen, 1950- 60- ….)
Informatics, physics, cellular automata, etc. (D. Dubois, 1982 - ….)
Models of society (Makarenko, 1998 - …)
ANTICIPATION
The anticipation property is that the individual makes a decision accounting the future states of the system [1].
One of the consequences is that the accounting for an anticipatory property leads to advanced mathematical models. Since 1992 starting from cellular automata the incursive relation had been introduced by D. Dubois for the case when
„the values of of state X(t+1) at time t+1 depends on values X(t-i) at time t-i, i=1,2,…, the value X(t) at time t and the value X(t+j) at time t+j, j=1,2,… as the function of command vector p‟ [1].
ANTICIPATION In the simplest cases of discrete systems this
leads to the formal dynamic equations (for the case of discrete time t=0, 1, ..., n, ... and finite number of elements M):
where R is the set of external parameters (environment, control), {si(t)} the state of the system at a moment of time t (i=1, 2, …, M), g(i) horizon of forecasting, {G} set of nonlinear functions for evolution of the elements states.
( 1) ({ ( )},...,{ ( ( ))}, ),i i i is t G s t s t g i R
“In the same way, the hyperincursion is an
extension of the hyper recursion in which several
different solutions can be generated at each time
step” [1, p.98].
According [1] the anticipation may be of „weak‟ type
(with predictive model for future states of system,
the case which had been considered by R. Rosen)
and of „strong‟ type when the system cannot make
predictions.
HOPFIELD TYPE NETWORK
WITH ANTICIPATION
SOME EXAMPLES OF
MODELS
( 1) ( ) ( 1)j ji i ji ix n f w x n w x n
N
i
iji
N
i
ijij nxwnxwfnx11
)1()()1()1(
EXAMPLE OF ACTIVATION
FUNCTION
0, 0
( ) , [0,1)
1, 1
якщо x
f x x якщо x
якщо x
Network with 2 coupled neurons
Single-valued periodicity
Neuronert with 2 neuroons
Multi-valued ciclicity
-0,2
0
0,2
0,4
0,6
0,8
1
1,2
-0,2 0 0,2 0,4 0,6 0,8 1 1,2
1 нейрон
2 н
ей
ро
н
Netework with 6 neurons. Ciclicity
Network with 8 neurons
The influence on anticipation
parameter
PROBLEMS AND PROSPECTS
RESEARCH DIRECTIONS
I. General investigations of abstract mathematical objects:
Definitions of regimes:
Periodicity;
Chaos;
Solitons;
Chimera states;
Bifurcations;
Attractors;
Etc.
RESEARCH DIRECTIONS
II. Investigation of concrete models and solutions
In artificial neural networks
In cellular automata
In coupled maps
Solitons, traveling waves
Self-organization
Collapses
Etc.
RESEARCH DIRECTIONS
III. Interpretations and applications
Traffic modeling
Crowds movement
Socio- economical systems
Control applications
Neuroscience
Conscious problem
Physics
IT
REFERENCES
1. Dubois D. Generation of fractals from incursive automata, digital
diffusion and wave equation systems. BioSystems, 43 (1997) 97-114.
2 Makarenko A., Goldengorin B. , Krushinski D. Game „Life‟ with
Anticipation Property. Proceed. ACRI 2008, Lecture Notes Computer
Science, N. 5191, Springer, Berlin-Heidelberg, 2008. p. 77-82
3. B. Goldengorin, D.Krushinski, A. Makarenko Synchronization of Movement for Large – Scale Crowd. In: Recent Advances in Nonlinear
Dynamics and Synchronization: Theory and applications. Eds. Kyamakya
K., Halang W.A., Unger H., Chedjou J.C., Rulkov N.F.. Li Z., Springer, Berlin/Heidelberg, 2009 277 – 303
4. Makarenko A., Stashenko A. (2006) Some two- steps discrete-time
anticipatory models with „boiling‟ multivaluedness. AIP Conference
Proceedings, vol.839, ed. Daniel M. Dubois, USA, pp.265-272.