3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also...
-
date post
21-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also...
![Page 1: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/1.jpg)
3.2. Neurons and their networks3.2.1 Biological neurons
Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more specifically the brain).
![Page 2: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/2.jpg)
The nervous system is made up of several different types of cells:
- Neurons
- Astrocytes
- Microglia
- Schwann cells
Neurons do the computing, the rest is infrastructure
![Page 3: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/3.jpg)
Astrocytes
Star-shaped, abundant, and versatile
Guide the migration of developing neurons
Act as K+ and NT buffers
Involved in the formation of the blood brain barrier
Function in nutrient transfer
![Page 4: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/4.jpg)
Microglia
Specialized immune cells that act as the macrophages of the central nervous system
![Page 5: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/5.jpg)
Schwann cells and Oligodendrocytes
Produce the myelin sheath which provides the electrical insulation for neurons and nerve fibers
Important in neuronal regeneration
![Page 6: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/6.jpg)
Myelination – electrically insulates the axon, which increases the transport speed of the action potential
![Page 7: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/7.jpg)
Types of neurons
SensoryNeuron
Motor Neuron
Brain
Lots of interneurons
![Page 8: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/8.jpg)
What they look like
![Page 9: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/9.jpg)
![Page 10: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/10.jpg)
...or schematically
![Page 11: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/11.jpg)
In fact, things are a bit more crowded
![Page 12: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/12.jpg)
Neurons communicate with each other, we will see later how this works. This will be the "neural network"
![Page 13: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/13.jpg)
Thus, neurons need to be able to conduct information in 2 ways:
1.From one end of a neuron to the other end.This is accomplished electrically via action potentials
2.Across the minute space separating one neuron from another. This is accomplished chemically via neurotransmitters.
![Page 14: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/14.jpg)
Cell Membrane at rest
Na+ Cl-K+
Na+
Cl-K+ A-
Outside of Cell
Inside of Cell
Potassium (K+) can pass through to equalize its concentration
Sodium and Chlorine cannot pass through
Result - inside is negative relative to outside
- 70 mV
Resting potential of neurons
![Page 15: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/15.jpg)
Now lets open a Na channel in the membrane...
If the initial amplitude of the GP is sufficient, it will spread all the way to the axon hillock where V-gated channels reside. At this point an action potential can be excited if the voltage is high enough.
![Page 16: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/16.jpg)
![Page 17: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/17.jpg)
![Page 18: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/18.jpg)
N.B. The gating properties of ion channels were determined long before it was known they existed from electrical measurements (conductivity of squid axons to Na and K)
Similar for the transport of K – the different coefficients imply the number of opening and gating bits...
![Page 19: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/19.jpg)
With modern crystalography, these effects have been observed...
![Page 20: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/20.jpg)
Transport of the action potential, like a row of dominos falling...
![Page 21: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/21.jpg)
This goes a lot faster with myelinated axons – saltating transport...
![Page 22: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/22.jpg)
![Page 23: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/23.jpg)
Once at the syapse, the signal is transmitted chamically via neurotransmitters (e.g. Acetylcholin) These are then used to excite a new graded potential in the next neuron
![Page 24: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/24.jpg)
This graded potential can be both positive and negative, depending on the environment
![Page 25: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/25.jpg)
![Page 26: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/26.jpg)
The intensity of the signal is given by the firing frequency
![Page 27: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/27.jpg)
These properties are caricatured in the McCulloch-Pitts neuron
Learning happens when the weights wij are changed in response to the environment – this needs an updating rule
![Page 28: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/28.jpg)
Common in informatics is the iterative learning, which needs a teacher. I.e. The weights are adjusted so that in every learning step, the distance to the correct answer is obtained.
This is known as the perceptron
![Page 29: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/29.jpg)
Inputlayer
Firsthiddenlayer
Secondhiddenlayer
Outputlayer
O u
t p
u t
S
i g n
a l
s
I n
p u
t S
i g
n a
l sWith the use of hidden layers, not linearly separable variable can be learnt...
![Page 30: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/30.jpg)
An example: letter recognition
![Page 31: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/31.jpg)
The problems that can be solved depend on the structure of the network
![Page 32: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/32.jpg)
3.2.2 Hebbian learning
This means that a synapse gets stronger as
neighbouring cells are more correlated
![Page 33: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/33.jpg)
Hebb’s Law can be represented in the form of Hebb’s Law can be represented in the form of two rules:two rules:
1. If two neurons on either side of a connection 1. If two neurons on either side of a connection are activated synchronously, then the weight of are activated synchronously, then the weight of that connection is increased. that connection is increased.
2. If two neurons on either side of a connection 2. If two neurons on either side of a connection are activated asynchronously, then the weight of are activated asynchronously, then the weight of that connection is decreased.that connection is decreased.
Hebb’s Law provides the basis for learning Hebb’s Law provides the basis for learning without a teacher. Learning here is a without a teacher. Learning here is a local local phenomenonphenomenon occurring without feedback occurring without feedback from the environment.from the environment.
![Page 34: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/34.jpg)
i j
I n
p u
t S
i g
n a
l s
O u
t p
u t
S i
g n
a l s
Hebbian learning in a neural network
)( )( )( pxpypw ijij
![Page 35: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/35.jpg)
A Hebbian Cell Assembly
By means of the Hebbian Learning Rule, a circuit of continuously firing neurons could be learned by the network.
The continuing activation in this cell assembly does not require external input.
The activation of the neurons in this circuit would correspond to the perception of a concept.
![Page 36: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/36.jpg)
A Cell Assembly
Input from the environment
![Page 37: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/37.jpg)
A Cell Assembly
Input from the environment
![Page 38: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/38.jpg)
A Cell Assembly
Input from the environment
![Page 39: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/39.jpg)
A Cell Assembly
Input from the environment
![Page 40: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/40.jpg)
Note that the input from theenvironment is gone...
A Cell Assembly
![Page 41: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/41.jpg)
A Cell Assembly
![Page 42: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/42.jpg)
Hebbian learning implies that weights can only Hebbian learning implies that weights can only increase. To resolve this problem, we might increase. To resolve this problem, we might impose a limit on the growth of synaptic impose a limit on the growth of synaptic weights. It can be done by introducing a non-weights. It can be done by introducing a non-linear linear forgetting factor into Hebb’s Law: into Hebb’s Law:
wherewhere is the forgetting factor.is the forgetting factor.
The fThe forgetting factor usually falls in the orgetting factor usually falls in the interval between 0 and 1, typically between interval between 0 and 1, typically between 0.01 and 0.1, to allow only a little “forgetting” 0.01 and 0.1, to allow only a little “forgetting” while limiting the weight growth.while limiting the weight growth.
)( )( )( )( )( pwpypxpypw ijjijij
![Page 43: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/43.jpg)
First simulation of Hebbian learning
• Rochester et al. attempted to simulate the emergence of cell assemblies in a small network of 69 neurons. They found that everything became active in their network.
• They decided that they needed to include inhibitory synapses. This worked and cell assemblies did, indeed, form.
• This was later confirmed in real brain circuitry.
![Page 44: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/44.jpg)
In fact, these inhibitory connections are distance dependent and as such
give rise to structure
![Page 45: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/45.jpg)
Exciation happens within columns and inhibition further away
![Page 46: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/46.jpg)
Connectionstrength
Distance
Excitatoryeffect
Inhibitoryeffect
Inhibitoryeffect
0
1
Long range inhibition and short range activation gives rise to patterns
See also the excursion into pattern formation in Sec 3.6
![Page 47: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/47.jpg)
Input layer
Kohonen layer
(a)
Input layer
Kohonen layer
1 0(b)
0 1
Feature mapping Kohonen model
ncompetitio theloses neuron if ,0
ncompetitio the winsneuron if ),(
j
jwxw
ijiij
![Page 48: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/48.jpg)
Set initial synaptic weights to small random values, say in an interval [0, 1], and assign a small positive value to the learning rate parameter .
Competitive learning
,)()()(
2/1
1
2][
n
iijij
jpwxpminpj WXX
)()()1( pwpwpw ijijij Update weights:
j(p) is the neighbourhood function centred around jX
Iterate...
![Page 49: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/49.jpg)
To illustrate competitive learning, consider the Kohonen network with 100 neurons arranged in the form of a two-dimensional lattice with 10 rows and 10 columns. The network is required to classify two-dimensional input vectors each neuron in the network should respond only to the input vectors occurring in its region.
The network is trained with 1000 two-dimensional input vectors generated randomly in a square region in the interval between –1 and +1. The learning rate parameter is equal to 0.1.
![Page 50: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/50.jpg)
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1W(1,j)
-1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1W
(2,j)
Initial random network
![Page 51: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/51.jpg)
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1W(1,j)
-1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1W
(2,j)
After 100 steps
![Page 52: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/52.jpg)
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1W(1,j)
-1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1W
(2,j)
After 1000 steps
![Page 53: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/53.jpg)
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1W(1,j)
-1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1W
(2,j)
After 10000 steps
![Page 54: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/54.jpg)
Or for letter recognition
![Page 55: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/55.jpg)
In the cortex, this gives rise to the homunculus, the spatial distribution of nerve cells responsible for senses
![Page 56: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/56.jpg)
Similar for other features in the cortex
![Page 57: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/57.jpg)
3.2.3 Associative networks
xi
x1
x2
xnI n
p u
t
S i
g n
a l
s
yi
y1
y2
yn
1
2
i
n
O u
t p
u t
S
i g
n a
l s
In a Hopfield Network, every neuron
is connected to every other neuron
![Page 58: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/58.jpg)
y1
y2
y3
(1, 1, 1)( 1, 1, 1)
( 1, 1, 1) (1, 1, 1)
(1, 1, 1)( 1, 1, 1)
(1, 1, 1)( 1, 1, 1)
0
Topological state analysis for a three neuron Hopfield network
IYYW MM
m
Tmm
1
![Page 59: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/59.jpg)
The stable state-vertex is determined by the weight matrix W, the current input vector X, and the threshold matrix . If the input vector is partially incorrect or incomplete, the initial state will converge into the stable state-vertex after a few iterations.
))(sgn()1( ijj iji tSwtS
![Page 60: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/60.jpg)
Energy function of Hopfield net: multidimensional landscape
jiji SSwH ,2
1
pj
piij vvw
![Page 61: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/61.jpg)
Example: Restoring corrupted memory patterns
Original T Half is corrupted
20% of T corrupted
![Page 62: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/62.jpg)
![Page 63: 3.2. Neurons and their networks 3.2.1 Biological neurons Tasks such as navigation, but also cognition, memory etc. happen in the nervous system (more.](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d5e5503460f94a3d15c/html5/thumbnails/63.jpg)
Recap Sec. 3.2
The brain is a network of neurons, whose properties are important in how we learn
Within neurons, signals are transported electrically, between chemically
This can be abstracted in a McCulloch Pitts neuron
Hebbian learning makes strong connections stronger (leads to pattern formation)
This is taken further in Kohonen networks and competitive learning