EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks...
Transcript of EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks...
![Page 1: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/1.jpg)
EE04 804(B) Soft Computing Ver. 1.2
Class 2. Neural Networks - I
Feb – 23 , 2012
1
Sasidharan Sreedharan www.sasidharan.webs.com
3/1/2012
![Page 2: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/2.jpg)
Syllabus
Artificial Intelligence Systems- Neural
Networks, fuzzy logic, genetic
algorithms, Artificial neural networks:
Biological neural networks, model of an
artificial neuron, Activation functions,
architectures, characteristics- learning
methods, brief history of ANN research-
Early ANN architectures (basics only)-
McCulloh & Pitts model, Perceptron,
ADALINE, MADALINE
![Page 3: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/3.jpg)
Biological Neural Nets
• Pigeons as art experts (Watanabe et al.
1995)
– Experiment:
• Pigeon in Skinner box
• Present paintings of two different artists (e.g.
Chagall / Van Gogh)
• Reward for pecking when presented a
particular artist (e.g. Van Gogh)
![Page 4: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/4.jpg)
![Page 5: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/5.jpg)
![Page 6: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/6.jpg)
![Page 7: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/7.jpg)
• Pigeons were able to discriminate between Van
Gogh and Chagall with 95% accuracy (when
presented with pictures they had been trained on)
• Discrimination still 85% successful for previously
unseen paintings of the artists
• Pigeons do not simply memorise the pictures
• They can extract and recognise patterns (the
‘style’)
• They generalise from the already seen to make
predictions
• This is what neural networks (biological and artificial)
are good at (unlike conventional computer)
![Page 8: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/8.jpg)
•Human brain contains a massively
interconnected net of 1010-1011 (10 billion)
neurons (cortical cells)
•Biological Neuron
•- The simple “arithmetic computing”
element
Brain Computer: What is it?
8
![Page 9: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/9.jpg)
•The schematic model of a biological
neuron
•Synapses
•Dendrites
•Soma
•Axon •Dendrite
from other
•Axon from other neuron
1. Soma or body cell - is a large, round central body in which almost all the logical functions of the neuron are realized.
2. The axon (output), is a nerve fibre
attached to the soma which can serve as a final output channel of the neuron. An axon is usually highly branched.
3. The dendrites (inputs)- represent a highly branching tree of fibers. These long irregularly shaped nerve fibers
(processes) are attached to the soma.
4. Synapses are specialized contacts on a neuron which are the termination points for the axons from other neurons.
Biological Neurons
9
![Page 10: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/10.jpg)
•?
•Brain-Like Computer
•Brain-like computer –
•is a mathematical model of humane-brain principles of computations. This computer consists of those elements which can be called the biological neuron prototypes, which are interconnected by direct links called connections and which cooperate to perform parallel distributed processing (PDP) in order to solve a desired computational task.
•Neurons and Neural Net
•The new paradigm of computing mathematics consists of the combination of such artificial neurons into some artificial neuron net.
•Artificial Neural Network – Mathematical Paradigms of Brain-Like Computer
Brain-like Computer
10
![Page 11: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/11.jpg)
•NN as an model of brain-like Computer
An artificial neural network (ANN) is a massively parallel distributed processor that has a natural property for storing experimental knowledge and making it available for use. It means that:
•
Knowledge is acquired by the network through a learning (training) process;
The strength of the interconnections between neurons is implemented by means of the synaptic weights used to store the knowledge.
•The learning process is a procedure of the adapting the weights with a learning algorithm in order to capture the knowledge. On more mathematically, the aim of the learning process is to map a given relation between inputs and output (outputs) of the network.
•Brain
The human brain is still not well understood and indeed its behavior is very complex!
There are about 10 billion neurons in the human cortex and 60 trillion synapses of connections
The brain is a highly complex, nonlinear and parallel computer (information-processing system)
ANN as a Brain-Like Computer
11
![Page 12: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/12.jpg)
•Artificial Intellect with
Neural Networks
•Intelligent
Control
•Technical
Diagnistics
•Intelligent
• Data Analysis
•and Signal
Processing
•Advance
Robotics
•Machine
Vision
•Image &
Pattern
Recognition
•Intelligent
Security
Systems
•Intelligent
Medicine
Devices
•Intelligent
Expert
Systems
Applications of Artificial Neural
Networks
12
![Page 13: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/13.jpg)
ANNs – The basics
• ANNs incorporate the two fundamental components
of biological neural nets:
1. Neurones (nodes)
2. Synapses (weights)
![Page 14: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/14.jpg)
• Neurone vs. Node
![Page 15: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/15.jpg)
• Structure of a node:
• Squashing function limits node output:
![Page 16: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/16.jpg)
• Synapse vs. weight
![Page 17: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/17.jpg)
Feed-forward nets
• Information flow is unidirectional
• Data is presented to Input layer
• Passed on to Hidden Layer
• Passed on to Output layer
• Information is distributed
• Information processing is parallel
Internal representation (interpretation) of data
![Page 18: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/18.jpg)
• Feeding data through the net:
(1 0.25) + (0.5 (-1.5)) = 0.25 + (-0.75) = - 0.5
0.37751
1
5.0
eSquashing:
![Page 19: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/19.jpg)
Artificial neuron
19
• Human brain is a complex structure of highly
interconnected network of simple processing element called
neurons.
• The behaviour of a neuron can be captured as a simple
model.
• Every component of the model bears a direct analogy to the
actual constituents of a biological neuron and is termed as the
artificial neuron and is the basis for ANN.
• Biological neuron receives all inputs through the dendrites
sums them and produces an output if the sum is greater than
a threshold value.
• The input signals are passed on to the cell body through the
synapse which may accelerate and retard an arriving signal.
• The acceleration or retardation of the input signals are
modelled by the weights.
![Page 20: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/20.jpg)
Artificial Neuron – Simple Model
20
1 1 2 2 n n
n
i i
i=1
I=w x +w x +...+w x
= w x
y= (I):W here is
activation function or transfer function or squash function
which releases the output.
![Page 21: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/21.jpg)
I z
•Linear activation
•Threshold activation •Hyperbolic tangent activation
•Logistic activation
2
2
1
1
u
u
eu tanh u
e
1
1I
Ie
1, 0,
sign ( )1, 0.
if II I
if I
•input
•z
•1
•-1
•1
•0
•0
Σ
1
-1
Artificial Neuron:
Classical Activation Functions
21
•input
![Page 22: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/22.jpg)
1. Threshold Activation function
22
i
1
y = w x
w here the step function
know n as the H eaviside function
( ) = 1, I 0
0, I 0
n
i
i
is
I
Output signal is either 1 or 0 resulting in the neuron being on or off
![Page 23: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/23.jpg)
2. Signum function
23
( ) = 1, I
1, I
I
![Page 24: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/24.jpg)
3. Sigmoid function
24
1 ( )
1T
Ie
The function is a continuous function that varies gradually between the
asymptotic values 0 and 1 or -1 and +1
![Page 25: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/25.jpg)
4. Hyperbolic tangent function
25
( ) tanh( )I I
![Page 26: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/26.jpg)
McCulloch – Pitts Model (1943)
![Page 27: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/27.jpg)
Neural Network Architectures
27
• ANN is defined as a data processing system consisting of a
large number of simple highly interconnected processing
elements (artificial neurons).
• ANN structure can be represented by a directed graph
(digraph – directed graph or oriented graph)
• The vertices of the graph represents neurons (input/output)
and the edges, the synaptic links. The edges are labelled by
the weights attached to the synaptic links
![Page 28: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/28.jpg)
1.Single Layer Feed Forward
Network
![Page 29: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/29.jpg)
2.Multi Layer Feed forward
network
• Presence of hidden intermediate layer which are useful for
computations.
![Page 30: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/30.jpg)
3.Recurrent ANN Architecture
• Presence of at least one feedback loop.
• There could be self – feedback links
![Page 31: EE04 804(B) Soft Computing Ver. 1 - sudhinpk 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb – 23 , 2012 1 Sasidharan Sreedharan 3/1/2012 Syllabus Artificial Intelligence](https://reader031.fdocuments.net/reader031/viewer/2022022520/5b1a3ce67f8b9a37258d6310/html5/thumbnails/31.jpg)
Regards
Dr. Sasidharan Sreedharan
www.sasisreedhar.webs.com