UNIT I

Introduction To Neural Networks Biological Neural Networks Characteristics of Neural Networks Models of Neurons

What is Neural Network?

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, to process information.

An artificial neural network is a system based on the operation of biological neural networks, in other words, is an emulation of biological neural system.

Why learn Neural Networks?

Neural networks, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques.

A trained neural network can be thought of as an "expert" in the category of information it has been given to analyze.

Advantages

Adaptive learning: An ability to learn how to do tasks based on the data given for training or initial experience.

Self-Organization: An ANN can create its own organization or representation of the information it receives during learning time.

Real Time Operation: ANN computations may be carried out in parallel, and special hardware devices are being designed and manufactured which take advantage of this capability.

Fault Tolerance via Redundant Information Coding: Partial destruction of a network leads to the corresponding degradation of performance. However, some network capabilities may be retained even with major network damage.

Neural networks versus conventional computers

Conventional computers use an algorithmic approach i.e. the computer follows a set of instructions in order to solve a problem.

Neural networks process information in a similar way the human brain does. The network is composed of a large number of highly interconnected processing elements (neurons) working in parallel to solve a specific problem. Neural networks learn by example. They cannot be programmed to perform a specific task.

Neural networks and conventional algorithmic computers are not in competition but complement each other.

Biological Neural Networks

A neural network is mans crude way of trying to simulate the brain electronically. So to understand how a neural net works we first must have a look at how the human brain works.

Neuron

The human brain contains about 10 billion nerve cells, or neurons.

Each neuron is connected to thousands of other neurons and communicates with them via electrochemical signals.

Signals coming into the neuron are received

via junctions called synapses, these in turn are located at the end of branches of the neuron cell called dendrites.

Characteristics of Neural Network

parallel, distributed information processing

high degree of connectivity among basic units

connections are modifiable based on experience

learning is a constant process, and usually unsupervised

learning is based only on local information performance degrades gracefully if some

units are removed

Summary

Neural Networks exhibits brain like behaviors like: Learning Association Categorization Generalization Feature Extraction Optimization Noise Immunity

Models of Neuron

An artificial neuron is simply an electronically modeled biological neuron. How many neurons are used depends on the task at hand.

Mathematical Model

McCulloch Pitts Model

The early model of an artificial neuron is introduced by Warren McCulloch a neuroscientist and Walter Pitts a logician in 1943.

The McCulloch-Pitts neural model is also known as linear threshold gate.

An Example

Object Purple? Round? Eat?

Blueberry 1 1 1

Golf Ball 0 1 0

Banana 0 0 0

Problems

Set the threshold so that the bird eats any object that is round or purple or both.

Set the threshold so that the bird eats all the objects.

Excitatory & Inhibitory

The signals are called excitatory because they excite the neuron toward possibly sending its own signal.

So as the neuron receives more and more excitatory signals, it gets more and more excited, until the threshold is reached, and the neuron sends out its own signal.

The Inhibitory signals have the effect of inhibiting the neuron from sending a signal.

When a neuron receives an inhibitory signal, it becomes less excited, and so it takes more excitatory signals to reach the neuron's threshold.

In effect, inhibitory signals subtract from the total of the excitatory signals, making the neuron more relaxed, and moving the neuron away from its threshold.

Perceptron

A perceptron which was introduced by Frank Rosenblatt in 1958.

Essentially the perceptron is an MCP neuron where the inputs are first passed through some "preprocessors" which are called association units.

These association units detect the presence of certain specific features in the inputs.

Perceptron

In essence an association unit is also an MCP neuron which is 1 if a single specific pattern of inputs is received, and it is 0 for all other possible patterns of  inputs.

Adaline (Adaptive linear element)

An important generalization of the perceptron training algorithm.

It was presented by Widrow and Hoff as the 'least mean square' (LMS) learning procedure, also known as the delta rule.

The main functional difference with the Perceptron training rule is the way the output of the system is used in the learning rule.

Neural Network Topologies

The arrangements of the processing units, connections, and pattern input /output is referred to as topology.

Connections can be made in two waysInterlayerIntralayer

Feedforward & Feedback

Feed-forward ANNs allow signals to travel one way only; from input to output.

There is no feedback (loops) i.e. the output of any layer does not affect that same layer.

They are extensively used in pattern recognition. This type of organization is also referred to as bottom-up or top-down.

Feedforward Network Diagram

Basic Learning Rules

Learning: The ability of the neural network (NN) to learn from its environment and to improve its performance through learning.

Learning is a process by which the free parameters of a neural network are adapted through a process of stimulation by the environment in which the network is embedded.

The type of the learning is determined by the manner in which the parameter changes take place.

Learning Rules

Hebb’s Law

Perceptron Learning Law

Delta Learning Law

Widrow and Hoff LMS Learning Law

Correlation Learning Law

Instar ( Winner – take – all)

Outstar Learning Law

Supervised Learning

During the training session of neural network, an input stimulus is applied that results in output response.

This response is compared with an priori desired output signal, the target response.

If the actual response differs from the target response, the neural network generates an error signal.

This error signal is used to calculate the adjustments that should be made to network’s synaptic weights so that the actual o/p matches the target output.

This error minimization process requires a special circuit known as supervisor or teacher, hence the name Supervised learning.

A neural net is said to learn supervised, if the desired output is already known.

Unsupervised Learning

Unsupervised learning does not require a teacher; that is there is no target output.

During the training session, the neural net receives at its input many different excitations or input patterns.

The neural net arbitrarily organizes the patterns into categories.

When the stimulus is applied later, the neural net provides an output response indicating the class to which the stimulus belongs.

If a class cannot be found for the input stimulus, a new class is generated.

Even though unsupervised learning does not require a teacher, it requires guidelines to determine how it will form groups.

Reinforced Learning

Reinforced learning requires one or more neurons at the output layer and a teacher.

Unlike supervised learning, teacher does not indicate how close the actual output is to the target output but whether the actual output is the same with target output or not.

The teacher does not present the target output to the network, but presents only a pass/fail indication.

Contd..

The error signal generated during the training session is binary: pass or fail.

If the teachers indication is bad, the network readjusts its parameters and tries again and again until it gets its output response right.

There is no clear indication if the output response is moving in the right direction or how close it is to the correct response it is.

Competitive Learning

It is another form of supervised learning that is distinctive because of its characteristics operation and architecture.

Several neurons are at the output layer. When an input stimulus is applied, each output neuron competes with the others to produce the closest output signal to the target.

Contd..

This output becomes the dominant one, and the other outputs cease producing an output signal for that stimulus.

For another stimulus, another output neuron becomes the dominant one.

Thus each output neuron is trained to respond to a different input stimulus.