CBP 2009-10Comp 3104 The Nature of Computing 1 Nature Inspired Computing Artificial Neural Nets -...

CBP 2009-10 Comp 3104 The Nature of Computing

1

Nature Inspired ComputingArtificial Neural Nets - Symbiosis between computer and cognitive sciences


2

Cajal -1-

Cajal Golgi

Cajal + Golgi indentification of independent neurons by staining, microscopy and looking. (Nobel Prize 1906)


3

Cajal -2-


4

Rat Neurons


5

Neuron Grows on Electrodes

Copyright © 2000 Yoonkey Nam - Department of Electrical & Computer Engineering; Imaging Technology Group; Beckman Institute; and the University of Illinois


6

Investigation of Single Neurons

Microelectrode recording of Biological Neuron activation using tungsten electrode

Hubel and Weisel. Nobel Prize 1958

Photomicrograph: Height = 1mm.


7

Biological Neurons

dendrites

axonsynapse

Signal flow Big Neurological principle #1 Neurons work using electricity, not blood or other special goo

Signal shape


8

Single Neuron

In 2

In 1

In 3

In 4In 1 + In 2 + In 3 + In 4

“input”

“activation”

“activation”

“input”

A

A

B

B

“threshold”

Big Neurological principle #2 “Integrate and Fire” Inputs summed. If above threshold output fires.


9

Learning in Neural Nets

Before Learning

After Learning

Big Neurological principle #3 “Hebbian Learning” Synapse strength increases if both cells A and B are firing

A

A

B

B


10

Brains Minds and Computers

Brains Computers

• Work using Electricity• Have inputs and outputs

• Can learn by experience• Can be taught

• Work using Electricity• Have inputs and outputs

• Can be programmed

So do we understand brains? Yep. Do we therefore understand Minds? Nope.


11

Artificial Neurons

In 1 + In 2 + In 3 + In 4

In 1

In 2

In 3

“output”

“input”

A

A

B

B

“threshold”

inputs

output


12

Learning Logical Gates

0.5

Threshold = 1

1

Threshold = 1

1

1.5

Ouput neuron fires only when sum is greater than the threshold

AND - gate

OR - gate

A B O

0 0 0

0 1 0

1 0 0

1 1 1

A B O

0 0 0

0 1 1

1 0 1

1 1 1


13

Training an Artificial Neural Net

eyes motors

right

left

1. We want to train the robot to move towards the light

2. So when the right eye gets light, the left motor neuron must fire and vice versa

3. We must therefore strengthen the cross - connections and kill the direct connections


14

Back Propagation of Errors

eyes motors

right

left

1 0.5

eyes motors

right

left

1

0.5

Let’s say desired motor drive is 1.0 (full forwards).

Here the right motor drive should be 0 but it is 0.5. So the error is

Desired - actual = 0.0 - 0.5 = -0.5

So we decrease the connection by 0.5

Here the right motor drive should be 1.0 but it is 0.5. So the error is

Desired - actual = 1.0 - 0.5 = +0.5

So we increase the connection by 0.5


15

Neural Net Solver


16

Medical Application

Flu

Neural Net

cough

headache


17

Medical Diagnosis

Cough

Headache

Meningitis

Flu

Pneuomonia

Not ill

1Cough

Headache 1

Flu 1


18

“Classical” Medical Diagnosis

If ( (symptom ! = cough) && (symptom != headache) )

illness = no illness;

else if ( (symptom ! = cough) && (symptom == headache) )

illness = meningitis;

else if ( (symptom == cough) && (symptom != headache) )

illness = pneumonia;

else if ( (symptom == cough) && (symptom == headache) )

illness = flu;

Rule-based Learning “ if … then …. else … “


19

ECG Interpretation

R-R interval

S-T elevation

P-R interval

QRS duration

AVF lead

QRS amplitude

SV tachycardia

Ventricular tachycardia

LV hypertrophy

RV hypertrophy

Myocardial infarction


20

NNets vs Expert Systems

Modeling ExamplesExplanation

Effort Needed Provided

Rule-based Exp. Syst. high low high

Bayesian Nets high low moderate

Classification Trees low high “high”

Neural Nets low high low

Regression Models high moderate moderate

CBP 2009-10Comp 3104 The Nature of Computing 1 Nature Inspired Computing Artificial Neural Nets -...

Documents

Transcript of CBP 2009-10Comp 3104 The Nature of Computing 1 Nature Inspired Computing Artificial Neural Nets -...