1

1. Brain = Neural Network in Topology

Chapter0. Overview of the Neural NetworkNeural Network = Computational Structure of the Brain= Distributed, Adaptive, Nonlinear Learning Machine built from many Processing Elements

Synonyms :

Artificial Neural System, Artificial Neural Network, Neuromorphic System,Parallel Distributed Processing, Adaptive Network, Connnectionism, Neurocomputer.

Cell body

2

● Sensory Perception (Pattern Recognition) and motor control – Vision, audition, olfaction, touch, temperature sensing

● Learning By Examples – Non-algorithmic, Trainability, Self-Organization

● Planning and Reasoning

● Learning and Adaptation by Generalization

● Reflex and Intuition (Similar to Table Lookup)

● Processing Ill-defined (Unstructured, Inconsistent, Probabilistic, Noisy) Information – fault-tolerant, flexible, robust

● As Computer: Massively parallel and distributed : Algorithm + Architecture

● As Computer: Wetware (Netware) vs. Software/Hardware ● As Math: Nonlinear and Adaptive Modeling Scheme

2. Features of the Human Brain (Differences from Digital Computers)

3

At birth, 10 11 neurons Innate, Tends to Decrease with Time. However, their interconnections evolve and new ones are created.

4

● Webster = Ability to learn or understand or to deal with new or trying situations : REASON

● Jang = Humanlike Expertise, adapt and learn to do better in changing env. And explain its decisions/actions

● Constituent Tech. = Synergy of NN, FL, EC EC = Systematic random search = Biological genetics + Natural selection)

● CI = Any methodology involving computing exhibiting ability to learn (do better) with new situations by reasoning (generalization, discovery, association, abstracting); also explain how it reasons.

Intelligence ?

3. Intelligent [Learning] Machines

5

History : Intelligence to Machines Freedom to Mankind !

Industrial Revolution (Machine) Information Technology Revolution Artificial Brain (Intelligence) Revolution : Creative, human-friendly, autonomous (adaptive) Brain = Final Frontier = Neural Nets

● Left Brain (Logic) – Program (Symbolic, Software, Structured) Machine Learning, Expert System : AI

● Right Brain (Intuition, Emotion) – Neural Network (Numeric, Hardware, Unstructured) Cybernetics, Bionics

6

(Ref. Eberhart, Chap. 1 of Computational Intelligence, IEEE Press, 95)

 Intelligence

CIBI

AI

SymbolicComp.

  Numeric Comp.

   Biology

MI = AI or CI

NN

FL

   EC

NFE

NE

NF

FE

CI

BI

Intelligence

Bezdek

4. Computational Intelligence and Soft Computing

7

8

9

10

5. Neural Network Architecture

w1

dendrite

cell (soma)

axon

x1

wm

xm

wm+1xm+1

y

ftn activation :

1

1

m

jjj xwy

Activation function

Linear Combination

11

12

13

(1) Feedforward Architecture

Input

Hidden Output

.... ....General

Layered(3-2-2)

Input Hidden Output

14

15

(2) Feedback, Recurrent Architecture

16

17

18

19

6 . Usage of the Neural Network – Function Approximation and Generalization

◆ Training

NN

NN

Teacher

Word

e-

+

Voice NN

Teacher

Object Image

e

Object Name

-

+

Amorphous

◆ Training

20

◆ Apply to Different Tasks After Training

NN

Voice Word

Object Image Object Name

21

Learning Mode (weights change) Performance Mode (weights fixed)

◆ Training – Function Approximation = Create Internal Representations Only through Examples

NeuralNetwork

x)(),( xwxy fF

y

x

y f(x)

f (x): Normally Unknown

7. Generalization By Nonlinear Interpolation

Digital Computer vs. Neural Computer

• Digital – recalculate even for same inputs

• Neural – can memorize and recall results of previous calculation [previous answers].

if w is fixed after training →

NN is a Model-Free EstimatorTraining Data

Discrete Samples

Underlying Function

22

◆ Generalization

xy = F ( x , w * )

NN

Function Approximation

F ( x , w * )

x

F ( x , w )F ( x , w " )

F ( x , w ' )

f ( x ) ② ①

②

① ① Approximation ② Generalization

23

24

25

26

8. Applications

1) Pattern Recognition & Character Recognition, Document Search

Face and Speech Recognition - Biometrics

Computer Vision : Image Understanding, Object Recognition, PCB Inspection

2) Model Building from Experimental Data: Function Mapping,

Regression, System Identification, Data Mining (Knowledge Discovery), Prediction

3) Image / Signal Processing and Communication

4) Optimization

5) Time Series Analysis and Financial Engineering

6) Medicine  - Patient Care and Clinical Decision Support,

Biomedical Engineering, Bio-mimetics, Bio-informatics

7) Robotics and Automation

◆ Service Robots, Pet Robots, Surveillance Robots

27

◆ Process Control Kodak - Film Making Amoco – Oil Exploration ◆ Aircraft Control

◆ Automotive Control ◆ Machine Control / Maintenance ◆ Machine Health Monitoring / Diagnosis ◆ Diagnostics and Quality Control ◆ Power System Control ( Canada Vancouver Island Power) ◆ Chemical Product Design ( AIWARE 사의 CAD/Chem ) ◆ Airline Luggage Inspection System

( -20 % cost + 50 % performance ) ◆ Active Vibration Cancellation

8) Music Composition 9) Neural Network Products in Korea:

Green Technology – Counterfeit Recognizer ( W 66B for 5 yrs) Korea Axis – Speech Recognition Toy (2000. 12) Slip Processing Machine for Banks using Handwritten Recognition (2000.12) Speech Recognition Chip Product which is Robust to Noise Applying Human

Auditory Model (2000. 7)

28

Industrial App of CIIndustrial App of CIGE

Imagination at work

Insurance underwriting, proactive maintenance Recom.

Paper Web time-to-break prediction with Fuzzy+nn

Equip Prognosis, anomally dete

CI model = domain knowledge + field data

Ford: CIS system

DOD Air Force Res. Lab.

Image Patterns below Clutter

Tracking & detection below clutter

NN appli – bioinfo, drug design, financial mkr pred, internet search engine, medical app, 30 commercial componemts,

Future Dir.

Drug design (big) nat lang under, search eng, high leve sensor fusion, sensor-web, neural & psycholinguistic study – working of mind, cog and emo, lang & musi

c.

29

9. NN Development Tools

◆ TypesGeneral Purpose Computer

1) S/W Simulation 2) S/W Simulation with H/W Accelerators3) S/W Simulation on a Parallel Computer

Special Purpose H/W1) Neurocomputing Workstations2) Electronic - VLSI 3) Optical - Laser Holography

10. Government Sponsored NN Research Worldwide 1990-2000 Decade of the Brain (US)1990-2090 Century of the Brain (Japan)1998-2007 Braintech 21(Korea) – Brain Research Promotion Act