Chapter0. Overview of the Neural Network
description
Transcript of Chapter0. Overview of the Neural Network
![Page 1: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/1.jpg)
1
1. Brain = Neural Network in Topology
Chapter0. Overview of the Neural NetworkNeural Network = Computational Structure of the Brain= Distributed, Adaptive, Nonlinear Learning Machine built from many Processing Elements
Synonyms :
Artificial Neural System, Artificial Neural Network, Neuromorphic System,Parallel Distributed Processing, Adaptive Network, Connnectionism, Neurocomputer.
Cell body
![Page 2: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/2.jpg)
2
● Sensory Perception (Pattern Recognition) and motor control – Vision, audition, olfaction, touch, temperature sensing
● Learning By Examples – Non-algorithmic, Trainability, Self-Organization
● Planning and Reasoning
● Learning and Adaptation by Generalization
● Reflex and Intuition (Similar to Table Lookup)
● Processing Ill-defined (Unstructured, Inconsistent, Probabilistic, Noisy) Information – fault-tolerant, flexible, robust
● As Computer: Massively parallel and distributed : Algorithm + Architecture
● As Computer: Wetware (Netware) vs. Software/Hardware ● As Math: Nonlinear and Adaptive Modeling Scheme
2. Features of the Human Brain (Differences from Digital Computers)
![Page 3: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/3.jpg)
3
At birth, 10 11 neurons Innate, Tends to Decrease with Time. However, their interconnections evolve and new ones are created.
![Page 4: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/4.jpg)
4
● Webster = Ability to learn or understand or to deal with new or trying situations : REASON
● Jang = Humanlike Expertise, adapt and learn to do better in changing env. And explain its decisions/actions
● Constituent Tech. = Synergy of NN, FL, EC EC = Systematic random search = Biological genetics + Natural selection)
● CI = Any methodology involving computing exhibiting ability to learn (do better) with new situations by reasoning (generalization, discovery, association, abstracting); also explain how it reasons.
Intelligence ?
3. Intelligent [Learning] Machines
![Page 5: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/5.jpg)
5
History : Intelligence to Machines Freedom to Mankind !
Industrial Revolution (Machine) Information Technology Revolution Artificial Brain (Intelligence) Revolution : Creative, human-friendly, autonomous (adaptive) Brain = Final Frontier = Neural Nets
● Left Brain (Logic) – Program (Symbolic, Software, Structured) Machine Learning, Expert System : AI
● Right Brain (Intuition, Emotion) – Neural Network (Numeric, Hardware, Unstructured) Cybernetics, Bionics
![Page 6: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/6.jpg)
6
(Ref. Eberhart, Chap. 1 of Computational Intelligence, IEEE Press, 95)
Intelligence
CIBI
AI
SymbolicComp.
Numeric Comp.
Biology
MI = AI or CI
NN
FL
EC
NFE
NE
NF
FE
CI
BI
Intelligence
Bezdek
4. Computational Intelligence and Soft Computing
![Page 7: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/7.jpg)
7
![Page 8: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/8.jpg)
8
![Page 9: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/9.jpg)
9
![Page 10: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/10.jpg)
10
5. Neural Network Architecture
w1
dendrite
cell (soma)
axon
x1
wm
xm
wm+1xm+1
y
ftn activation :
1
1
m
jjj xwy
Activation function
Linear Combination
![Page 11: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/11.jpg)
11
![Page 12: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/12.jpg)
12
![Page 13: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/13.jpg)
13
(1) Feedforward Architecture
Input
Hidden Output
.... ....General
Layered(3-2-2)
Input Hidden Output
![Page 14: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/14.jpg)
14
![Page 15: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/15.jpg)
15
(2) Feedback, Recurrent Architecture
![Page 16: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/16.jpg)
16
![Page 17: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/17.jpg)
17
![Page 18: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/18.jpg)
18
![Page 19: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/19.jpg)
19
6 . Usage of the Neural Network – Function Approximation and Generalization
◆ Training
NN
NN
Teacher
Word
e-
+
Voice NN
Teacher
Object Image
e
Object Name
-
+
Amorphous
◆ Training
![Page 20: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/20.jpg)
20
◆ Apply to Different Tasks After Training
NN
Voice Word
Object Image Object Name
![Page 21: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/21.jpg)
21
Learning Mode (weights change) Performance Mode (weights fixed)
◆ Training – Function Approximation = Create Internal Representations Only through Examples
NeuralNetwork
x)(),( xwxy fF
y
x
y f(x)
f (x): Normally Unknown
7. Generalization By Nonlinear Interpolation
Digital Computer vs. Neural Computer
• Digital – recalculate even for same inputs
• Neural – can memorize and recall results of previous calculation [previous answers].
if w is fixed after training →
NN is a Model-Free EstimatorTraining Data
Discrete Samples
Underlying Function
![Page 22: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/22.jpg)
22
◆ Generalization
xy = F ( x , w * )
NN
Function Approximation
F ( x , w * )
x
F ( x , w )F ( x , w " )
F ( x , w ' )
f ( x ) ② ①
②
① ① Approximation ② Generalization
![Page 23: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/23.jpg)
23
![Page 24: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/24.jpg)
24
![Page 25: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/25.jpg)
25
![Page 26: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/26.jpg)
26
8. Applications
1) Pattern Recognition & Character Recognition, Document Search
Face and Speech Recognition - Biometrics
Computer Vision : Image Understanding, Object Recognition, PCB Inspection
2) Model Building from Experimental Data: Function Mapping,
Regression, System Identification, Data Mining (Knowledge Discovery), Prediction
3) Image / Signal Processing and Communication
4) Optimization
5) Time Series Analysis and Financial Engineering
6) Medicine - Patient Care and Clinical Decision Support,
Biomedical Engineering, Bio-mimetics, Bio-informatics
7) Robotics and Automation
◆ Service Robots, Pet Robots, Surveillance Robots
![Page 27: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/27.jpg)
27
◆ Process Control Kodak - Film Making Amoco – Oil Exploration ◆ Aircraft Control
◆ Automotive Control ◆ Machine Control / Maintenance ◆ Machine Health Monitoring / Diagnosis ◆ Diagnostics and Quality Control ◆ Power System Control ( Canada Vancouver Island Power) ◆ Chemical Product Design ( AIWARE 사의 CAD/Chem ) ◆ Airline Luggage Inspection System
( -20 % cost + 50 % performance ) ◆ Active Vibration Cancellation
8) Music Composition 9) Neural Network Products in Korea:
Green Technology – Counterfeit Recognizer ( W 66B for 5 yrs) Korea Axis – Speech Recognition Toy (2000. 12) Slip Processing Machine for Banks using Handwritten Recognition (2000.12) Speech Recognition Chip Product which is Robust to Noise Applying Human
Auditory Model (2000. 7)
![Page 28: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/28.jpg)
28
Industrial App of CIIndustrial App of CIGE
Imagination at work
Insurance underwriting, proactive maintenance Recom.
Paper Web time-to-break prediction with Fuzzy+nn
Equip Prognosis, anomally dete
CI model = domain knowledge + field data
Ford: CIS system
DOD Air Force Res. Lab.
Image Patterns below Clutter
Tracking & detection below clutter
NN appli – bioinfo, drug design, financial mkr pred, internet search engine, medical app, 30 commercial componemts,
Future Dir.
Drug design (big) nat lang under, search eng, high leve sensor fusion, sensor-web, neural & psycholinguistic study – working of mind, cog and emo, lang & musi
c.
![Page 29: Chapter0. Overview of the Neural Network](https://reader036.fdocuments.net/reader036/viewer/2022062222/56815e0c550346895dcc65e9/html5/thumbnails/29.jpg)
29
9. NN Development Tools
◆ TypesGeneral Purpose Computer
1) S/W Simulation 2) S/W Simulation with H/W Accelerators3) S/W Simulation on a Parallel Computer
Special Purpose H/W1) Neurocomputing Workstations2) Electronic - VLSI 3) Optical - Laser Holography
10. Government Sponsored NN Research Worldwide 1990-2000 Decade of the Brain (US)1990-2090 Century of the Brain (Japan)1998-2007 Braintech 21(Korea) – Brain Research Promotion Act