A Brief History of the Development of Artificial Neural...
Transcript of A Brief History of the Development of Artificial Neural...
![Page 1: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/1.jpg)
A Brief History of the Development of ArtificialNeural Networks
Prof. Bernard Widrow
Department of Electrical Engineering
Stanford University
Baidu • July 18, 2018
Prof. Widrow @ Berkeley A Brief History of Neural Networks 1 / 54
![Page 2: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/2.jpg)
Adaptive linear neuron (Adaline)
Figure 1: Adaline. An adaptive linear neuron
Widrow talk The Hebbian-LMS Algorithm
LMS algorithm
𝑊𝑘+1 = 𝑊𝑘 + 2𝜇𝜖𝑘𝑋𝑘𝜖𝑘 = 𝑑𝑘 − 𝑋𝑘𝑇𝑊𝑘
Prof. Widrow @ Berkeley A Brief History of Neural Networks 2 / 54
![Page 3: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/3.jpg)
Knobby Adaline, 1959
FIGURE 2 Knobby adaline
Widrow talk The Hebbian-LMS Algorithm
Prof. Widrow @ Berkeley A Brief History of Neural Networks 3 / 54
![Page 4: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/4.jpg)
Prof. Widrow @ Berkeley A Brief History of Neural Networks 4 / 54
![Page 5: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/5.jpg)
An adaptive linear neuron(ADALINE)
Prof. Widrow @ Berkeley A Brief History of Neural Networks 5 / 54
![Page 6: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/6.jpg)
A Two-Input Adaline
Prof. Widrow @ Berkeley A Brief History of Neural Networks 6 / 54
![Page 7: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/7.jpg)
Separating line in pattern space
Prof. Widrow @ Berkeley A Brief History of Neural Networks 7 / 54
![Page 8: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/8.jpg)
Adaline Capacity
Prof. Widrow @ Berkeley A Brief History of Neural Networks 8 / 54
![Page 9: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/9.jpg)
Adaline with polynomial preprocessor
Prof. Widrow @ Berkeley A Brief History of Neural Networks 9 / 54
![Page 10: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/10.jpg)
An elliptical separating boundary for the Exclusive
NOR Function
Prof. Widrow @ Berkeley A Brief History of Neural Networks 10 / 54
![Page 11: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/11.jpg)
A two-Adaline form of Madaline I
Prof. Widrow @ Berkeley A Brief History of Neural Networks 11 / 54
![Page 12: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/12.jpg)
Separating lines for the two-element Madaline
Prof. Widrow @ Berkeley A Brief History of Neural Networks 12 / 54
![Page 13: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/13.jpg)
A neuronal implementation of AND, OR, and MAJ
logic function
Prof. Widrow @ Berkeley A Brief History of Neural Networks 13 / 54
![Page 14: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/14.jpg)
A three-layer adaptive neural network
Prof. Widrow @ Berkeley A Brief History of Neural Networks 14 / 54
![Page 15: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/15.jpg)
Principal of Minimal Disturbance
Prof. Widrow @ Berkeley A Brief History of Neural Networks 15 / 54
![Page 16: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/16.jpg)
Error-Correction Algorithms for the Single Element
Prof. Widrow @ Berkeley A Brief History of Neural Networks 16 / 54
![Page 17: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/17.jpg)
α-LMS Algorithm
Prof. Widrow @ Berkeley A Brief History of Neural Networks 17 / 54
![Page 18: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/18.jpg)
Weight correction by the LMS rule
Prof. Widrow @ Berkeley A Brief History of Neural Networks 18 / 54
![Page 19: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/19.jpg)
Rosenblatt’s Perceptron
Prof. Widrow @ Berkeley A Brief History of Neural Networks 19 / 54
![Page 20: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/20.jpg)
Adaptive Threshold Element in the Perceptron
Prof. Widrow @ Berkeley A Brief History of Neural Networks 20 / 54
![Page 21: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/21.jpg)
Perceptron Rule
Prof. Widrow @ Berkeley A Brief History of Neural Networks 21 / 54
![Page 22: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/22.jpg)
May’s Rule
Prof. Widrow @ Berkeley A Brief History of Neural Networks 22 / 54
![Page 23: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/23.jpg)
May’s Increment Adaptation Rule
Prof. Widrow @ Berkeley A Brief History of Neural Networks 23 / 54
![Page 24: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/24.jpg)
May’s Modified Relaxation Rule
Prof. Widrow @ Berkeley A Brief History of Neural Networks 24 / 54
![Page 25: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/25.jpg)
Error-Correction Algorithms for Multi-Element
Networks
Prof. Widrow @ Berkeley A Brief History of Neural Networks 25 / 54
![Page 26: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/26.jpg)
A five-Adaline example of the MR I Architecture
Prof. Widrow @ Berkeley A Brief History of Neural Networks 26 / 54
![Page 27: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/27.jpg)
MR II of B. Widrow and R. Winter
Prof. Widrow @ Berkeley A Brief History of Neural Networks 27 / 54
![Page 28: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/28.jpg)
Steepest-Descent Algorithms for the Single Element
Prof. Widrow @ Berkeley A Brief History of Neural Networks 28 / 54
![Page 29: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/29.jpg)
Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 29 / 54
![Page 30: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/30.jpg)
Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 30 / 54
![Page 31: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/31.jpg)
Conventional LMS
Prof. Widrow @ Berkeley A Brief History of Neural Networks 31 / 54
![Page 32: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/32.jpg)
Implementation of Conventional LMS
Prof. Widrow @ Berkeley A Brief History of Neural Networks 32 / 54
![Page 33: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/33.jpg)
“Sigmoid” LMS(Back-Prop)
Prof. Widrow @ Berkeley A Brief History of Neural Networks 33 / 54
![Page 34: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/34.jpg)
Implementation of “Sigmoid” LMS(Back-Prop)
Prof. Widrow @ Berkeley A Brief History of Neural Networks 34 / 54
![Page 35: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/35.jpg)
“Sigmoid” LMS(MR III)
Prof. Widrow @ Berkeley A Brief History of Neural Networks 35 / 54
![Page 36: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/36.jpg)
Implementation of Sigmoid LMS-MRIII
Prof. Widrow @ Berkeley A Brief History of Neural Networks 36 / 54
![Page 37: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/37.jpg)
Error Surfaces for the Single Element
Prof. Widrow @ Berkeley A Brief History of Neural Networks 37 / 54
![Page 38: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/38.jpg)
Linear MSE, Sigmoid MSE, and Signum MSE
Prof. Widrow @ Berkeley A Brief History of Neural Networks 38 / 54
![Page 39: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/39.jpg)
Linear Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 39 / 54
![Page 40: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/40.jpg)
Sigmoid Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 40 / 54
![Page 41: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/41.jpg)
Signum Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 41 / 54
![Page 42: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/42.jpg)
Linear Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 42 / 54
![Page 43: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/43.jpg)
Sigmoid Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 43 / 54
![Page 44: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/44.jpg)
Signum Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 44 / 54
![Page 45: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/45.jpg)
Steepest-Descent Algorithms for Multi-Element
Networks
Prof. Widrow @ Berkeley A Brief History of Neural Networks 45 / 54
![Page 46: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/46.jpg)
Backpropagation Network
Prof. Widrow @ Berkeley A Brief History of Neural Networks 46 / 54
![Page 47: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/47.jpg)
Detail of Backpropagation Node
Prof. Widrow @ Berkeley A Brief History of Neural Networks 47 / 54
![Page 48: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/48.jpg)
MRIII of David Andes
Prof. Widrow @ Berkeley A Brief History of Neural Networks 48 / 54
![Page 49: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/49.jpg)
Prof. Widrow @ Berkeley A Brief History of Neural Networks 49 / 54
![Page 50: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/50.jpg)
Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 50 / 54
![Page 51: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/51.jpg)
Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 51 / 54
![Page 52: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/52.jpg)
Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 52 / 54
![Page 53: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/53.jpg)
Mean Square Error Surface
Prof. Widrow @ Berkeley A Brief History of Neural Networks 53 / 54
![Page 54: A Brief History of the Development of Artificial Neural ...research.baidu.com/Public/ueditor/upload/file/... · 7/19/2018 · A two-Adaline form of Madaline I Prof. Widrow@Berkeley](https://reader030.fdocuments.net/reader030/viewer/2022040306/5ec51eb4848dae130f387327/html5/thumbnails/54.jpg)
Learning Rules
Prof. Widrow @ Berkeley A Brief History of Neural Networks 54 / 54