Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.
-
Upload
allyson-ridgely -
Category
Documents
-
view
219 -
download
2
Transcript of Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.
![Page 1: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/1.jpg)
Hopefully a clearer version of Neural Network
![Page 2: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/2.jpg)
I1
O2
O1H1
H2I2
![Page 3: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/3.jpg)
Layers of Weights
• We Name Sets of Weights between layersAs W1 for weights between input Layer and
First Hidden LayerW2 for weights between next 2 layers and WN-1 for Weights between N-1th and Nth
Layer(i.e. Output Layer)In our example Net we just have 3 layersInput Hidden and Output So we have just
W1 and W2
![Page 4: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/4.jpg)
I1
O2
O1H1
H2I2
W1W2
![Page 5: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/5.jpg)
Weights along Individual Links
• Convention
• Each Weight is named as follows
• WNij
• N refers to the Layer of Weights
• So Between Input and First Hiden Layer i.e. W2ij is the Reference
• Between Hidden and Output W2ij
![Page 6: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/6.jpg)
Individual Weights within a layer
• Reference WNij
• WN refers to the Weight Layer
• ij refers to the indices of the source and destination nodes.
• So for example the weight between hidden node h1 and output node o2
• It belongs to weight layer 2 so W2
• i = 1 and j = 2 so Weight is W212
![Page 7: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/7.jpg)
I1
O2
O1H1
H2I2
W1W2
W212
![Page 8: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/8.jpg)
Full Naming of Weight Set
![Page 9: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/9.jpg)
I1
O2
O1H1
H2I2
W1 W2
W212W112
W221
W211
W222
W121
W111
W122
![Page 10: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/10.jpg)
With Actual Weights
![Page 11: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/11.jpg)
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
![Page 12: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/12.jpg)
Inputs
• 1 and 0
• Target outputs {0.7,0.6}
![Page 13: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/13.jpg)
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
1
0
![Page 14: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/14.jpg)
Hidden Layer Computation
• Xi =iW1 = • 1 * 1 + 0 * -1 = 1, • 1 * -1 + 0 * 1 = -1 = • { 1 - 1} = {Xi1,Xi2} = Xi
xF
1
1
![Page 15: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/15.jpg)
• h = F(X)• h1 = F(Xi1) = F(1)• h2 = F(Xi2) = F(-1)
27.01
1
1
1)2(
73.01
1
1
1)1(
)1(2
)1(1
xi
xi
XiF
XiF
![Page 16: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/16.jpg)
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
1
0
0.73
0.27
![Page 17: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/17.jpg)
Next Outputs
![Page 18: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/18.jpg)
Output Layer Computation
• X = hW2 = • 0.73 * -1 + 0.27 * 0 = -0.73, • 0.73 * 0 + 0.27 * -1 = -0.27 =• { -0.73 - 0.27} = {X1,X2} = X
xF
1
1
![Page 19: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/19.jpg)
• O = F(X)• O1 = F(X1)• O2 = F(X2)
433.01
1
1
1)2(
325.01
1
1
1)1(
)27.0(2
)73.0(1
x
x
XF
XF
![Page 20: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/20.jpg)
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
1
0
0.73
0.27
0.325
0.433
![Page 21: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/21.jpg)
Error
• D= Output(1 – Output)(Target – Output)• Target T1 = 0.7 , O1 = 0.325 = 0.33
• d1 = 0.33( 1 -0.33)(0.7 -0.33 ) = 0.33 (0.67)(0.37) = 0.082
• Target T2 = 0.6 , O2 = 0.433 = 0.43
• d2 = 0.43(1 - 0.43)(0.6-0.43) = 0.43(0.57)(0.17) = 0.42
![Page 22: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/22.jpg)
Weight Adjustment
• △W2t = α hd + Θ △W2t-1
• where α = 1• Time t = 1 so no previous time
2212
211121
2
1
dhdh
dhdhdd
h
hhd
)042.0*27.0()082.0*27.0(
)042.0*73.0()082.0*73.0(042.0082.0
27.0
73.0hd
![Page 23: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/23.jpg)
Weight Adjustments
)012.0()022.0(
)031.0()06.0(
![Page 24: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/24.jpg)
Weight Change
)012.02()22.02(
)031.02()06.02(
2221
1211
WW
WW
![Page 25: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/25.jpg)
Equals
)012.01()022.00(
)031.00()06.01(
![Page 26: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/26.jpg)
Equals
)988.0()022.0(
)031.0()94.0(
![Page 27: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/27.jpg)
Putting these new weights in the diagram
• To get
![Page 28: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/28.jpg)
I1
O2
O1H1
H2I2
W1 W2
0.031-1
0.022
-0.94
-0.988
0
1
1
![Page 29: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/29.jpg)
Next
• Calculate Change on W1 layer weights
![Page 30: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/30.jpg)
Error Calculatione = h(1 - h)W2d
21
11
h
h
2
1
d
d
2221
1211
22
22
WW
WW 21 hh
2
1
e
e
![Page 31: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/31.jpg)
Another Way to write the error
outputsk
kikiih dWhhe )1(
![Page 32: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/32.jpg)
What is this
• Outputs are O1 and O2
• So k = {1,2}• So if i = 1
outputsk
kikdW
summation
![Page 33: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/33.jpg)
2121112,1,1
dWdWdWdWki
kikoutputsk
kik
![Page 34: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/34.jpg)
I1
O2
O1H1
H2I2
W1 W2
0.031-1
0.022
-0.94
-0.988
0
1
1
![Page 35: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/35.jpg)
This equals
• e1 = (h1(1-h1)W11 D1 +W12D2• e2 = (h2(1-h2)) W21 D1 +W22D2• d1 = 0.082 d2 = = 0.042e1 = (0.73(1-0.73))( -1* 0.082 +0*0.042)• e2 =( 0.27(1-0.27)) (0 *0.082 +-1*0.042)
• e1 = (0.73(0.27)( -0.082))• e2 =( 0.27(0.73)) (-0.042)• e1 = -0.016• e2 = -0.0083
![Page 36: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/36.jpg)
Weight Adjustment
• △W1t = α Ie + Θ △W2t-1
• where α = 1
2212
211121
2
1
eIeI
eIeIee
I
IIe
)0083.0*0()016.0*0(
)0083.0*1()016.0*1(0083.0016.0
0
1Ie
![Page 37: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/37.jpg)
Weight Adjustment
)0()0(
)0083.0()016.0(
![Page 38: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/38.jpg)
Existing W1
10
11
11
11
2221
1211
WW
WW
![Page 39: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/39.jpg)
Weight Change W1
)01()0(
)0083.01()016.01(
![Page 40: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/40.jpg)
New W1
10
0083.1884.0
![Page 41: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/41.jpg)
• Changing Net
![Page 42: Hopefully a clearer version of Neural Network. I1 O2 O1 H1 H2I2.](https://reader035.fdocuments.net/reader035/viewer/2022062712/56649c755503460f949298b0/html5/thumbnails/42.jpg)
I1
O2
O1H1
H2I2
W1 W2
-0.102-1.0083
-0.04
-1.109
-1.038
0
0.884
1