Neural Networks Chapter 8. 8.1 Feed-Forward Neural Networks.
-
date post
21-Dec-2015 -
Category
Documents
-
view
261 -
download
0
Transcript of Neural Networks Chapter 8. 8.1 Feed-Forward Neural Networks.
Neural Networks
Chapter 8
8.1 Feed-Forward Neural Networks
Figure 8.1 A fully connected feed-forward neural network
Node 1
Node 2
Node i
Node j
Node k
Node 3
Input Layer Output LayerHidden Layer
1.0
0.7
0.4
Wjk
Wik
W3i
W3j
W2i
W2j
W1i
W1j
Table 8.1 • Initial Weight Values for the Neural Network Shown in Figure 8.1
Wlj
Wli
W2j
W2i
W3j
W3i
Wjk
Wik
0.20 0.10 0.30 –0.10 –0.10 0.20 0.10 0.50
Equation 8.1
Neural Network Input Format
valueattribute possiblelargest theis uemaximumVal
attribute for the valuepossiblesmallest theis ueminimumVal
converted be to value theis lueoriginalVa
range interval [0,1] thein falling valuecomputed theis newValue
where
ueminimumValuemaximumVal
ueminimumVallueoriginalVanewValue
Neural Network Output Format
Equation 8.2
The Sigmoid Function
2.718282.by edapproximat logarithms natural of base theis
where
1
1)(
e
xexf
Figure 8.2 The sigmoid function
0.000
0.200
0.400
0.600
0.800
1.000
1.200
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
f(x)
x
8.2 Neural Network Training: A Conceptual View
Supervised Learning with Feed-Forward Networks
• Backpropagation Learning
• Genetic Learning
Table 8.2 • A Population of Weight Elements for the Network in Figure 8.1
PopulationElement
Wlj
Wli
W2j
W2i
W3j
W3i
Wjk
Wik
1 0.20 0.10 0.30 –0.10 –0.10 0.20 0.10 0.502 0.14 0.38 0.19 0.25 –0.17 0.27 0.11 0.543 0.20 0.10 0.38 –0.16 –0.16 0.24 0.12 0.534 0.23 0.10 0.39 –0.18 –0.17 0.26 0.15 0.54
Unsupervised Clustering with Self-Organizing Maps
Figure 8.3 A 3x3 Kohonen network with two input layer nodes
Output Layer
Input Layer
Node 2Node 1
8.3 Neural Network Explanation
• Sensitivity Analysis
• Average Member Technique
8.4 General Considerations
• What input attributes will be used to build the network? • How will the network output be represented?• How many hidden layers should the network contain?• How many nodes should there be in each hidden layer?• What condition will terminate network training?
Neural Network Strengths
• Work well with noisy data.• Can process numeric and categorical data.• Appropriate for applications requiring a time element.• Have performed well in several domains.• Appropriate for supervised learning and unsupervised clustering.
Weaknesses
• Lack explanation capabilities.• May not provide optimal solutions to problems.• Overtraining can be a problem.
8.5 Neural Network Training: A Detailed View
The Backpropagation Algorithm: An Example
Equation 8.3
Backpropagation Error Output Layer
k nodeat function sigmoid theinput to the
function sigmoid theof derivativeorder -first The)('
erroroutput actual The)(
k nodeat output computed The
output target The
where
)]('[)()(
k
k
k
k
kk
x
xf
OT
O
T
xfOTkError
Equation 8.4
Backpropagation Error Output Layer
)1()()( kkk OOOTkError
Equation 8.5
Backpropagation Error Hidden Layer
)1(
toevaluates )(' 8.3, Eq.in As j. nodeat function sigmoid theinput to The
function sigmoid theof derivativeorder -first The)('
k nodeoutput and j node betweenlink the withassociated weight The
k nodeat error output computed The)(
where
)(')()(
jj
jj
j
jk
jk
jk
OO
xfx
xf
W
kError
xfWkErrorjError
Equations 8.6 and 8.7
The Delta Rule
jkjkjk wcurrentwneww )()(
j node ofoutput The
k nodeat error computed The)(
01 withparameter rate learning The
where
))](()[(
j
jjk
O
kError
rr
OkErrorrw
Equation 8.8
Root Mean Squared Error
nodeoutput th and instance th for theoutput computed the
nodeoutput th theand instance nth for theoutput target the
nodesoutput ofnumber total the
instancesset trainingofnumber totalthe
where
)(
inO
iT
i
n
ni
n iOT
in
in
inin
Kohonen Self-Organizing Maps: An Example
Figure 8.4 Connections for two output layer nodes
Node 1
Node 2 Node j
Input Layer Output Layer
0.4
0.7
W1i = .2Node i
W2j = .6
W1j = .3
W2i = .1
Equation 8.9
Classifying a New Instance Output Node = j
jnodeoutputatnodeinputthithewithassociatedweighttheisijw
inodeinputatvalueattributetheisin
ijwi in2)(
Equation 8.10
Adjusting the Weight Vectors Output Node = j
10
)(
where
)()(
r
wnrw
wcurrentwneww
ijiij
ijijij
Building Neural Networks with iDA
Chapter 9
9.1 A Four-Step Approach for Backpropagation Learning
1. Prepare the data to be mined.
2. Define the network architecture.
3. Watch the network train.
4. Read and interpret summary results.
Example 1: Modeling the Exclusive-OR Function
Table 9.1 • The Exclusive-OR Function
Input 1 Input 2 XOR
1 1 00 1 11 0 10 0 0
Figure 9.1A graph of the XOR function
0
0.2
0.4
0.6
0.8
1
1.2
0 0.2 0.4 0.6 0.8 1 1.2
Input 2
Input 1
A B
AB
Step 1: Prepare The Data To Be Mined
Figure 9.2 XOR training data
Step 2: Define The Network Architecture
Figure 9.3 Dialog box for supervised learning
Figure 9.4 Training options for backpropagation learning
Step 3: Watch The Network Train
Figure 9.5 Neural network execution window
Step 4: Read and Interpret Summary Results
Figure 9.6 XOR output file for Experiment 1
Figure 9.7 XOR output file for Experiment 2
Example 2: The Satellite Image Dataset
Step 1: Prepare The Data To Be Mined
Figure 9.8 Satellite image data
Step 2: Define The Network Architecture
Figure 9.9 Backpropagation learning parameters for the satellite image data
Step 3: Watch The Network Train
Step 4: Read And Interpret Summary Results
Figure 9.10 Statistics for the satellite image data
Figure 9.11 Satellite image data: Actual and computed output
9.2 A Four-Step Approach for Neural Network Clustering
Step 1: Prepare The Data To Be Mined
The Deer Hunter Dataset
Step 2: Define The Network Architecture
Figure 9.12 Learning parameters for unsupervised clustering
Step 3: Watch The Network Train
Figure 9.13 Network execution window
Step 4: Read And Interpret Summary Results
Figure 9.14 Deer hunter data: Unsupervised summary statistics
Figure 9.15 Output clusters for the deer hunter dataset
9.3 ESX for Neural Network Cluster Analysis