Neural Networks: Prediction (i.e. the forward pass) · Based on slides and material from Geoffrey...

MachineLearning

NeuralNetworks:Prediction

(i.e.theforwardpass)

1BasedonslidesandmaterialfromGeoffreyHinton,RichardSocher,DanRoth,

Yoav Goldberg,ShaiShalev-Shwartz andShaiBen-David,andothers

NeuralNetworks

• Whatisaneuralnetwork?

• Predictingwithaneuralnetwork

• Trainingneuralnetworks

• Practicalconcerns

2

Thislecture

• Whatisaneuralnetwork?

• Predictingwithaneuralnetwork

• Trainingneuralnetworks

• Practicalconcerns

3

Letusconsideranexamplenetwork

4

Wewillusethisexamplenetwork

astointroducethegeneral

principleofhowtomake

predictionswithaneuralnetwork.


5

Namingconventionsforthis

example

• Inputs:x

• Hidden:z

• Output:y


6

Biasfeature,

always1


example

• Inputs:x

• Hidden:z

• Output:y


8

Biasfeature,

always1

Sigmoidactivations

Linearactivation


example

• Inputs:x

• Hidden:z

• Output:y


9

NamingConventionforWeights

!"#$%,'$'(#)*'_,(-*#


10

NamingConventionforWeights

!"#$%,'$'(#)*'_,(-*#

!./$Fromneuron#0

toneuron#1in

outputlayer

Howtopredictwithaneuralnetwork:The

forwardpass

11

Givenaninputx,howistheoutputpredicted

Theforwardpass

12

z/ = 2(!./4 + !//4 6/ + !7/4 67)


Theforwardpass

13

97 = 2(!.74 + !/74 6/ + !774 67)


z/ = 2(!./4 + !//4 6/ + !7/4 67)

Theforwardpass

14

y = !./$ + !//$ 9/ + !7/$ 97output


97 = 2(!.74 + !/74 6/ + !774 67)

z/ = 2(!./4 + !//4 6/ + !7/4 67)

Theforwardpass

15

y = !./$ + !//$ 9/ + !7/$ 97output


97 = 2(!.74 + !/74 6/ + !774 67)

z/ = 2(!./4 + !//4 6/ + !7/4 67)

Questions?

Ingeneral,beforevisiting(i.e.computing)thevalueofanode,visitallnodesthatserveasinputstoit.

Neural Networks: Prediction (i.e. the forward pass) · Based on slides and material from Geoffrey...

Documents

Transcript of Neural Networks: Prediction (i.e. the forward pass) · Based on slides and material from Geoffrey...