Associative Learning. Simple Associative Network.

28
Associative Learning
  • date post

    15-Jan-2016
  • Category

    Documents

  • view

    262
  • download

    1

Transcript of Associative Learning. Simple Associative Network.

Page 1: Associative Learning. Simple Associative Network.

Associative Learning

Page 2: Associative Learning. Simple Associative Network.

Simple Associative Network

Page 3: Associative Learning. Simple Associative Network.

Banana Associator

Page 4: Associative Learning. Simple Associative Network.

Unsupervised Hebb Rule

Page 5: Associative Learning. Simple Associative Network.

Banana Recognition Example

Page 6: Associative Learning. Simple Associative Network.

Example

Page 7: Associative Learning. Simple Associative Network.

Problems with Hebb Rule

• Weights can become arbitrarily large

• There is no mechanism for weights to decrease

Page 8: Associative Learning. Simple Associative Network.

Hebb Rule with Decay

Page 9: Associative Learning. Simple Associative Network.

Example: Banana Associator

Page 10: Associative Learning. Simple Associative Network.

Example

Page 11: Associative Learning. Simple Associative Network.

Problem of Hebb with Decay

Page 12: Associative Learning. Simple Associative Network.

Instar (Recognition Network)

Page 13: Associative Learning. Simple Associative Network.

Instar Operation

Page 14: Associative Learning. Simple Associative Network.

Vector Recognition

Page 15: Associative Learning. Simple Associative Network.

Instar Rule

Page 16: Associative Learning. Simple Associative Network.

Graphical Representation

Page 17: Associative Learning. Simple Associative Network.

Example

Page 18: Associative Learning. Simple Associative Network.

Training

Page 19: Associative Learning. Simple Associative Network.

Further Training

Page 20: Associative Learning. Simple Associative Network.

Kohonen Rule

Page 21: Associative Learning. Simple Associative Network.

Outstar (Recall Network)

Page 22: Associative Learning. Simple Associative Network.

Outstar Operation

Page 23: Associative Learning. Simple Associative Network.

Outstar Rule

Page 24: Associative Learning. Simple Associative Network.

Example - Pineapple Recall

Page 25: Associative Learning. Simple Associative Network.

Definitions

Page 26: Associative Learning. Simple Associative Network.

Iteration 1

Page 27: Associative Learning. Simple Associative Network.

Convergence

Page 28: Associative Learning. Simple Associative Network.

Boltzmann Learning• Stochastic learning process with a recurrent structure• State of a neuron is +1 or –1 and some neurons are free (adaptive state)

and others are clamped (frozen state)• Boltzmann machine is characterized by an energy function

• Free neurons change state with probability:

• The learning rule is given by:

Where kjis the correlation with neurons in clamped states and

kj is the correlation with the neurons in a frozen state

j jk

jkkj xxwE 21

)/exp(1

1)(

TExxP

kkk

kjppw kjkjkj

Hidden

Z-1

Z-1

Z-1

Z-1

Delay

Visible

Clamped