Neural Turing Machines

Post on 06-Jul-2015

661 views 4 download

Tags:

description

Seminar overview of the third article produced by Google DeepMind. This one again contains conceptual novelties: adding external memory to machine learning pipeline (using an Artificial Neural Network as a Controller, which decides how to use this memory). System is differentiable, meaning that you can give it inputs, show the outputs it should produce, define an error-function (cross-entropy in this case) and then train the whole thing using gradient descent. The amazing outcome is that the system learns not the statistical relations between the input and the output as your usual ML, but attempts to learn an algorithm, which allows it to generalize well and perform correctly on problem instances which are bigger or different from what is has been trained on.

Transcript of Neural Turing Machines

We have introduced the Neural Turing Machine, a neural network

architecture that takes inspiration from both models of biological

working memory and the design of digital computers. Like

conventional neural networks, the architecture is differentiable

end-to-end and can be trained with gradient descent. Our

experiments demonstrate that it is capable of learning simple

algorithms from example data and of using these algorithms to

generalise well outside its training regime."