Neural Turing Machines
89
-
Upload
ilya-kuzovkin -
Category
Science
-
view
660 -
download
4
description
Seminar overview of the third article produced by Google DeepMind. This one again contains conceptual novelties: adding external memory to machine learning pipeline (using an Artificial Neural Network as a Controller, which decides how to use this memory). System is differentiable, meaning that you can give it inputs, show the outputs it should produce, define an error-function (cross-entropy in this case) and then train the whole thing using gradient descent. The amazing outcome is that the system learns not the statistical relations between the input and the output as your usual ML, but attempts to learn an algorithm, which allows it to generalize well and perform correctly on problem instances which are bigger or different from what is has been trained on.
Transcript of Neural Turing Machines
We have introduced the Neural Turing Machine, a neural network
architecture that takes inspiration from both models of biological
working memory and the design of digital computers. Like
conventional neural networks, the architecture is differentiable
end-to-end and can be trained with gradient descent. Our
experiments demonstrate that it is capable of learning simple
algorithms from example data and of using these algorithms to
generalise well outside its training regime."
