Neural Turing Machines

Abstract

Neural Turing Machines augment neural networks with differentiable external memory for efficient training via gradient descent, enabling them to learn algorithms like copying and sorting.

Introduction

Neural Turing Machines enhance recurrent neural networks with a large, addressable memory, making them differentiable computers capable of learning algorithmic tasks.

Working Memory Analogy

NTMs resemble human working memory by storing and manipulating information with an attentional process, allowing them to learn and execute simple programs.

Psychology and Neuroscience

Working memory research in psychology focuses on information manipulation and capacity limits, while neuroscience links it to prefrontal cortex and basal ganglia function.

Cognitive Science Models

Computational models of working memory, such as Hazy et al.'s, use gated memory slots and are relevant to NTM development, though they often lack sophisticated addressing mechanisms.

Cognitive Science and Linguistics

Cognitive science and linguistics historically used symbol-processing metaphors, but the connectionist revolution shifted focus to sub-symbolic processes, leading to debates about variable-binding and variable-length structure handling in neural networks.

Recursive Processing

Recursive processing of variable-length structures is considered a hallmark of human cognition and essential for cognitive flexibility, with ongoing debate about its evolutionary origins and uniqueness to language.

Recurrent Neural Networks

Recurrent neural networks possess dynamic state for context-dependent computation, and Long Short-Term Memory (LSTM) architectures address gradient vanishing/exploding problems using perfect integrators with programmable gates.

RNNs and Variable-Length Structures

Recurrent networks naturally process variable-length structures and are applied to tasks like speech recognition and machine translation, making explicit parse tree construction less critical.

Neural Turing Machine Architecture

A Neural Turing Machine consists of a neural network controller and a memory bank, interacting via differentiable read and write operations parameterized by 'heads' and constrained by an attentional focus mechanism.

Reading Mechanism

Reading in an NTM involves a weighted sum of memory locations, determined by a normalized weighting vector from a read head, which is differentiable with respect to memory and weighting.

Writing Mechanism

Writing in an NTM uses an erase and add operation, controlled by a weighting vector and an erase vector, allowing for selective memory updates that are differentiable.

Addressing Mechanisms

NTMs use a combination of content-based addressing, which matches keys to memory content, and location-based addressing, which allows for iteration and jumps, to produce weightings for memory access.

Focusing by Content

Content-based addressing produces a normalized weighting by comparing a controller-emitted key vector to memory vectors using a similarity measure, with a precision controlled by a strength parameter.

Focusing by Location

Location-based addressing shifts weightings using a rotational mechanism, blending previous and content-based weightings via an interpolation gate and applying a normalized shift distribution.

Addressing System Modes

The NTM addressing system combines content and location-based mechanisms to operate in modes that allow direct content access, content-based jumps, or pure location-based iteration, with a final sharpening step.

Controller Network

The NTM controller can be feedforward or recurrent (like LSTM), with recurrent controllers offering internal memory analogous to CPU registers, while feedforward controllers provide greater transparency but can create bottlenecks.

Experiments Overview

Preliminary experiments evaluate NTMs with feedforward and LSTM controllers on algorithmic tasks like copying and sorting, comparing their ability to learn compact programs that generalize beyond training data.

Copy Task

The copy task demonstrates that NTMs, with either controller type, learn to store and recall sequences significantly faster and to a lower cost than standard LSTMs, suggesting a qualitative advantage in bridging long time delays.

NTM vs LSTM Copy Task

NTM learns a copy algorithm by interacting with its memory, enabling it to generalize to longer sequences than LSTM.

Repeat Copy Task

NTM learns the repeat copy task faster than LSTM and generalizes better to longer sequences and more repetitions.

Linked List Task

NTM rapidly learns to navigate a linked list by combining content-based lookup with location-based offsetting, significantly outperforming LSTM.

Dynamic N-Grams Task

NTM uses its memory as a re-writable table to count transition statistics, achieving a performance advantage over LSTM in predicting sequences.

Sorting Task

The NTM is tested on its ability to sort data by priority, with the goal of observing if it can implement a binary heap sort.

Priority Sort Task Illustration and Memory Analysis

The NTM uses priorities to write data to memory and reads it in sorted order, demonstrating effective information processing and ordering.

Experimental Details for NTM and LSTM Training

The RMSProp algorithm with momentum was used for training, and gradient components were clipped, with NTM parameter count independent of memory size, unlike LSTMs.

Conclusion: Neural Turing Machine Capabilities

The NTM is a novel, fully differentiable neural network capable of learning and generalizing algorithms from data, showing promise for sequence processing tasks.