[Paper Review] One-shot Learning with Memory-Augmented Neural Networks

Adam Santoro, Sergey Bartunov+3 more|arXiv (Cornell University)|May 19, 2016

Domain Adaptation and Few-Shot Learning17 references298 citations

TL;DR

This paper shows that memory-augmented neural networks (MANNs) can learn from very few examples and rapidly adapt via an external memory, enabling one-shot classification and regression. It introduces a memory access module focused on content-based retrieval and a Least Recently Used (LRU) memory writing mechanism.

ABSTRACT

Despite recent breakthroughs in the applications of deep neural networks, one setting that presents a persistent challenge is that of "one-shot learning." Traditional gradient-based networks require a lot of data to learn, often through extensive iterative training. When new data is encountered, the models must inefficiently relearn their parameters to adequately incorporate the new information without catastrophic interference. Architectures with augmented memory capacities, such as Neural Turing Machines (NTMs), offer the ability to quickly encode and retrieve new information, and hence can potentially obviate the downsides of conventional models. Here, we demonstrate the ability of a memory-augmented neural network to rapidly assimilate new data, and leverage this data to make accurate predictions after only a few samples. We also introduce a new method for accessing an external memory that focuses on memory content, unlike previous methods that additionally use memory location-based focusing mechanisms.

Motivation & Objective

• Demonstrate that memory-augmented neural networks can rapidly assimilate new data from only a few samples.
• Showcase a memory access mechanism that emphasizes content-based retrieval over location-based addressing.
• Evaluate meta-learning capabilities on Omniglot classification and Gaussian process-based regression tasks.
• Compare MANN performance with standard RNNs, LSTMs, and k-nearest neighbors across low-data regimes.

Proposed method

• Utilize a Neural Turing Machine (NTM)–style external memory with read/write heads controlled by a neural controller.
• Introduce a Least Recently Used Access (LRUA) memory writer that favors writing to least-used or most-recent memory slots based on a learned gate.
• Address memory with cosine similarity to retrieve relevant bindings for prediction.
• Structure tasks as episodes for meta-learning, where labels are provided in a temporally offset manner to require memory-based binding.
• Train end-to-end via gradient descent to minimize expected loss across a distribution of tasks (episodes).
• Evaluate on Omniglot for classification and Gaussian-process-sampled functions for regression.

Experimental results

Research questions

• RQ1 Can a memory-augmented neural network rapidly bind new data representations to labels after a single presentation?
• RQ2 Does content-based memory addressing with an LRUA writer improve one-shot learning over location-based addressing in NTMs?
• RQ3 How does a MANN perform meta-learning for classification and regression with sparse data compared to baselines?
• RQ4 What is the impact of memory interference across episodes on one-shot learning performance?

Key findings

• On Omniglot, the MANN achieves 82.8% first-episode accuracy and up to 98.1% after ten presentations for five-class episodes with one-hot labels.
• With five-character string labels, the MANN reaches 0% first-instance accuracy but 69.5–93.1% by the 2nd–6th instances across various setups, demonstrating rapid one-shot learning.
• The MANN outperforms feedforward nets, LSTMs, and kNN baselines in one-shot Omniglot classification across multiple configurations.
• Memory interference (not wiping memory between episodes) degrades performance, illustrating the need for controlled memory management to prevent cross-task contamination.
• Curriculum training progressively increases the number of classes per episode while maintaining high performance, indicating scalable one-shot learning能力 under the MANN framework.
• In regression on functions drawn from a Gaussian process, the MANN’s predictions track the underlying function and improve as more samples are stored in memory, with uncertainty increasing for distant inputs.