[Paper Review] Neural Programmer-Interpreters

Scott Reed, Nando de Freitas|arXiv (Cornell University)|Jan 1, 2016

Advanced Neural Network Applications148 citations

TL;DR

The Neural Programmer-Interpreter (NPI) is a recurrent, compositional neural network that learns to represent and execute programs using a task-agnostic recurrent core, a persistent key-value program memory, and domain-specific encoders. It reduces sample complexity and improves generalization by composing lower-level programs, enabling a single model to learn 21 subprograms and execute addition, sorting, and 3D model canonicalization tasks from a small number of rich, fully-supervised execution traces.

ABSTRACT

Abstract: We propose the neural programmer-interpreter (NPI): a recurrent and compositional neural network that learns to represent and execute programs. NPI has three learnable components: a task-agnostic recurrent core, a persistent key-value program memory, and domain-specific encoders that enable a single NPI to operate in multiple perceptually diverse environments with distinct affordances. By learning to compose lower-level programs to express higher-level programs, NPI reduces sample complexity and increases generalization ability compared to sequence-to-sequence LSTMs. The program memory allows efficient learning of additional tasks by building on existing programs. NPI can also harness the environment (e.g. a scratch pad with read-write pointers) to cache intermediate results of computation, lessening the long-term memory burden on recurrent hidden units. In this work we train the NPI with fully-supervised execution traces; each program has example sequences of calls to the immediate subprograms conditioned on the input. Rather than training on a huge number of relatively weak labels, NPI learns from a small number of rich examples. We demonstrate the capability of our model to learn several types of compositional programs: addition, sorting, and canonicalizing 3D models. Furthermore, a single NPI learns to execute these programs and all 21 associated subprograms.

Motivation & Objective

• To develop a neural model that learns to represent and execute programs in a compositional and generalizable way.
• To reduce sample complexity in program learning by leveraging rich, fully-supervised execution traces instead of weak supervision.
• To enable efficient learning of new tasks by building on existing programs through a persistent key-value memory.
• To offload long-term computation from recurrent units by utilizing an external environment (e.g. scratch pad) for intermediate results.
• To demonstrate generalization across perceptually diverse environments using a single, unified model architecture.

Proposed method

• The NPI uses a task-agnostic recurrent core to process input sequences and maintain hidden states across program execution.
• It employs a persistent key-value program memory to store and retrieve intermediate program states and subprograms.
• Domain-specific encoders are used to condition the model on perceptually distinct environments, enabling transfer across different affordances.
• The model learns to compose lower-level programs into higher-level programs through supervised execution traces that show subprogram calls conditioned on inputs.
• Intermediate computation results are cached in an external environment (e.g. scratch pad with read-write pointers), reducing burden on recurrent hidden units.
• Training is performed using fully-supervised execution traces, where each program is associated with example sequences of subprogram calls.

Experimental results

Research questions

• RQ1 Can a single neural model learn to compose and execute multiple types of compositional programs across diverse perceptual environments?
• RQ2 Does using rich, fully-supervised execution traces reduce sample complexity compared to weak supervision?
• RQ3 Can a persistent program memory enable efficient learning of new tasks by reusing existing subprograms?
• RQ4 To what extent does offloading intermediate computation to an external environment improve model performance and generalization?
• RQ5 Can a unified model architecture generalize across tasks like addition, sorting, and 3D model canonicalization?

Key findings

• A single NPI model successfully learns to execute 21 distinct subprograms and their associated higher-level programs, including addition, sorting, and 3D model canonicalization.
• The model achieves improved generalization and reduced sample complexity by learning from a small number of rich, fully-supervised execution traces.
• The persistent key-value program memory enables efficient transfer learning, allowing new tasks to be learned by composing existing subprograms.
• Offloading intermediate computation to an external environment (e.g. scratch pad) reduces the long-term memory burden on the recurrent hidden units.
• The model generalizes across perceptually diverse environments due to domain-specific encoders that adapt the input representation without retraining the core.
• The compositional architecture enables the model to learn complex programs by composing simpler, reusable subprograms.