[Paper Review] Generalized Inner Loop Meta-Learning

Edward Grefenstette, Brandon Amos+7 more|arXiv (Cornell University)|Oct 3, 2019

Domain Adaptation and Few-Shot Learning34 references88 citations

TL;DR

The paper formalizes generalized inner loop meta-learning as Gimli, derives a universal algorithm, and releases a PyTorch library higher to implement such nested optimization across various models and optimizers.

ABSTRACT

Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches. Based on this analysis and algorithm, we describe a library of our design, higher, which we share with the community to assist and enable future research into these kinds of meta-learning approaches. We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate.

Motivation & Objective

• Formalize the common pattern of nested inner-outer loop optimization in meta-learning and name it Gimli.
• Derive a general algorithm that enables exact, efficient implementation of Gimli-compatible methods.
• Develop and release a PyTorch library (higher) to facilitate easy implementation and experimentation of Gimli-based meta-learning approaches.
• Demonstrate practical applicability through illustrative experiments and ablations that highlight research directions enabled by the framework and library.

Proposed method

• Define a nested optimization framework where an outer set of meta-parameters governs inner-loop optimization of model parameters.
• Derive conditions under which gradient-based meta-learning is feasible and how to backpropagate through the inner loop (Gimli 2.4, 2.5).
• Present an exact Gimli update algorithm that unrolls inner loops and backpropagates higher-order gradients via a stop-gradient construction (Algorithm 1).
• Introduce and describe the higher library that makes stateful modules and differentiable optimizers workable for Gimli in PyTorch (monkey-patching, differentiable optimizers).
• Provide examples and discuss related work to illustrate Gimli-compatible meta-learning variants such as hyperparameter learning and MAML-style initializations.

Experimental results

Research questions

• RQ1 How can a unified formalism capture diverse nested-optimization meta-learning methods?
• RQ2 What are the exact gradient-based requirements to enable Gimli-style meta-training?
• RQ3 How can we implement a general, efficient Gimli update that is agnostic to model and optimizer choice?
• RQ4 Can a practical library be built to enable researchers to implement Gimli-compatible meta-learning with minimal code changes?

Key findings

• Gimli subsumes several recent meta-learning approaches under a single formal framework.
• A general, exact algorithm (Algorithm 1) enables differentiable, backpropagable meta-training through inner loops.
• The higher library enables implementing Gimli-compatible methods with minimal non-canonical PyTorch code changes.
• The framework supports ablation studies and experimentation on meta-learning components such as learning rates and loss parameterizations.
• The paper discusses practical considerations for differentiable optimization and stateful module handling to facilitate nested-optimization research.