[Paper Review] Learning where to learn: Gradient sparsity in meta and continual learning

Johannes von Oswald, Dominic Zhao+5 more|Zurich Open Repository and Archive (University of Zurich)|Jan 1, 2021

Domain Adaptation and Few-Shot Learning24 citations

TL;DR

The paper introduces sparse-MAML, a meta-learning method that learns which weights to update via per-parameter gradient masking, producing sparse gradient descent that improves generalization and reduces interference in few-shot and continual learning across architectures and tasks.

ABSTRACT

Finding neural network weights that generalize well from small datasets is difficult. A promising approach is to learn a weight initialization such that a small number of weight changes results in low generalization error. We show that this form of meta-learning can be improved by letting the learning algorithm decide which weights to change, i.e., by learning where to learn. We find that patterned sparsity emerges from this process, with the pattern of sparsity varying on a problem-by-problem basis. This selective sparsity results in better generalization and less interference in a range of few-shot and continual learning problems. Moreover, we find that sparse learning also emerges in a more expressive model where learning rates are meta-learned. Our results shed light on an ongoing debate on whether meta-learning can discover adaptable features and suggest that learning by sparse gradient descent is a powerful inductive bias for meta-learning systems.

Motivation & Objective

• Investigate whether learning where to learn (via gradient masking) yields better generalization than standard MAML.
• Analyze how emergent gradient sparsity patterns vary across layers and tasks.
• Examine the impact of sparse gradient updates on few-shot performance, cross-domain adaptation, and continual learning.
• Compare sparse-MAML and its variants to existing gradient modulation methods.
• Explore scalability to deeper models and online continual-learning settings.

Proposed method

• Introduce a binary gradient mask m that modulates inner-loop updates through phi_{tau,k+1}=phi_{tau,k}-alpha * (1_{m>=0} ∘ ∇_phi L_in_tau(phi_{tau,k})).
• Update m via a first-order meta-gradient that aligns outer-loss gradient with the accumulated inner-loop gradients.
• Use straight-through estimator to backprop through the non-differentiable mask.
• Apply sparse-MAML to 5-way miniImageNet with ConvNet and ResNet-12, evaluating 1-shot and 5-shot regimes.
• Compare against MAML, ANIL, BOIL, Meta-SGD, MT-nets, and variations with sparse-ReLU or binary masks.
• Extend the approach to continual learning by combining with La-MAML and sparse-La-MAML (and sparse-ReLU variants).

Experimental results

Research questions

• RQ1 Does learning which weights to adapt (via gradient masks) lead to better few-shot generalization than standard MAML or ANIL?
• RQ2 What sparsity patterns emerge across layers and tasks, and how do they relate to network depth and data regime?
• RQ3 Do sparse gradient updates reduce interference in cross-domain adaptation and continual learning?
• RQ4 How do sparse-MAML variants (binary masks vs rectified learning rates) compare in performance and sparsity?

Key findings

• Sparse-MAML induces gradient sparsity that increases in earlier layers and decreases toward deeper layers, correlating with better few-shot performance.
• Higher inner-loop steps and larger inner-loop learning rates promote more sparsity and improved generalization.
• Binary gradient masks and rectified learning-rate variants can both achieve strong performance, with sparse-ReLU-MAML performing especially well on deeper models like ResNet-12.
• Sparse learning improves cross-domain adaptation (miniImageNet to TieredImageNet, CUB, Cars) over hand-crafted freezing baselines.
• In continual learning, sparse-La-MAML and sparse-ReLU-C-MAML maintain or improve retained accuracy and reduce interference across tasks, with sparsity patterns adapting to capacity and task structure.
• Overall, sparse gradient descent acts as a robust inductive bias that improves generalization and reduces forgetting across few-shot and continual learning settings.