[Paper Review] Task Agnostic Continual Learning Using Online Variational Bayes

Chen Zeno, Itay Golan+2 more|arXiv (Cornell University)|Mar 27, 2018

Domain Adaptation and Few-Shot LearningComputer Science44 references70 citations

TL;DR

Introduces Bayesian Gradient Descent (BGD), a task-agnostic continual learning method that updates weight posteriors online without task boundaries, and shows the benefits of a labels trick for class learning.

ABSTRACT

Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, research for scenarios in which task boundaries are unknown during training has been lacking. In this paper we present, for the first time, a method for preventing catastrophic forgetting (BGD) for scenarios with task boundaries that are unknown during training --- task-agnostic continual learning. Code of our algorithm is available at https://github.com/igolan/bgd.

Motivation & Objective

• Define and categorize continual learning scenarios with unknown task boundaries.
• Present Bayesian Gradient Descent (BGD) as a task-agnostic online Bayesian update rule for neural networks.
• Introduce the labels trick to improve class-learning performance under task-agnostic conditions.
• Empirically evaluate BGD on continuous and discrete task-agnostic scenarios and compare to existing methods.

Proposed method

• Model the network weights with a diagonal Gaussian posterior q_n(θ|φ) across iterations.
• Derive closed-form online updates for μ (mean) and σ (std) of θ from online Bayes, using a single explicit Monte Carlo estimate.
• Use the reparameterization θ_i = μ_i + ε_i σ_i with ε_i ~ N(0,1) to compute unbiased gradients.
• Update rules: μ_i = m_i − σ_i^2 E[∂L_n/∂θ_i], and a derived σ_i update that incorporates the curvature-like term E[∂L_n/∂θ_i · ε_i].
• Approximate expectations via Monte Carlo with K samples and adjust convergence with a learning-rate η.
• Argue and show properties that learning rates depend on uncertainty σ_i and the observed curvature of the loss.

Experimental results

Research questions

• RQ1 How can continual learning be performed when task identities and boundaries are unknown at training and testing?
• RQ2 Can online Bayesian updates (BGD) mitigate catastrophic forgetting in task-agnostic settings without task switches?
• RQ3 Does the proposed labels trick improve performance specifically in class-learning scenarios under task-agnostic conditions?
• RQ4 What is the behavior of weight uncertainty (σ) over sequential tasks in task-agnostic continual learning?

Key findings

• BGD maintains high average accuracy in continuous task-agnostic scenarios (e.g., permuted MNIST) without task-switch information.
• In discrete task-agnostic settings, weight uncertainty concentrates on a small subset of parameters, supporting the idea that only some weights are task-relevant across tasks.
• BGD performs on-par with state-of-the-art methods (SI and VCL) in discrete task-agnostic setups, despite not using task-switch information.
• The labels trick significantly improves class-learning performance, boosting average accuracy from ~20% to ~50% on Split MNIST across several baselines.
• In class-learning with non-shared heads, the labels trick enables training only the relevant heads for existing labels in a batch, reducing interference.