[Paper Review] Posterior Meta-Replay for Continual Learning

Christian Henning, Maria R. Cervera+7 more|Zurich Open Repository and Archive (University of Zurich)|Mar 1, 2021

Domain Adaptation and Few-Shot Learning98 references4 citations

TL;DR

This paper proposes posterior meta-replay, a Bayesian continual learning framework that learns task-specific posterior parameter distributions via a shared probabilistic hypernetwork, avoiding recursive Bayesian updates that amplify approximation errors. It achieves near-zero forgetting on standard benchmarks and outperforms prior-focused Bayesian methods, with task inference via predictive uncertainty identified as the main performance bottleneck.

ABSTRACT

Learning a sequence of tasks without access to i.i.d. observations is a widely studied form of continual learning (CL) that remains challenging. In principle, Bayesian learning directly applies to this setting, since recursive and one-off Bayesian updates yield the same result. In practice, however, recursive updating often leads to poor trade-off solutions across tasks because approximate inference is necessary for most models of interest. Here, we describe an alternative Bayesian approach where task-conditioned parameter distributions are continually inferred from data. We offer a practical deep learning implementation of our framework based on probabilistic task-conditioned hypernetworks, an approach we term posterior meta-replay. Experiments on standard benchmarks show that our probabilistic hypernetworks compress sequences of posterior parameter distributions with virtually no forgetting. We obtain considerable performance gains compared to existing Bayesian CL methods, and identify task inference as our major limiting factor. This limitation has several causes that are independent of the considered sequential setting, opening up new avenues for progress in CL.

Motivation & Objective

• To address the limitations of recursive Bayesian updating in continual learning, which amplifies approximation errors and forces trade-offs between task performance.
• To develop a Bayesian continual learning framework that learns distinct task-specific posteriors instead of a single combined posterior.
• To enable effective knowledge compression and retention through a shared meta-model that conditions on task identity.
• To investigate the role of predictive uncertainty in task-agnostic inference, especially in class-incremental learning settings.
• To identify and analyze the primary bottleneck in performance, which is found to be task inference rather than the core CL mechanism.

Proposed method

• The method uses probabilistic task-conditioned hypernetworks to model task-specific posterior distributions over network weights, with a single shared meta-model.
• It employs a regularizer to replay parameters of previous task posteriors, preventing forgetting at the meta-learning level.
• The framework avoids recursive Bayesian updates by learning each task's posterior independently, thus eliminating error propagation.
• Task identity is inferred at test time using principled uncertainty estimates from Bayesian models, without generative modeling.
• The approach allows explicit control over knowledge transfer via freely chosen priors for each task, decoupling prior specification from previous task posteriors.
• The method is evaluated on standard continual learning benchmarks, with performance compared to prior-focused Bayesian baselines.

Experimental results

Research questions

• RQ1 Can a Bayesian continual learning framework that learns task-specific posteriors outperform recursive, prior-focused Bayesian methods in terms of forgetting and accuracy?
• RQ2 How effective is predictive uncertainty in enabling task-agnostic inference when task identity is not provided during testing?
• RQ3 What are the primary sources of performance degradation in the proposed framework, and are they inherent to the CL mechanism or related to inference components?
• RQ4 Can knowledge compression via a shared hypernetwork maintain high performance across a sequence of tasks without significant forgetting?
• RQ5 To what extent can the proposed method decouple plasticity and stability by allowing independent prior specification for each task?

Key findings

• The proposed posterior meta-replay framework achieves near-zero forgetting on standard continual learning benchmarks, significantly outperforming existing Bayesian continual learning methods.
• Performance gains are substantial compared to prior-focused Bayesian baselines, particularly in class-incremental learning settings.
• The main bottleneck in performance is identified as task inference at test time, not the core continual learning mechanism.
• Limitations in task inference are attributed to the combination of approximate inference, architecture, uncertainty measure, and prior—factors independent of the sequential learning setting.
• The framework demonstrates that explicit control over priors enables better knowledge transfer and avoids the stability-plasticity trade-off inherent in recursive Bayesian updates.
• Regularization via posterior replay effectively prevents forgetting at the meta-level, enabling stable compression of multiple task posteriors in a single hypernetwork.