[Paper Review] Variational Continual Learning

Cuong V. Nguyen, Yingzhen Li+2 more|arXiv (Cornell University)|Oct 29, 2017

Domain Adaptation and Few-Shot Learning8 references295 citations

TL;DR

A principled Bayesian framework for continual learning that combines online variational inference with episodic memory to mitigate forgetting in both discriminative and generative neural models.

ABSTRACT

This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and entirely new tasks emerge. Experimental results show that VCL outperforms state-of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.

Motivation & Objective

• Develop a general online Bayesian framework for continual learning that naturally updates with new data without revisiting all past data.
• Extend online variational inference to neural networks in continual task settings.
• Incorporate a small episodic memory (coreset) to further reduce forgetting and preserve knowledge from past tasks.
• Demonstrate VCL on both discriminative models and deep generative models with automatic forgetting control.
• Show that VCL outperforms state-of-the-art continual learning methods on multiple benchmarks without requiring hyper-parameter tuning in its objective.

Proposed method

• Formulate continual learning as sequential Bayesian updating with an intractable posterior approximated by a variational distribution q_t(theta).
• Define a KL-divergence projection step that minimizes KL(q_t || (1/Z_t) q_{t-1} p(D_t|theta)).
• Incorporate an episodic memory (coreset) to refresh memory by decomposing the posterior into non-coreset and coreset contributions and performing sequential projections.
• Apply variational continual learning to deep discriminative networks with shared parameters and task-specific heads, using a Gaussian mean-field posterior and online VI.
• Extend the framework to deep generative models by integrating with variational auto-encoders and handling shared vs. task-specific components, using the same VI-based posterior updates.
• Provide a training objective equivalent to online variational free energy that includes a KL penalty term between successive posteriors.

Experimental results

Research questions

• RQ1 Can variational continual learning provide automatic control of forgetting without extensive hyper-parameter tuning?
• RQ2 How can online VI be adapted to sequential tasks in deep discriminative models with shared representations and task-specific outputs?
• RQ3 Does integrating a small episodic memory improve retention across tasks in VCL?
• RQ4 Can VCL be effectively extended to deep generative models like VAEs, maintaining knowledge across tasks?
• RQ5 How does VCL perform compared to existing continual learning methods on standard benchmarks?

Key findings

• VCL outperforms state-of-the-art continual learning methods on a variety of discriminative and generative tasks.
• The approach retains knowledge across sequential tasks with less forgetting due to the Bayesian posterior framework.
• Incorporating a coreset episodic memory provides additional gains, improving retention further.
• VAEs and other deep generative models can be trained continually under VCL with competitive memory of past tasks.
• VCL is hyper-parameter free in its objective, relying on the variational posterior updates rather than tuned regularization strengths.