[Paper Review] Optimal Continual Learning has Perfect Memory and is NP-hard

Jeremias Knoblauch, Hisham Husain+1 more|arXiv (Cornell University)|Jun 9, 2020

Domain Adaptation and Few-Shot Learning33 citations

TL;DR

The paper argues that optimal continual learning algorithms would typically solve an NP-hard problem and require perfect memory, explaining why experience replay and episodic memory aid continual learning.

ABSTRACT

Continual Learning (CL) algorithms incrementally learn a predictor or representation across multiple sequentially observed tasks. Designing CL algorithms that perform reliably and avoid so-called catastrophic forgetting has proven a persistent challenge. The current paper develops a theoretical approach that explains why. In particular, we derive the computational properties which CL algorithms would have to possess in order to avoid catastrophic forgetting. Our main finding is that such optimal CL algorithms generally solve an NP-hard problem and will require perfect memory to do so. The findings are of theoretical interest, but also explain the excellent performance of CL algorithms using experience replay, episodic memory and core sets relative to regularization-based approaches.

Motivation & Objective

• Motivate the investigation into why continual learning models suffer from catastrophic forgetting.
• Characterize the computational properties an optimal continual learning algorithm would need.
• Explain the empirical success of memory-based approaches relative to regularization-based methods.

Proposed method

• Define Idealized Continual Learning (Idealized CL) with algorithmic functions A_I and A_θ.
• Establish a formal framework for sequences of tasks and data distributions under a fixed criterion.
• Argue that optimal CL requires solving an NP-hard problem with perfect memory to avoid forgetting.

Experimental results

Research questions

• RQ1 What computational properties must an optimal continual learning algorithm possess to avoid catastrophic forgetting?
• RQ2 Do practical CL strategies (like replay, episodic memory, core sets) align with the properties of optimal CL?
• RQ3 Why do memory-based approaches outperform regularization-based approaches in continual learning?
• RQ4 Under what formal conditions would Idealized CL be achievable, given task sequences and data distributions?

Key findings

• Optimal continual learning algorithms generally solve an NP-hard problem.
• Such algorithms would require perfect memory to prevent forgetting.
• Memory-based methods (experience replay, episodic memory, core sets) provide better performance explanations than regularization-based methods.