[Paper Review] Meta Continual Learning via Dynamic Programming

Krishnan Raghavan, Prasanna Balaprakash|arXiv (Cornell University)|Aug 5, 2020

Domain Adaptation and Few-Shot Learning21 references4 citations

TL;DR

This paper proposes a dynamic programming-based theoretical framework for meta continual learning (MCL) that models catastrophic forgetting and generalization as interdependent costs. By applying Bellman's principle of optimality, the authors derive DPMCL—a stochastic-gradient-driven method that alternately minimizes these costs, achieving state-of-the-art performance on benchmark datasets with improved balance between retention and generalization.

ABSTRACT

Meta continual learning algorithms seek to train a model when faced with similar tasks observed in a sequential manner. Despite promising methodological advancements, there is a lack of theoretical frameworks that enable analysis of learning challenges such as generalization and catastrophic forgetting. To that end, we develop a new theoretical approach for meta continual learning~(MCL) where we mathematically model the learning dynamics using dynamic programming, and we establish conditions of optimality for the MCL problem. Moreover, using the theoretical framework, we derive a new dynamic-programming-based MCL method that adopts stochastic-gradient-driven alternating optimization to balance generalization and catastrophic forgetting. We show that, on MCL benchmark data sets, our theoretically grounded method achieves accuracy better than or comparable to that of existing state-of-the-art methods.

Motivation & Objective

• To address the lack of theoretical frameworks for analyzing catastrophic forgetting and generalization in meta continual learning (MCL).
• To develop a theoretically grounded MCL approach that does not require full task distribution knowledge in advance.
• To enable dynamic adaptation to changing data distributions by avoiding fixed representations.
• To establish optimality conditions for MCL using dynamic programming and optimal control theory.
• To empirically validate the method on standard MCL benchmarks with improved generalization and retention.

Proposed method

• Formulates the MCL problem as a dynamic programming optimization over a continuous-time horizon, using Bellman's principle of optimality to decompose the cost function.
• Defines two key cost components: catastrophic forgetting cost (evaluated on previous tasks via task memory) and generalization cost (evaluated on the new task).
• Uses stochastic gradient descent with alternating optimization to minimize the sum of both costs over multiple iterations.
• Applies the framework to discrete-time MCL by discretizing the continuous-time theory, enabling practical implementation.
• Employs an experience replay buffer to store and replay samples from previous tasks to compute forgetting costs.
• Derives existing MCL methods (e.g., MAML, OML, CML) as special cases of the proposed theoretical framework.

Experimental results

Research questions

• RQ1 Can a dynamic programming framework be used to model and optimize the trade-off between catastrophic forgetting and generalization in meta continual learning?
• RQ2 What theoretical conditions ensure optimality in the MCL setting under sequential task exposure?
• RQ3 How can a method be designed to balance forgetting and generalization without requiring prior knowledge of the full task distribution?
• RQ4 To what extent does the proposed DPMCL method outperform existing SOTA methods in terms of accuracy and stability across tasks?
• RQ5 Can the theoretical framework explain or unify existing MCL approaches such as MAML and OML?

Key findings

• The DPMCL method achieves a cumulative error (CME) of 0.020 ± 0.001 on MNIST, outperforming OML (0.023 ± 0.001) and ER (0.030 ± 0.001).
• On CIFAR10, DPMCL achieves a CME of 0.496 ± 0.003, significantly better than OML (0.676 ± 0.006) and ER (0.464 ± 0.002).
• For the SINE dataset, DPMCL achieves a CME of 10⁻⁵ (0), matching the best-performing methods and outperforming OML (10⁻⁴) and ER (10⁻⁵).
• On the OMNI dataset, DPMCL achieves a CME of 0.171 ± 0.007, outperforming OML (0.224 ± 0.010) and ER (0.194 ± 0.008).
• The method demonstrates robustness across diverse benchmarks, with consistently low new task error (NTE), including 0.003 ± 0 for MNIST and 0.231 ± 0.008 for CIFAR10.
• Theoretical analysis confirms that prior MCL methods like MAML and OML are special cases of the proposed framework, validating its generality and unifying potential.