[Paper Review] Assessor-Guided Learning for Continual Environments

M. Anwar Ma’sum, Mahardhika Pratama+3 more|arXiv (Cornell University)|Mar 21, 2023

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes Assessor-Guided Learning Approach (AGLA), a meta-learning-based continual learning framework that uses a sequence-aware assessor to dynamically soft-weight samples and loss functions (cross-entropy, DER, knowledge distillation) to balance plasticity and stability. AGLA achieves state-of-the-art performance in both class-incremental and task-incremental learning with improved accuracy and reduced forgetting, especially under limited memory and class-imbalanced conditions via its novel compensated over-sampling (COS) strategy.

ABSTRACT

This paper proposes an assessor-guided learning strategy for continual learning where an assessor guides the learning process of a base learner by controlling the direction and pace of the learning process thus allowing an efficient learning of new environments while protecting against the catastrophic interference problem. The assessor is trained in a meta-learning manner with a meta-objective to boost the learning process of the base learner. It performs a soft-weighting mechanism of every sample accepting positive samples while rejecting negative samples. The training objective of a base learner is to minimize a meta-weighted combination of the cross entropy loss function, the dark experience replay (DER) loss function and the knowledge distillation loss function whose interactions are controlled in such a way to attain an improved performance. A compensated over-sampling (COS) strategy is developed to overcome the class imbalanced problem of the episodic memory due to limited memory budgets. Our approach, Assessor-Guided Learning Approach (AGLA), has been evaluated in the class-incremental and task-incremental learning problems. AGLA achieves improved performances compared to its competitors while the theoretical analysis of the COS strategy is offered. Source codes of AGLA, baseline algorithms and experimental logs are shared publicly in \url{https://github.com/anwarmaxsum/AGLA} for further study.

Motivation & Objective

• To address catastrophic forgetting and class imbalance in continual learning under limited memory budgets.
• To develop a meta-learning-based assessor that guides the base learner by dynamically weighting samples and loss functions.
• To improve generalization and stability in continual learning by balancing plasticity and stability per sample.
• To reduce reliance on large memory replay by introducing a data-augmentation strategy that corrects for out-of-distribution samples.
• To enable effective continual learning in low-shot and privacy-constrained settings, including few-shot and federated scenarios.

Proposed method

• An assessor network is trained via meta-learning to predict sample weights that reflect the potential for positive forward and backward transfer.
• The base learner's loss is a meta-weighted combination of cross-entropy, dark experience replay (DER), and knowledge distillation losses, with weights determined by the assessor.
• A compensated over-sampling (COS) mechanism is introduced to address class imbalance in episodic memory by correcting synthetic samples during augmentation.
• The assessor uses a soft-weighting mechanism to assign higher influence to high-quality samples (low loss, positive transfer) and lower influence to poor samples (high loss, catastrophic forgetting risk).
• The method integrates sequence awareness, allowing the assessor to consider temporal context when evaluating sample utility across tasks.
• Theoretical complexity analysis shows AGLA’s computational cost is O(eN), where e is the number of epochs and N is total data size, scaling linearly with data.

Experimental results

Research questions

• RQ1 Can a meta-learned assessor effectively guide continual learning by dynamically weighting samples and loss functions to improve performance and reduce forgetting?
• RQ2 How does the proposed compensated over-sampling (COS) strategy mitigate class imbalance in episodic memory without introducing out-of-distribution samples?
• RQ3 What is the impact of each component (assessor, COS, loss weighting) on overall performance, as shown by ablation studies?
• RQ4 How does AGLA perform under limited memory budgets compared to existing regularization and memory-based continual learning methods?
• RQ5 Can AGLA generalize to few-shot and unsupervised continual learning settings, where labeled data is scarce or unavailable?

Key findings

• AGLA achieves state-of-the-art accuracy across four continual learning benchmarks in both class-incremental and task-incremental settings, with noticeable margins over ten competing methods.
• The method demonstrates the lowest average forgetting index among non-memory-based approaches and is second only to memory-based methods, indicating strong stability.
• Ablation studies show that removing the assessor or COS leads to major performance drops, confirming their critical role in the framework.
• AGLA maintains strong performance even with minimal memory budgets, indicating efficiency and scalability.
• The theoretical complexity of AGLA is O(eN), and with a constant number of epochs, it scales linearly with data size, making it computationally efficient.