[Paper Review] Efficient Architecture Search for Continual Learning

Qiang Gao, Zhipeng Luo+1 more|arXiv (Cornell University)|Jun 7, 2020

Domain Adaptation and Few-Shot Learning24 references4 citations

TL;DR

CLEAS proposes a novel continual learning framework that uses neuron-level neural architecture search (NAS) to dynamically expand neural networks with minimal complexity, reusing only the most useful old neurons and adding few new ones. It achieves state-of-the-art accuracy while reducing model complexity by up to 51% compared to SOTA methods, with zero catastrophic forgetting due to fixed old weights.

ABSTRACT

Continual learning with neural networks is an important learning framework in AI that aims to learn a sequence of tasks well. However, it is often confronted with three challenges: (1) overcome the catastrophic forgetting problem, (2) adapt the current network to new tasks, and meanwhile (3) control its model complexity. To reach these goals, we propose a novel approach named as Continual Learning with Efficient Architecture Search, or CLEAS in short. CLEAS works closely with neural architecture search (NAS) which leverages reinforcement learning techniques to search for the best neural architecture that fits a new task. In particular, we design a neuron-level NAS controller that decides which old neurons from previous tasks should be reused (knowledge transfer), and which new neurons should be added (to learn new knowledge). Such a fine-grained controller allows one to find a very concise architecture that can fit each new task well. Meanwhile, since we do not alter the weights of the reused neurons, we perfectly memorize the knowledge learned from previous tasks. We evaluate CLEAS on numerous sequential classification tasks, and the results demonstrate that CLEAS outperforms other state-of-the-art alternative methods, achieving higher classification accuracy while using simpler neural architectures.

Motivation & Objective

• To address the three core challenges in continual learning: catastrophic forgetting, effective adaptation to new tasks, and control of model complexity.
• To develop a dynamic network expansion strategy that minimally expands architecture while maintaining high performance on all tasks.
• To eliminate catastrophic forgetting by preserving all previously trained weights without fine-tuning.
• To design a NAS controller that operates at the neuron level for precise, efficient architecture selection.
• To reduce network complexity without sacrificing performance, especially in sequential learning scenarios.

Proposed method

• A reinforcement learning-based NAS controller operates at the neuron level, deciding which old neurons to reuse and how many new neurons to add for each new task.
• The controller uses the entire network configuration as its state, enabling more informed decisions than per-neuron state definitions.
• Old neuron weights are frozen, ensuring perfect knowledge retention and eliminating catastrophic forgetting.
• The method supports filter size optimization in CNNs, allowing dynamic adjustment of kernel sizes per task.
• Architecture search is performed per task, resulting in a compact, task-specific network that balances reuse and expansion.
• The approach is evaluated on sequential classification tasks, including MNIST permutations and CIFAR-100, with ablation on controller design and hyperparameter sensitivity.

Experimental results

Research questions

• RQ1 Can a neuron-level NAS controller achieve better continual learning performance with lower model complexity than existing methods?
• RQ2 Does maintaining fixed old weights eliminate catastrophic forgetting while still enabling effective adaptation to new tasks?
• RQ3 Can dynamic filter size adjustment in CNNs further improve performance without significantly increasing complexity?
• RQ4 How does the choice of controller state representation (entire network vs. single neuron) affect search performance?
• RQ5 To what extent is the proposed method robust to hyperparameter variations compared to prior approaches?

Key findings

• CLEAS improves classification accuracy by 0.21%, 0.21%, and 6.70% over the SOTA method RCL on three benchmark datasets.
• CLEAS reduces network complexity by 29.9%, 19.0%, and 51.0% compared to RCL across the same datasets.
• The controller’s use of the full network as a state improves performance by 0.31%, 0.29%, and 0.75% on the three datasets.
• CLEAS-C, which optimizes filter sizes, achieves 67.4% accuracy on CIFAR-100, a 0.7% improvement over CLEAS (66.9%), though with 92.6% higher complexity.
• Both CLEAS and RCL add a similar number of new neurons per task, but CLEAS reuses significantly fewer old neurons, indicating superior selection of useful prior knowledge.
• Hyperparameter sensitivity analysis shows CLEAS is less sensitive than DEN and RCL, maintaining top performance across all tested settings.