[Paper Review] Continual Learning Using Multi-view Task Conditional Neural Networks

Haoyu Li, Payam Barnaghi+2 more|arXiv (Cornell University)|May 8, 2020

Domain Adaptation and Few-Shot Learning44 references4 citations

TL;DR

This paper proposes Multi-view Task Conditional Neural Networks (Mv-TCNN), a continual learning model that detects and adapts to reoccurring tasks without prior knowledge of task identities. By combining multi-view data augmentation with a probabilistic layer to estimate task likelihood, Mv-TCNN outperforms state-of-the-art methods on MNIST, CIFAR10, CIFAR100, and a real-world time-series healthcare dataset (TIHM), achieving superior accuracy and robustness in dynamic, changing environments.

ABSTRACT

Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning change, a continual model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. The changes in goals or data are referred to as new tasks in a continual learning model. Most of the continual learning methods have a task-known setup in which the task identities are known in advance to the learning model. We propose Multi-view Task Conditional Neural Networks (Mv-TCNN) that does not require to known the reoccurring tasks in advance. We evaluate our model on standard datasets using MNIST, CIFAR10, CIFAR100, and also a real-world dataset that we have collected in a remote healthcare monitoring study (i.e. TIHM dataset). The proposed model outperforms the state-of-the-art solutions in continual learning and adapting to new tasks that are not defined in advance.

Motivation & Objective

• Address catastrophic forgetting in continual learning where models forget previously learned tasks when exposed to new ones.
• Overcome the limitation of existing continual learning methods that require prior knowledge of task identities (task-known setup).
• Develop a scalable, adaptive model capable of detecting and learning reoccurring tasks in dynamic environments such as remote healthcare monitoring.
• Enable automatic task identification and expert selection without task labels during inference, enhancing adaptability in real-world applications.
• Improve performance on both image and numerical time-series data, including imbalanced and non-i.i.d. data distributions.

Proposed method

• Introduce a Multi-view Task Conditional Neural Network (Mv-TCNN) architecture with multiple task-specific experts for different learning tasks.
• Apply multi-view data augmentation to generate diverse, contextually relevant views of input data, enhancing feature robustness and generalization.
• Incorporate a probabilistic layer that estimates the likelihood of a given input belonging to each task-specific expert, enabling dynamic expert selection.
• Use the task likelihood score to determine which expert should be activated for a given input, allowing the model to infer task identity without explicit task labels.
• Train the model end-to-end using a combination of cross-entropy loss and task likelihood estimation to balance accuracy and stability.
• Leverage linear path analysis and loss surface visualization to demonstrate reduced catastrophic forgetting during sequential task learning.

Experimental results

Research questions

• RQ1 Can a continual learning model detect and adapt to reoccurring tasks without prior knowledge of task identities?
• RQ2 How does multi-view data augmentation improve the robustness and generalization of task-specific experts in continual learning?
• RQ3 To what extent does a probabilistic layer for task likelihood estimation enhance task identification and reduce catastrophic forgetting?
• RQ4 How does Mv-TCNN perform compared to state-of-the-art continual learning methods on both image and time-series datasets?
• RQ5 Can Mv-TCNN effectively handle imbalanced and non-stationary data distributions typical in real-world applications like remote healthcare monitoring?

Key findings

• Mv-TCNN outperforms state-of-the-art continual learning methods on MNIST, CIFAR10, CIFAR100, and the real-world TIHM time-series dataset, demonstrating superior accuracy and stability.
• The model achieves significantly better task likelihood estimation than baseline models, with in-task samples clearly distinguishable from out-task samples in the probabilistic layer.
• Abllation studies confirm that both multi-view augmentation and the probabilistic layer are essential for accurate task identity inference, with Mv-TCNN showing the lowest standard deviation and most accurate mean task likelihood for out-of-task samples.
• The selection rate of the correct expert increases with the number of tasks learned, indicating strong scalability and reliable inference under increasing task complexity.
• Mv-TCNN maintains high performance on data not suitable for multi-view augmentation by leveraging previously learned parameters, showing robustness in real-world deployment.
• The model effectively detects changes in data distribution and adapts to periodic or irregularly reoccurring tasks in remote healthcare monitoring, reducing the need for retraining or multiple model maintenance.