[Paper Review] Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Models

Rudy Semola, Vincenzo Lomonaco+1 more|arXiv (Cornell University)|Jun 14, 2022

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper introduces Continual-Learning-as-a-Service (CLaaS), a novel MLOps-enabled software service paradigm that enables on-demand, efficient, and stateful model adaptation using continual learning (CL) strategies. It implements CLaaS via 'Continual Brain,' a microservices-based system that outperforms traditional cumulative retraining by reducing training time and memory usage by up to 8×, demonstrating significant efficiency gains in real-world vision and fashion recognition tasks.

ABSTRACT

Predictive machine learning models nowadays are often updated in a stateless and expensive way. The two main future trends for companies that want to build machine learning-based applications and systems are real-time inference and continual updating. Unfortunately, both trends require a mature infrastructure that is hard and costly to realize on-premise. This paper defines a novel software service and model delivery infrastructure termed Continual Learning-as-a-Service (CLaaS) to address these issues. Specifically, it embraces continual machine learning and continuous integration techniques. It provides support for model updating and validation tools for data scientists without an on-premise solution and in an efficient, stateful and easy-to-use manner. Finally, this CL model service is easy to encapsulate in any machine learning infrastructure or cloud system. This paper presents the design and implementation of a CLaaS instantiation, called LiquidBrain, evaluated in two real-world scenarios. The former is a robotic object recognition setting using the CORe50 dataset while the latter is a named category and attribute prediction using the DeepFashion-C dataset in the fashion domain. Our preliminary results suggest the usability and efficiency of the Continual Learning model services and the effectiveness of the solution in addressing real-world use-cases regardless of where the computation happens in the continuum Edge-Cloud.

Motivation & Objective

• To address the lack of scalable, efficient, and user-friendly continual learning infrastructure for industrial ML deployment.
• To bridge the gap between research-driven continual learning and real-world MLOps workflows in dynamic, non-stationary environments.
• To enable low-cost, stateful model updates without requiring deep expertise in continual learning.
• To support real-time inference and continuous model adaptation in edge-to-cloud computing continua.

Proposed method

• Designing CLaaS as a service-oriented paradigm that integrates continual learning with continuous integration and deployment (CI/CD) pipelines.
• Implementing Continual Brain as a microservices-based instantiation of CLaaS, extending the Avalanche framework for modularity and scalability.
• Using replay-based CL strategies with fixed memory buffers to mitigate catastrophic forgetting while maintaining inference efficiency.
• Automating model retraining and validation through CI/CD pipelines, enabling on-demand, stateful updates without full retraining.
• Benchmarking CLaaS on two real-world datasets: CORe50 for object recognition and DeepFashion-C for fashion attribute and category prediction.
• Configuring hyperparameters such as memory_size (5000 for CORe50, 15000 for DeepFashion-C) to optimize performance-efficiency trade-offs.

Experimental results

Research questions

• RQ1 Can a service-oriented CLaaS architecture effectively support on-demand, efficient, and stateful model adaptation in real-world ML applications?
• RQ2 How does CLaaS compare to traditional cumulative retraining in terms of training time, memory usage, and predictive performance?
• RQ3 To what extent can CLaaS be integrated into existing MLOps and ML serving systems across edge-to-cloud deployments?
• RQ4 Can CLaaS maintain high predictive accuracy while significantly reducing computational and memory costs in non-stationary data environments?

Key findings

• The Replay CL strategy reduced training time and memory usage by up to 6× for 9 experiences and up to 8× for 10 experiences compared to cumulative retraining on the DeepFashion-C dataset.
• On the CORe50 dataset, CL-based strategies achieved predictive performance close to the upper bound while reducing computational cost significantly.
• The training time for the Replay CL strategy remained stable across increasing numbers of experiences, indicating scalability and efficiency.
• The CLaaS architecture enabled efficient, automated, and reproducible model updates without requiring full retraining, demonstrating feasibility for industrial MLOps pipelines.
• The system was successfully deployed and evaluated in two real-world domains—robotic object recognition and fashion attribute prediction—confirming its practical usability.
• The integration of CLaaS with existing ML serving systems and MLOps toolkits was seamless, supporting both edge and cloud deployment scenarios.