[Paper Review] SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training

Ahsan Ali, Syed Zawad+4 more|arXiv (Cornell University)|May 4, 2022

Cloud Computing and Resource Management4 citations

TL;DR

SMLT is a serverless framework that enables scalable, adaptive, and user-centric training of machine learning models by dynamically scheduling resources and overcoming inherent limitations of serverless platforms—such as statelessness, short function durations, and communication overhead. It achieves up to 8× faster training and 3× lower cost compared to state-of-the-art VM-based and serverless ML systems.

ABSTRACT

In today's production machine learning (ML) systems, models are continuously trained, improved, and deployed. ML design and training are becoming a continuous workflow of various tasks that have dynamic resource demands. Serverless computing is an emerging cloud paradigm that provides transparent resource management and scaling for users and has the potential to revolutionize the routine of ML design and training. However, hosting modern ML workflows on existing serverless platforms has non-trivial challenges due to their intrinsic design limitations such as stateless nature, limited communication support across function instances, and limited function execution duration. These limitations result in a lack of an overarching view and adaptation mechanism for training dynamics and an amplification of existing problems in ML workflows. To address the above challenges, we propose SMLT, an automated, scalable, and adaptive serverless framework to enable efficient and user-centric ML design and training. SMLT employs an automated and adaptive scheduling mechanism to dynamically optimize the deployment and resource scaling for ML tasks during training. SMLT further enables user-centric ML workflow execution by supporting user-specified training deadlines and budget limits. In addition, by providing an end-to-end design, SMLT solves the intrinsic problems in serverless platforms such as the communication overhead, limited function execution duration, need for repeated initialization, and also provides explicit fault tolerance for ML training. SMLT is open-sourced and compatible with all major ML frameworks. Our experimental evaluation with large, sophisticated modern ML models demonstrate that SMLT outperforms the state-of-the-art VM based systems and existing serverless ML training frameworks in both training speed (up to 8X) and monetary cost (up to 3X)

Motivation & Objective

• To address the limitations of existing serverless platforms in supporting dynamic, long-running ML training workflows.
• To enable automated, adaptive resource scheduling for ML tasks under user-specified deadlines and budget constraints.
• To eliminate communication overhead, function timeout limitations, and repeated initialization in serverless ML execution.
• To provide explicit fault tolerance and end-to-end workflow management for ML training in serverless environments.
• To deliver a fully compatible, open-source framework that supports all major ML frameworks.

Proposed method

• SMLT employs an automated, adaptive scheduling mechanism that dynamically allocates and scales resources based on real-time training dynamics and workload demands.
• It decomposes ML training into fine-grained, stateful serverless functions to manage long-running training processes beyond native function duration limits.
• The framework implements a lightweight coordination layer to enable inter-function communication and state persistence, reducing communication overhead.
• SMLT integrates fault tolerance by checkpointing model states and enabling recovery from failures without reprocessing entire training jobs.
• It supports user-defined constraints such as training deadlines and budget limits, adjusting resource allocation accordingly.
• The system is designed as an end-to-end pipeline that abstracts serverless complexities while maintaining compatibility with popular ML frameworks like PyTorch and TensorFlow.

Experimental results

Research questions

• RQ1 How can serverless computing be effectively adapted to support long-running, dynamic machine learning training workflows despite inherent platform limitations?
• RQ2 What scheduling and coordination mechanisms are required to minimize communication overhead and function initialization in serverless ML training?
• RQ3 Can a serverless framework achieve both superior training performance and cost efficiency compared to traditional VM-based and existing serverless ML systems?
• RQ4 How can user-specified constraints such as deadlines and budget limits be integrated into a fully automated and adaptive training system?
• RQ5 What architectural patterns enable fault tolerance and state management in stateless serverless environments for ML training?

Key findings

• SMLT achieves up to 8× faster training speed compared to state-of-the-art VM-based systems, significantly reducing time-to-train for large models.
• The framework reduces monetary costs by up to 3× compared to existing serverless ML training solutions and VM-based baselines.
• SMLT successfully mitigates the impact of function timeouts and statelessness by enabling stateful execution through coordinated function chains and checkpointing.
• The system maintains high compatibility with major ML frameworks, including PyTorch and TensorFlow, without requiring model or framework modifications.
• User-specified training deadlines and budget constraints are effectively met through dynamic resource allocation and adaptive scheduling.
• End-to-end evaluation confirms SMLT’s robustness and efficiency across diverse, large-scale modern ML models.