[Paper Review] Machine Learning Operations: A Survey on MLOps Tool Support

Nipuni Tharushika Hewage, Dulani Meedeniya|arXiv (Cornell University)|Feb 21, 2022

Big Data and Business Intelligence36 citations

TL;DR

The paper surveys commercially available MLOps tool support, comparing features and usability, and discusses limitations and future research directions.

ABSTRACT

Machine Learning (ML) has become a fast-growing, trending approach in solution development in practice. Deep Learning (DL) which is a subset of ML, learns using deep neural networks to simulate the human brain. It trains machines to learn techniques and processes individually using computer algorithms, which is also considered to be a role of Artificial Intelligence (AI). In this paper, we study current technical issues related to software development and delivery in organizations that work on ML projects. Therefore, the importance of the Machine Learning Operations (MLOps) concept, which can deliver appropriate solutions for such concerns, is discussed. We investigate commercially available MLOps tool support in software development. The comparison between MLOps tools analyzes the performance of each system and its use cases. Moreover, we examine the features and usability of MLOps tools to identify the most appropriate tool support for given scenarios. Finally, we recognize that there is a shortage in the availability of a fully functional MLOps platform on which processes can be automated by reducing human intervention.

Motivation & Objective

• Motivate the need for MLOps by highlighting data science and DevOps integration challenges in ML projects.
• Survey commercially available MLOps platforms and their collaborative, iterative LIFECYCLE support.
• Analyze tool features, usability, and fit for different scenarios to guide platform selection.
• Identify limitations and automation gaps that hinder fully automated ML lifecycles.
• Provide future research directions and recommendations for improving MLOps tool ecosystems.

Proposed method

• Review and categorize widely used MLOps platforms (Kubeflow, MLFlow, Iterative Enterprise, DataRobot, Allegro/ClearML, MLReef, Streamlit) and cloud offerings.
• Compare platforms across key functionalities: data versioning, hyperparameter tuning, model/experiment/versioning, pipeline/versioning, CI/CD, deployment, and performance monitoring.
• Assess platform independence and programming language support for cross-ecosystem usability.
• Analyze strengths, weaknesses, and use-case applicability of each platform based on described features.
• Discuss current challenges in achieving automated, end-to-end MLOps and outline future research directions.

Experimental results

Research questions

• RQ1 What functionalities are provided by existing MLOps platforms for managing the ML lifecycle?
• RQ2 How do MLOps tools compare regarding data/version/model/experiment/pipeline versioning, CI/CD, deployment, and monitoring?
• RQ3 What are the limitations and gaps in current MLOps tool support that hinder full automation?
• RQ4 How do language and framework supports influence platform suitability for different teams?
• RQ5 What future research directions could advance the development of automated, user-friendly MLOps dashboards?

Key findings

• Many platforms offer components for ML lifecycle management, but none provide a fully automated end-to-end solution yet.
• Kubeflow lacks a dedicated CI/CD component, using pipelines to enable reproducible work plans instead.
• MLflow provides tracking, projects, models, and a registry but lacks built-in notebooks, user management, and full customization.
• Cloud-provider offerings (SageMaker, Azure ML, Google AI Platform) offer broad MLOps capabilities but can be costly and lack a single unified dashboard.
• Data-centric platforms (Iterative, ClearML, MLReef) strengthen data/experiment/model/versioning and collaboration but vary in scope and integration.
• Overall, there is a need for a unified, user-friendly MLOps dashboard to automate and streamline ML workflows.