[Paper Review] MLOps: A Review

Samar Wazir, Gautam Siddharth Kashyap+1 more|arXiv (Cornell University)|Aug 19, 2023

Big Data and Business Intelligence23 citations

TL;DR

This paper surveys 22 studies on MLOps, compares tool stacks, and discusses challenges, capabilities, and future directions for operationalizing ML pipelines.

ABSTRACT

Recently, Machine Learning (ML) has become a widely accepted method for significant progress that is rapidly evolving. Since it employs computational methods to teach machines and produce acceptable answers. The significance of the Machine Learning Operations (MLOps) methods, which can provide acceptable answers for such problems, is examined in this study. To assist in the creation of software that is simple to use, the authors research MLOps methods. To choose the best tool structure for certain projects, the authors also assess the features and operability of various MLOps methods. A total of 22 papers were assessed that attempted to apply the MLOps idea. Finally, the authors admit the scarcity of fully effective MLOps methods based on which advancements can self-regulate by limiting human engagement.

Motivation & Objective

• Provide background on DevOps and MLOps and the ML lifecycle for readers new to the terms.
• Survey and synthesize findings from 22 MLOps-related papers to identify common approaches and limitations.
• Compare features and operability of various MLOps tools to guide tool selection for different projects.
• Discuss future directions and research gaps in the MLOps landscape.

Proposed method

• Review of 22 papers that applied the MLOps concept.
• Analysis of MLOps tool stacks and their features (pipeline, data, and model handling).
• Comparison of tools across feature criteria (HT, DV, PV, MD, CI/CD, PM, MEV).
• Evaluation of language and platform support for popular MLOps systems.
• Discussion of challenges, scalability, and directions for automated, low-human-intervention MLOps.

Experimental results

Research questions

• RQ1 What are the core components and lifecycle stages in MLOps, and how do they relate to DevOps?
• RQ2 Which MLOps tools and stacks are commonly used, and what features do they offer?
• RQ3 What are the major challenges and limitations preventing fully autonomous MLOps?
• RQ4 How do cloud providers and open-source platforms compare in supporting MLOps workflows?
• RQ5 What future directions are suggested to advance MLOps research and practice?

Key findings

• MLOps integrates ML with DevOps to automate and monitor ML lifecycles across model, code, and data components.
• A survey of 22 papers shows extensive use of tool stacks like Kubeflow, MLflow, Iterative, DataRobot, ClearML, MLReef, and cloud platforms such as AWS SageMaker, Azure ML, and Google Cloud offerings.
• Current MLOps tools offer various capabilities (CI/CD, data and model versioning, deployment, monitoring), but no single solution covers all needs effectively.
• There is an observed scarcity of fully effective MLOps methods that can self-regulate with minimal human involvement.
• MLOps faces challenges in data quality, scalability, reproducibility, and cost, motivating future research toward automated and self-managing pipelines.