[Paper Review] The Case for Task Sampling based Learning for Cluster Job Scheduling

Akshay Jajoo, Yuanming Hu+2 more|ArXiv.org|Aug 24, 2021

Cloud Computing and Resource Management32 references4 citations

TL;DR

This paper proposes SLearn, a task-sampling-based learning approach for cluster job scheduling that improves runtime prediction accuracy by executing pilot tasks of a job to estimate its overall runtime, reducing average job completion time by 1.28× to 1.56× compared to history-based methods like 3Sigma, even under dynamic workloads and non-stationary job behaviors.

ABSTRACT

The ability to accurately estimate job runtime properties allows a scheduler to effectively schedule jobs. State-of-the-art online cluster job schedulers use history-based learning, which uses past job execution information to estimate the runtime properties of newly arrived jobs. However, with fast-paced development in cluster technology (in both hardware and software) and changing user inputs, job runtime properties can change over time, which lead to inaccurate predictions. In this paper, we explore the potential and limitation of real-time learning of job runtime properties, by proactively sampling and scheduling a small fraction of the tasks of each job. Such a task-sampling-based approach exploits the similarity among runtime properties of the tasks of the same job and is inherently immune to changing job behavior. Our study focuses on two key questions in comparing task-sampling-based learning (learning in space) and history-based learning (learning in time): (1) Can learning in space be more accurate than learning in time? (2) If so, can delaying scheduling the remaining tasks of a job till the completion of sampled tasks be more than compensated by the improved accuracy and result in improved job performance? Our analytical and experimental analysis of 3 production traces with different skew and job distribution shows that learning in space can be substantially more accurate. Our simulation and testbed evaluation on Azure of the two learning approaches anchored in a generic job scheduler using 3 production cluster job traces shows that despite its online overhead, learning in space reduces the average Job Completion Time (JCT) by 1.28x, 1.56x, and 1.32x compared to the prior-art history-based predictor.

Motivation & Objective

• Address the instability of history-based learning in predicting job runtime due to non-recurrent jobs and evolving system conditions.
• Overcome the limitations of relying on past job executions when job behavior changes over time due to hardware, software, or user script updates.
• Explore whether learning in the spatial dimension (via task sampling) can yield more accurate predictions than learning in the temporal dimension (via historical data).
• Evaluate whether the overhead of sampling pilot tasks is compensated by improved scheduling accuracy and reduced job completion time.
• Extend the sampling-based learning approach to support DAG workloads and assess its performance relative to history-based predictors.

Proposed method

• Proactively sample a small fraction of tasks (pilot tasks) from each incoming job and schedule them to run to completion before scheduling the remaining tasks.
• Use the observed runtime of pilot tasks to estimate the total runtime of the entire job, leveraging task similarity within the same job.
• Integrate the SLearn prediction model into a generic job scheduler that uses estimated job runtime to prioritize jobs via algorithms like Shortest Job First (SJF).
• Design a workload-aware scheduling policy that accounts for prediction error and resistance (delay due to pending jobs), minimizing job completion time.
• Implement and evaluate SLearn in both simulation and real testbed experiments on Azure using three production traces: two from Google (2011, 2019) and one from 2Sigma.
• Extend SLearn to DAG jobs by sampling representative tasks from each job phase and using their runtimes to predict end-to-end job completion times.

Experimental results

Research questions

• RQ1 Can learning in space (via task sampling) achieve higher prediction accuracy than learning in time (via historical data) for cluster job scheduling?
• RQ2 Does the performance gain from improved prediction accuracy outweigh the scheduling overhead introduced by delaying non-pilot tasks?
• RQ3 How does prediction error impact job resistance and completion time under different workloads?
• RQ4 Can the task-sampling approach be effectively extended to DAG-structured jobs with multiple phases?
• RQ5 Under what workload conditions does SLearn outperform state-of-the-art history-based predictors like 3Sigma?

Key findings

• SLearn reduces average job completion time (JCT) by 1.28×, 1.56×, and 1.32× compared to 3Sigma across three production traces, demonstrating consistent speedups.
• The prediction error of SLearn is significantly lower than 3Sigma: 30.65% (P50) vs. 898.5% (P50) for positive errors, and -26.79% (P50) vs. -37.0% (P50) for negative errors.
• SLearn misplaces only 2.85% of jobs in priority queues, compared to 12.19% under 3Sigma, due to higher prediction accuracy.
• During high workloads (e.g., 600–620s and 800–840s), SLearn achieves speedups over 3Sigma because its lower prediction error results in less resistance and better scheduling decisions.
• The resistance experienced by jobs under SLearn closely tracks the Oracle (ideal) resistance curve, while 3Sigma shows high spikes in resistance due to large positive prediction errors.
• In low-workload periods (e.g., 400–600s), SLearn does not hurt performance due to work conservation, meaning non-sampled tasks do not wait unnecessarily for pilot tasks.