[Paper Review] Improving Spark Application Throughput Via Memory Aware Task Co-location: A Mixture of Experts Approach

Vicent Sanz Marco, Ben Taylor+2 more|arXiv (Cornell University)|Oct 2, 2017

Cloud Computing and Resource Management58 references14 citations

TL;DR

This paper proposes a mixture-of-experts approach to predict Spark application memory usage by selecting from multiple pre-trained memory models at runtime, enabling accurate task co-location on shared hosts. It achieves 8.69x higher system throughput and 49% faster turnaround time compared to isolated execution, outperforming state-of-the-art co-location schemes by 1.28x and 1.68x respectively.

ABSTRACT

Data analytic applications built upon big data processing frameworks such as Apache Spark are an important class of applications. Many of these applications are not latency-sensitive and thus can run as batch jobs in data centers. By running multiple applications on a computing host, task co-location can significantly improve the server utilization and system throughput. However, effective task co-location is a non-trivial task, as it requires an understanding of the computing resource requirement of the co-running applications, in order to determine what tasks, and how many of them, can be co-located. In this paper, we present a mixture-of-experts approach to model the memory behavior of Spark applications. We achieve this by learning, off-line, a range of specialized memory models on a range of typical applications; we then determine at runtime which of the memory models, or experts, best describes the memory behavior of the target application. We show that by accurately estimating the resource level that is needed, a co-location scheme can effectively determine how many applications can be co-located on the same host to improve the system throughput, by taking into consideration the memory and CPU requirements of co-running application tasks. Our technique is applied to a set of representative data analytic applications built upon the Apache Spark framework. We evaluated our approach for system throughput and average normalized turnaround time on a multi-core cluster. Our approach achieves over 83.9% of the performance delivered using an ideal memory predictor. We obtain, on average, 8.69x improvement on system throughput and a 49% reduction on turnaround time over executing application tasks in isolation, which translates to a 1.28x and 1.68x improvement over a state-of-the-art co-location scheme for system throughput and turnaround time respectively.

Motivation & Objective

• To address the challenge of inefficient resource utilization in big data workloads by co-locating latency-insensitive Spark applications on shared hosts.
• To overcome the limitations of fixed resource prediction models that either require user-provided estimates or use one-size-fits-all functions.
• To improve system throughput and reduce application turnaround time through accurate runtime prediction of Spark application memory footprints.
• To enable dynamic, adaptive selection of specialized memory models based on application and data characteristics at runtime.
• To provide a scalable, extensible framework for modeling diverse Spark application memory behaviors without manual tuning.

Proposed method

• Train multiple specialized memory models (linear and non-linear) off-line on representative Spark applications using historical data.
• Use a machine learning classifier at runtime to select the most appropriate model (expert) for a given application and dataset.
• Dynamically compose predictions from selected experts to estimate the memory footprint of a target Spark application.
• Integrate the prediction framework with a runtime task scheduler to co-locate compatible tasks on the same host.
• Ensure co-location does not exceed physical memory limits by using accurate memory predictions to prevent out-of-memory errors.
• Support extensibility by allowing new models to be added and selected only when appropriate, enabling long-term adaptability.

Experimental results

Research questions

• RQ1 Can a mixture-of-experts framework improve the accuracy of memory footprint prediction for diverse Spark applications compared to monolithic models?
• RQ2 How does runtime selection of specialized memory models affect system throughput and application turnaround time in co-located environments?
• RQ3 To what extent can memory-aware task co-location reduce resource underutilization in Spark-based data centers?
• RQ4 How does the proposed approach compare to state-of-the-art co-location schemes in terms of throughput and latency?
• RQ5 Can the framework be extended to support new applications without retraining the entire model set?

Key findings

• The proposed method achieves 83.9% of the performance of an ideal memory predictor in terms of system throughput.
• On average, the approach delivers 8.69x higher system throughput compared to executing Spark tasks in isolation.
• The approach reduces average normalized turnaround time by 49% compared to isolated execution.
• It improves system throughput by 1.28x and turnaround time by 1.68x over a state-of-the-art co-location scheme.
• The mixture-of-experts framework enables accurate, adaptive memory prediction across 44 representative Spark applications on a 40-node cluster.
• The system supports incremental model addition, allowing the framework to evolve and support new application types over time.