[Paper Review] uPredict: A User-Level Profiler-Based Predictive Framework for Single VM Applications in Multi-Tenant Clouds

Hamidreza Moradi, Wei Wang+2 more|arXiv (Cornell University)|Aug 13, 2019

Cloud Computing and Resource Management56 references7 citations

TL;DR

uPredict is a user-level performance prediction framework for single-VM applications in multi-tenant clouds that uses custom micro-benchmarks to profile CPU, memory, and disk contention in real time. By combining in-situ profiling with regression and neural network models, it achieves prediction errors of 9.8–17% on private clouds and under 4% on public clouds, enabling a 19% reduction in execution time via intelligent load balancing.

ABSTRACT

Most existing studies on performance prediction for virtual machines (VMs) in multi-tenant clouds are at system level and generally require access to performance counters in Hypervisors. In this work, we propose uPredict, a user-level profiler-based performance predictive framework for single-VM applications in multi-tenant clouds. Here, three micro-benchmarks are specially devised to assess the contention of CPUs, memory and disks in a VM, respectively. Based on measured performance of an application and micro-benchmarks, the application and VM-specific predictive models can be derived by exploiting various regression and neural network based techniques. These models can then be used to predict the application's performance using the in-situ profiled resource contention with the micro-benchmarks. We evaluated uPredict extensively with representative benchmarks from PARSEC, NAS Parallel Benchmarks and CloudSuite, on both a private cloud and two public clouds. The results show that the average prediction errors are between 9.8% to 17% for various predictive models on the private cloud with high resource contention, while the errors are within 4% on public clouds. A smart load-balancing scheme powered by uPredict is presented and can effectively reduce the execution and turnaround times of the considered application by 19% and 10%, respectively.

Motivation & Objective

• To address the challenge of performance unpredictability in multi-tenant cloud environments where users lack access to hypervisor-level metrics.
• To enable ordinary cloud users to predict application performance despite unknown co-located workloads and limited system visibility.
• To develop a practical, user-level predictive framework that operates without hypervisor access or prior knowledge of co-located VMs.
• To evaluate the framework’s accuracy and utility in real-world cloud environments, including private and public clouds.
• To demonstrate the framework’s effectiveness in improving application scheduling through predictive load balancing.

Proposed method

• Design and deploy three application-agnostic micro-benchmarks to probe CPU, memory, and disk contention in a target VM at runtime.
• Collect performance data from both the micro-benchmarks and the target application during repeated, sequential executions to capture dynamic contention.
• Train application- and VM-specific predictive models using regression (e.g., 2nd-degree polynomial) and neural network techniques on the collected profiling data.
• Optimize neural network hyper-parameters to improve prediction accuracy while balancing training overhead.
• Integrate the predictive model into a smart load-balancing scheme that selects VMs with minimal predicted contention.
• Evaluate the framework across diverse benchmarks (PARSEC, NAS, CloudSuite) on private and public cloud infrastructures (AWS, GCE).

Experimental results

Research questions

• RQ1 Can user-level micro-benchmarks accurately capture resource contention in a VM without hypervisor access?
• RQ2 How accurately can regression and neural network models predict application performance based on in-situ profiling of CPU, memory, and disk contention?
• RQ3 How does the prediction accuracy of uPredict vary across different cloud environments with varying contention levels?
• RQ4 Can uPredict-based predictive load balancing outperform traditional queue-based or heuristic load-balancing strategies in reducing execution and turnaround times?
• RQ5 What is the impact of profiling overhead and model training cost on the practicality of real-time performance prediction?

Key findings

• On private clouds with high resource contention, uPredict achieves average prediction errors between 9.8% and 17% across different predictive models.
• Neural network-based models with hyper-parameter optimization reduce prediction errors by approximately 9% compared to regression-based models, albeit with higher training overhead.
• On public clouds with lower and more stable contention, average prediction errors are consistently below 4%.
• The uPredict-powered load-balancing scheme reduces average execution time by 19% and turnaround time by 10% compared to a queue-based load-balancing strategy.
• The framework effectively captures performance fluctuations caused by dynamic, unknown co-located workloads, enabling accurate in-situ performance prediction for single-VM applications.
• The micro-benchmarks are effective for profiling system-level contention, but future work is needed to extend them to cache and network contention.