[Paper Review] Seeing Shapes in Clouds: On the Performance-Cost trade-off for Heterogeneous Infrastructure-as-a-Service

Gordon Inggs, David B. Thomas+2 more|arXiv (Cornell University)|Jun 22, 2015

Distributed systems and fault tolerance5 references3 citations

TL;DR

This paper proposes a formal mixed-integer linear programming (MILP) approach to optimize workload partitioning across heterogeneous Infrastructure-as-a-Service (IaaS) platforms—comprising CPUs, GPUs, and FPGAs—balancing performance and cost. By modeling task allocation as a multi-objective optimization problem, the MILP method achieves a Pareto-optimal trade-off, reducing latency by up to 110% and cost by over 50% compared to heuristic methods in a real-world option pricing workload.

ABSTRACT

In the near future FPGAs will be available by the hour, however this new Infrastructure as a Service (IaaS) usage mode presents both an opportunity and a challenge: The opportunity is that programmers can potentially trade resources for performance on a much larger scale, for much shorter periods of time than before. The challenge is in finding and traversing the trade-off for heterogeneous IaaS that guarantees increased resources result in the greatest possible increased performance. Such a trade-off is Pareto optimal. The Pareto optimal trade-off for clusters of heterogeneous resources can be found by solving multiple, multi-objective optimisation problems, resulting in an optimal allocation of tasks to the available platforms. Solving these optimisation programs can be done using simple heuristic approaches or formal Mixed Integer Linear Programming (MILP) techniques. When pricing 128 financial options using a Monte Carlo algorithm upon a heterogeneous cluster of Multicore CPU, GPU and FPGA platforms, the MILP approach produces a trade-off that is up to 110% faster than a heuristic approach, and over 50% cheaper. These results suggest that high quality performance-resource trade-offs of heterogeneous IaaS are best realised through a formal optimisation approach.

Motivation & Objective

• To address the challenge of efficiently partitioning computational workloads across heterogeneous IaaS platforms (CPUs, GPUs, FPGAs) to balance performance and cost.
• To formalize the performance-cost trade-off as a multi-objective optimization problem for heterogeneous infrastructure.
• To evaluate whether formal optimization (MILP) outperforms heuristic approaches in generating efficient, Pareto-optimal task allocations.
• To demonstrate the practical benefits of formal optimization in real-world HPC workloads, such as financial option pricing.
• To validate that heterogeneous IaaS platforms can deliver significant performance and cost advantages when intelligently partitioned.

Proposed method

• Model the performance and cost characteristics of each IaaS platform (CPU, GPU, FPGA) using predictive runtime models based on hardware metrics and usage rates.
• Formulate the workload partitioning problem as a mixed-integer linear programming (MILP) problem with objectives to minimize total cost and latency.
• Define decision variables for task allocation across platforms, incorporating non-linear effects such as fixed setup times and variable time quantum costs.
• Implement a heuristic approach that prioritizes platforms based on absolute latency and cost without considering non-linearities or time quantum effects.
• Use a benchmark workload of 128 Monte Carlo option pricing tasks across 16 heterogeneous IaaS platforms from AWS, GCE, and Azure.
• Validate model predictions against actual execution measurements to assess accuracy and practicality of the optimization approach.

Experimental results

Research questions

• RQ1 Can a formal multi-objective optimization approach generate a more efficient performance-cost trade-off than heuristic methods in heterogeneous IaaS?
• RQ2 How do non-linear factors such as fixed setup times and time quantum costs affect the effectiveness of heuristic versus formal optimization in workload partitioning?
• RQ3 To what extent can heterogeneous IaaS platforms (CPU, GPU, FPGA) outperform homogeneous platforms in terms of performance and cost efficiency?
• RQ4 How accurate are predictive models in estimating the latency and cost of task allocations across diverse IaaS platforms?
• RQ5 What is the impact of platform heterogeneity on the Pareto-optimal trade-off between performance and cost in real-world HPC workloads?

Key findings

• The MILP-based optimization approach achieved a 110% reduction in latency compared to the heuristic method in the best-case scenario.
• The MILP approach reduced total cost by over 50% compared to the heuristic approach under the same performance constraints.
• The heuristic method failed to utilize CPU platforms effectively due to its inability to model time quantum effects, while the MILP approach leveraged them for cost and performance gains.
• The model predictions for both approaches were within 10% of actual measured results, indicating strong predictive accuracy for practical use.
• The Pareto-optimal trade-off generated by MILP outperformed the heuristic in both median and upper cost bounds, demonstrating consistent superiority.
• Heterogeneous IaaS platforms collectively delivered significantly better performance and cost efficiency than any single platform type alone.