[Paper Review] A Cost-Effective Workload Allocation Strategy for Cloud-Native Edge Services

Valentino Armani, Francescomaria Faticanti+3 more|arXiv (Cornell University)|Oct 25, 2021

Cloud Computing and Resource Management4 citations

TL;DR

This paper proposes a cost-effective workload allocation strategy for cloud-native edge services in hybrid cloud environments using a novel heuristic based on replicasArray to balance load and minimize public cloud costs. The approach reduces costs by over 50% compared to load-balancing-focused solutions while maintaining acceptable performance under dynamic workloads, validated through simulations and real Kubernetes deployments.

ABSTRACT

Nowadays IoT applications consist of a collection of loosely coupled modules, namely microservices, that can be managed and placed in a heterogeneous environment consisting of private and public resources. It follows that distributing the application logic introduces new challenges in guaranteeing performance and reducing costs. However, most existing solutions are focused on reducing pay-per-use costs without considering a microservice-based architecture. We propose a cost-effective workload allocation for microservice-based applications. We model the problem as an integer programming problem and we formulate an efficient and near-optimal heuristic solution given the NP-hardness of the original problem. Numerical results demonstrate the good performance of the proposed heuristic in terms of cost reduction and performance with respect to optimal and state-of-the-art solutions. Moreover, an evaluation conducted in a Kubernetes cluster running in an OpenStack ecosystem confirms the feasibility and the validity of the proposed solution.

Motivation & Objective

• To address the challenge of minimizing public cloud costs in hybrid cloud environments while meeting application performance constraints.
• To enable efficient workload orchestration in microservice-based, cloud-native edge applications across heterogeneous private and public cloud resources.
• To develop a scalable, real-time solution that dynamically redirects traffic based on cost and load balancing criteria.
• To validate the heuristic's performance against optimal and state-of-the-art solutions in both simulated and real-world Kubernetes/OpenStack environments.
• To enable elastic adaptation to workload variations through dynamic region activation and deactivation.

Proposed method

• Formulates the workload allocation problem as an integer programming model equivalent to the Virtual Network Embedding (VNE) problem.
• Proposes a novel heuristic algorithm driven by two criteria: cost reduction and load balancing across microservices.
• Introduces the concept of 'replicasArray'—a centralized traffic-splitting mechanism that distributes requests based on residual host capacity.
• Employs a dynamic service reconfiguration mechanism (DSR) that activates or deactivates regions based on real-time performance and cost thresholds.
• Uses simulation and real-world evaluation in a Kubernetes cluster within an OpenStack environment to validate the heuristic.
• Applies runtime estimation and adaptive scheduling to adjust deployment configurations in response to workload changes.

Experimental results

Research questions

• RQ1 How can public cloud costs be minimized in a hybrid cloud environment without compromising application performance?
• RQ2 What is the impact of dynamic traffic redirection on cost and performance in microservice-based edge applications?
• RQ3 How effective is a replicasArray-based heuristic in balancing load and reducing costs compared to optimal and state-of-the-art solutions?
• RQ4 Can the proposed heuristic adapt effectively to varying workload patterns in real-time?
• RQ5 What role does the size of the replicasArray play in system performance and cost efficiency?

Key findings

• The proposed heuristic reduces public cloud costs by more than 50% compared to a load-balancing-focused baseline solution.
• The heuristic maintains application completion times within predefined thresholds across all tested workload scenarios, including sudden traffic spikes.
• In high-load conditions, the heuristic ensures stable performance with minimal variance in request completion times, demonstrating effective load balancing.
• The DSR algorithm successfully detects workload trends and dynamically activates/deactivates regions, reducing unnecessary public resource usage.
• The solution is feasible and effective in real-world deployments, as confirmed by evaluation in a Kubernetes cluster on OpenStack.
• The heuristic achieves near-optimal cost performance despite the NP-hard nature of the underlying problem.