[Paper Review] The Hidden cost of the Edge: A Performance Comparison of Edge and Cloud Latencies

Ahmed Ali-Eldin, Bin Wang+1 more|arXiv (Cornell University)|Apr 28, 2021

Cloud Computing and Resource Management4 citations

TL;DR

This paper identifies and analyzes the 'edge performance inversion' problem, where edge computing's lower network latency is offset by higher queuing delays due to resource constraints, resulting in worse end-to-end latency than the cloud at moderate to high utilizations. Using queuing theory and real-world experiments with Azure workloads, the authors show that edge latency can exceed cloud latency even at 40–60% utilization, especially under workload skew, necessitating overprovisioning to avoid performance degradation.

ABSTRACT

Edge computing has emerged as a popular paradigm for running latency-sensitive applications due to its ability to offer lower network latencies to end-users. In this paper, we argue that despite its lower network latency, the resource-constrained nature of the edge can result in higher end-to-end latency, especially at higher utilizations, when compared to cloud data centers. We study this edge performance inversion problem through an analytic comparison of edge and cloud latencies and analyze conditions under which the edge can yield worse performance than the cloud. To verify our analytic results, we conduct a detailed experimental comparison of the edge and the cloud latencies using a realistic application and real cloud workloads. Both our analytical and experimental results show that even at moderate utilizations, the edge queuing delays can offset the benefits of lower network latencies, and even result in performance inversion where running in the cloud would provide superior latencies. We finally discuss practical implications of our results and provide insights into how application designers and service providers should design edge applications and systems to avoid these pitfalls.

Motivation & Objective

• To identify and formalize the edge performance inversion problem, where edge end-to-end latency exceeds that of the cloud despite lower network latency.
• To analyze conditions under which edge queuing delays outweigh network latency benefits using queuing models.
• To validate the analytic findings through real-world experiments using Azure trace workloads on actual cloud and edge platforms.
• To provide practical guidance for application designers and cloud providers on avoiding performance inversion through proper resource allocation and overprovisioning.

Proposed method

• Formalizing the edge performance inversion problem using M/M/1 queuing models to compare end-to-end latency between edge and cloud systems.
• Deriving closed-form analytic expressions for edge and cloud end-to-end latencies, including network delay and server queuing delay components.
• Modeling workload distribution across k edge sites with Poisson arrivals, assuming balanced and skewed workloads to assess peak capacity requirements.
• Calculating required edge capacity as $ C_{edge} = \lambda + 2\sqrt{k\lambda} $, showing it exceeds cloud capacity $ C_{cloud} = \lambda + 2\sqrt{\lambda} $ due to lack of statistical multiplexing gain.
• Conducting real-world experiments using Azure trace workloads and a realistic application stack on physical cloud and edge platforms to measure mean and tail latencies.
• Comparing experimental results with analytic predictions to validate the occurrence of performance inversion under realistic conditions.

Experimental results

Research questions

• RQ1 Under what conditions does the edge's higher queuing delay cause end-to-end latency to exceed that of the cloud, despite lower network latency?
• RQ2 How does workload skew across edge sites affect the likelihood and severity of performance inversion?
• RQ3 To what extent do moderate utilization levels (e.g., 40–60%) lead to worse edge latency than the cloud due to queuing effects?
• RQ4 What level of overprovisioning is required at the edge to prevent performance inversion compared to the cloud?
• RQ5 How do statistical multiplexing gains in the cloud reduce peak capacity requirements relative to distributed edge deployments?

Key findings

• Edge end-to-end latency can exceed cloud latency even at moderate utilization levels of 40–60%, particularly when workloads are skewed across edge sites.
• The peak capacity required at the edge, $ C_{edge} = \lambda + 2\sqrt{k\lambda} $, is higher than the cloud’s $ C_{cloud} = \lambda + 2\sqrt{\lambda} $, due to the lack of statistical multiplexing gains.
• Performance inversion is more likely as the latency to traditional clouds decreases due to increased cloud deployment density.
• Real-world experiments using Azure workloads confirm that both mean and tail latencies on the edge can be worse than in the cloud under the same load conditions.
• The hidden cost of edge computing—higher queuing delays—necessitates significant overprovisioning at the edge to avoid performance degradation.
• Cloud providers face higher economic costs for edge deployments due to the need for extra capacity to prevent performance inversion.