[Paper Review] Dissecting Service Mesh Overheads

Xiangfeng Zhu, Guozhen She+10 more|arXiv (Cornell University)|Jul 2, 2022

Cloud Computing and Resource Management6 citations

TL;DR

This paper introduces MeshInsight, a compositional performance analysis tool that dissects service mesh overheads by modeling sidecar operations as independent components (e.g., IPC, socket writes, protocol parsing). It reveals that overheads vary widely—up to 185% latency increase and 92% higher CPU usage—depending on configuration and workload, with protocol parsing dominating in HTTP/gRPC modes and IPC/socket operations in TCP mode.

ABSTRACT

Service meshes play a central role in the modern application ecosystem by providing an easy and flexible way to connect different services that form a distributed application. However, because of the way they interpose on application traffic, they can substantially increase application latency and resource consumption. We develop a decompositional approach and a tool, called MeshInsight, to systematically characterize the overhead of service meshes and to help developers quantify overhead in deployment scenarios of interest. Using MeshInsight, we confirm that service meshes can have high overhead -- up to 185% higher latency and up to 92% more virtual CPU cores for our benchmark applications -- but the severity is intimately tied to how they are configured and the application workload. The primary contributors to overhead vary based on the configuration too. IPC (inter-process communication) and socket writes dominate when the service mesh operates as a TCP proxy, but protocol parsing dominates when it operates as an HTTP proxy. MeshInsight also enables us to study the end-to-end impact of optimizations to service meshes. We show that not all seemingly-promising optimizations lead to a notable overhead reduction in realistic settings.

Motivation & Objective

• To systematically characterize the performance overhead of service meshes across diverse configurations and application workloads.
• To identify the primary contributors to overhead in service mesh sidecars, such as IPC, socket operations, and protocol parsing.
• To enable developers to estimate end-to-end performance impact without deploying full configurations.
• To evaluate the real-world effectiveness of optimizations like Unix domain sockets and zero-copy writes on service mesh performance.
• To guide service mesh and application developers in making informed trade-offs between functionality and performance.

Proposed method

• Decomposes sidecar proxy operations into independent components: read, write, IPC, and protocol parsing.
• Models application workloads using call graphs, request rates, and message sizes to estimate end-to-end overhead.
• Employs a measurement-driven approach to quantify component-level overheads using real Envoy sidecar deployments.
• Uses a toolchain to instrument and profile sidecar behavior across different service mesh configurations (TCP, HTTP, gRPC modes).
• Applies Linux kernel features (Unix domain sockets, zero-copy writes) to evaluate performance improvements.
• Combines component-level measurements with workload models to predict system-wide performance impact without full deployment.

Experimental results

Research questions

• RQ1 What are the dominant sources of latency and CPU overhead in service mesh sidecars across different configurations?
• RQ2 How do application-specific characteristics (e.g., request rate, message size, call graph) influence service mesh performance?
• RQ3 To what extent do common optimizations (e.g., Unix domain sockets, zero-copy I/O) reduce end-to-end overhead in real-world scenarios?
• RQ4 How does protocol parsing overhead compare to system-level operations (e.g., IPC, socket I/O) in different proxy modes?
• RQ5 Can a compositional modeling approach accurately predict end-to-end performance without full deployment?

Key findings

• Service mesh overheads vary dramatically—latency increases by 30–185% and CPU usage by 41–92% depending on configuration and application.
• In HTTP/gRPC mode, protocol parsing accounts for 62–73% of total overhead, making it the dominant performance bottleneck.
• In TCP mode, IPC and socket write operations are the primary contributors to overhead, collectively dominating performance costs.
• Individual filters can add between 3% and 85% extra latency on top of baseline overhead, indicating high variability in filter efficiency.
• Optimizations like Unix domain sockets and zero-copy writes reduce overhead, but their real-world impact is context-dependent and not uniformly beneficial.
• Overhead varies by orders of magnitude across different applications under the same configuration, underscoring the need for application-aware performance analysis.