[Paper Review] Asymptotically Optimal Load Balancing in Large-scale Heterogeneous Systems with Multiple Dispatchers

Xingyu Zhou, Ness B. Shroff+1 more|arXiv (Cornell University)|Feb 20, 2020

Advanced Queuing Theory Analysis25 references4 citations

TL;DR

This paper introduces the Local-Estimation-Driven (LED) framework for load balancing in large-scale heterogeneous systems with multiple dispatchers, where dispatchers use local, possibly outdated queue length estimates to route jobs. It establishes sufficient conditions for LED policies to achieve both throughput optimality and delay optimality in heavy traffic, resolving an open problem on delay-optimal load balancing with delayed information and enabling new low-overhead, delay-optimal policies for heterogeneous, multi-dispatcher systems.

ABSTRACT

We consider the load balancing problem in large-scale heterogeneous systems with multiple dispatchers. We introduce a general framework called Local-Estimation-Driven (LED). Under this framework, each dispatcher keeps local (possibly outdated) estimates of queue lengths for all the servers, and the dispatching decision is made purely based on these local estimates. The local estimates are updated via infrequent communications between dispatchers and servers. We derive sufficient conditions for LED policies to achieve throughput optimality and delay optimality in heavy-traffic, respectively. These conditions directly imply delay optimality for many previous local-memory based policies in heavy traffic. Moreover, the results enable us to design new delay optimal policies for heterogeneous systems with multiple dispatchers. Finally, the heavy-traffic delay optimality of the LED framework directly resolves a recent open problem on how to design optimal load balancing schemes using delayed information.

Motivation & Objective

• To address the lack of theoretical delay performance guarantees for load balancing in large-scale heterogeneous systems with multiple dispatchers.
• To design a scalable load balancing framework that uses only delayed, local information to minimize message overhead.
• To resolve the open problem of achieving delay optimality in heavy traffic when only outdated queue state information is available.
• To generalize existing local-memory-based policies (e.g., LSQ) and enable systematic design of new delay-optimal policies.
• To provide sufficient conditions for both throughput and delay optimality under the LED framework.

Proposed method

• Propose the Local-Estimation-Driven (LED) framework, where each dispatcher maintains local, possibly outdated estimates of server queue lengths.
• Use infrequent communication between dispatchers and servers to update local estimates, minimizing message overhead.
• Derive sufficient conditions on the update strategy that guarantee throughput and delay optimality in heavy traffic.
• Apply state-space collapse and Lyapunov drift analysis to prove heavy-traffic delay optimality under these conditions.
• Generalize prior policies like LSQ and show they satisfy the sufficient conditions, thus inheriting delay optimality.
• Use Cauchy-Schwarz and moment-boundedness arguments to bound the expected product of queue and imbalance vectors in steady state.

Experimental results

Research questions

• RQ1 Can delay performance guarantees be established for load balancing in heterogeneous systems with multiple dispatchers?
• RQ2 Is it possible to achieve delay optimality in heavy traffic using only delayed, local information?
• RQ3 What conditions on the update strategy ensure both throughput and delay optimality in the LED framework?
• RQ4 How can existing local-memory-based policies be generalized and proven optimal under the LED framework?
• RQ5 Can new low-overhead, delay-optimal policies be systematically designed for heterogeneous, multi-dispatcher systems?

Key findings

• The LED framework achieves throughput optimality under mild conditions on the update strategy, ensuring high server utilization.
• Sufficient conditions are derived that guarantee delay optimality in heavy traffic, resolving a recent open problem in load balancing with delayed information.
• Many prior local-memory-based policies, including LSQ, satisfy the sufficient conditions and are therefore delay optimal in heavy traffic.
• The framework enables the design of new delay-optimal policies with zero dispatching delay and low message overhead for heterogeneous systems.
• The heavy-traffic delay optimality is proven via state-space collapse and moment-boundedness arguments, showing convergence of key performance metrics to zero as system load approaches capacity.
• The framework provides a systematic method to explore trade-offs between local memory usage and communication overhead in load balancing design.