[Paper Review] Non-clairvoyant Scheduling of Coflows

Akhil Bhimaraju, Debanuj Nayak+1 more|arXiv (Cornell University)|Apr 13, 2020

Cloud Computing and Resource Management26 references4 citations

TL;DR

This paper proposes BlindFlow, a non-clairvoyant algorithm for coflow scheduling that operates without prior knowledge of flow sizes, using only the number of flows and input-output port pairs upon arrival. It achieves an 8p-approximation guarantee for minimizing weighted sum of coflow completion times, where p is the maximum number of input-output port pairs per coflow, and this bound holds even in the online setting with causal information only.

ABSTRACT

The coflow scheduling problem is considered: given an input/output switch with each port having a fixed capacity, find a scheduling algorithm that minimizes the weighted sum of the coflow completion times respecting the port capacities, where each flow of a coflow has a demand per input/output port, and coflow completion time is the finishing time of the last flow of the coflow. The objective of this paper is to present theoretical guarantees on approximating the sum of coflow completion time in the non-clairvoyant setting, where on a coflow arrival, only the number of flows, and their input-output port is revealed, while the critical demand volumes for each flow on the respective input-output port is unknown. The main result of this paper is to show that the proposed BlindFlow algorithm is $8p$-approximate, where $p$ is the largest number of input-output port pairs that a coflow uses. This result holds even in the online case, where coflows arrive over time and the scheduler has to use only causal information. Simulations reveal that the experimental performance of BlindFlow is far better than the theoretical guarantee.

Motivation & Objective

• To address the coflow scheduling problem in the non-clairvoyant setting, where flow demands are unknown at arrival.
• To provide theoretical approximation guarantees for minimizing the weighted sum of coflow completion times when only partial information (number of flows and port pairs) is available.
• To design an online, implementable algorithm that performs well despite lack of knowledge about flow sizes.
• To close the gap between theoretical bounds and empirical performance in non-clairvoyant coflow scheduling.

Proposed method

• BlindFlow uses a rate allocation strategy based on the number of active flows and their port assignments, without relying on flow size estimates.
• It employs a time-stretched scheduling approach with a base schedule derived from a relaxed LP formulation, adjusted via a time-scaling factor of 4p.
• The algorithm dynamically allocates bandwidth to flows based on their progress, ensuring fairness and progress under port capacity constraints.
• A key component is the use of a modified version of the CLAS (Least Attained Service) principle adapted for coflows in a non-clairvoyant setting.
• Theoretical analysis leverages a dual fitting technique and compares the algorithm’s performance to a clairvoyant offline optimal solution.
• The analysis introduces a time-stretched base schedule and uses a charging argument to bound the approximation ratio at 8p.

Experimental results

Research questions

• RQ1 Can a non-clairvoyant coflow scheduling algorithm achieve a constant approximation ratio with respect to the clairvoyant offline optimal solution?
• RQ2 How does the approximation ratio scale with the maximum number of input-output port pairs per coflow (p) in the non-clairvoyant setting?
• RQ3 Does the proposed algorithm outperform existing non-clairvoyant heuristics in practice despite theoretical bounds?
• RQ4 Can theoretical guarantees be derived for online, non-clairvoyant coflow scheduling under causal information only?

Key findings

• BlindFlow achieves an 8p-approximation ratio for minimizing the weighted sum of coflow completion times, where p is the maximum number of input-output port pairs used by any coflow.
• The theoretical guarantee holds even in the online setting, where coflows arrive over time and only causal information is available.
• Simulations on synthetic and real-world Facebook MapReduce traces show that BlindFlow's empirical performance is significantly better than the 8p theoretical bound.
• On real-world data, BlindFlow performs much closer to the clairvoyant lower bound than predicted, with the performance ratio not scaling with p as the theory suggests.
• BlindFlow outperforms the non-clairvoyant heuristic Aalo in practice, though Aalo is slightly better in some cases, and BlindFlow is easier to implement and analyze.
• The analysis suggests that the theoretical bound may be loose, and the actual performance is much better than the worst-case guarantee.