[Paper Review] Fast Distributed Algorithms for Computing Separable Functions

Damon Mosk-Aoyama, Devavrat Shah|ArXiv.org|Apr 8, 2005

Neural Networks and Applications4 citations

TL;DR

This paper presents a novel randomized distributed algorithm for computing separable functions—functions expressible as sums of individual node values—using a minimum computation subroutine based on exponential random variables. By leveraging a gossip-based information spreading mechanism, the algorithm achieves faster convergence than iterative averaging on graphs like grids, with running time scaling as O(dn^{1/d} log²n) for d-dimensional grids, outperforming known iterative methods by a factor of roughly the square root of the runtime.

ABSTRACT

The problem of computing functions of values at the nodes in a network in a totally distributed manner, where nodes do not have unique identities and make decisions based only on local information, has applications in sensor, peer-to-peer, and ad-hoc networks. The task of computing separable functions, which can be written as linear combinations of functions of individual variables, is studied in this context. Known iterative algorithms for averaging can be used to compute the normalized values of such functions, but these algorithms do not extend in general to the computation of the actual values of separable functions. The main contribution of this paper is the design of a distributed randomized algorithm for computing separable functions. The running time of the algorithm is shown to depend on the running time of a minimum computation algorithm used as a subroutine. Using a randomized gossip mechanism for minimum computation as the subroutine yields a complete totally distributed algorithm for computing separable functions. For a class of graphs with small spectral gap, such as grid graphs, the time used by the algorithm to compute averages is of a smaller order than the time required by a known iterative averaging scheme.

Motivation & Objective

• To design a totally distributed algorithm for computing separable functions in networks where nodes lack unique identities and only have local knowledge of neighbors.
• To overcome the limitation of existing iterative averaging algorithms, which cannot compute actual values of separable functions without normalization.
• To reduce the running time of distributed computation by exploiting order- and duplicate-insensitive statistics, particularly the minimum of values.
• To analyze and optimize the performance of the algorithm using conductance-based bounds on information spreading time.
• To demonstrate that the algorithm achieves significantly faster convergence than known iterative schemes on graphs with small spectral gaps, such as grid graphs.

Proposed method

• The algorithm reduces the problem of computing a separable function to computing the minimum of a set of values, one per node, using properties of exponential random variables.
• Each node generates an exponential random variable and computes the minimum of its own value and the exponential variable, enabling order- and duplicate-insensitive aggregation.
• A randomized gossip mechanism is used as a subroutine to disseminate these minimum values across the network, ensuring all nodes eventually learn the global minimum.
• The running time of the overall algorithm is bounded by the information spreading time of the gossip subroutine, which depends on the conductance of the underlying stochastic matrix governing communication choices.
• The analysis uses spectral graph theory, particularly the conductance Φ(P) of the transition matrix P, to derive upper bounds on the (ε, δ)-computing time.
• For specific graph classes like d-dimensional grids, the algorithm’s running time is derived explicitly in terms of n, d, ε, and δ.

Experimental results

Research questions

• RQ1 Can a totally distributed algorithm compute the actual value of a separable function without requiring global coordination or node identifiers?
• RQ2 How can the minimum of a set of values be used as a building block for computing separable functions in a distributed, order-insensitive manner?
• RQ3 What is the running time of a distributed algorithm for computing separable functions when using a randomized gossip mechanism as a subroutine?
• RQ4 How does the performance of this algorithm compare to known iterative averaging algorithms on graphs such as grids and expanders?
• RQ5 Can the algorithm be used to improve the efficiency of fundamental distributed tasks like averaging or convex optimization with linear constraints?

Key findings

• On d-dimensional grid graphs, the algorithm computes averages with (1±ε) accuracy and probability at least 1−δ in O(ε⁻²(1+log δ⁻¹)(log n + log δ⁻¹)dn^{1/d}) time.
• For fixed d and δ=1/n, the running time scales as O(dn^{1/d} log²n), which is asymptotically faster than the Ω(n²/d log n) time required by iterative averaging algorithms.
• The algorithm’s performance is bounded by the conductance of the communication matrix, and it achieves a running time proportional to 1/Φ(P), matching the lower bound for iterative schemes based on stochastic matrices.
• The algorithm outperforms known iterative averaging methods by a factor of roughly the square root of the runtime on graphs where spectral gap is proportional to the square of the conductance.
• The approach enables efficient distributed computation of separable functions on graphs with small spectral gaps, such as paths, rings, and grids, due to favorable information spreading dynamics.
• The framework is extensible to other distributed tasks, including approximate convex minimization with linear constraints and packet marking in the Internet, via summation as a core subroutine.