[Paper Review] Nonhomogeneous Place-Dependent Markov Chains, Unsynchronised AIMD, and Network Utility Maximization

Fabian Wirth, Sonja Stuedli+3 more|arXiv (Cornell University)|Apr 20, 2014

Network Traffic and Congestion Control12 references16 citations

TL;DR

This paper proposes a minimal-communication algorithm for network utility maximization using unsynchronized AIMD with probabilistic response to intermittent capacity signals. It proves almost sure convergence to the social optimum via nonhomogeneous place-dependent Markov chains, requiring only one-bit feedback and no inter-agent communication or common clock.

ABSTRACT

We present a solution of a class of network utility maximization (NUM) problems using minimal communication. The constraints of the problem are inspired less by TCP-like congestion control but by problems in the area of internet of things and related areas in which the need arises to bring the behavior of a large group of agents to a social optimum. The approach uses only intermittent feedback, no inter-agent communication, and no common clock. The proposed algorithm is a combination of the classical AIMD algorithm in conjunction with a simple probabilistic rule for the agents to respond to a capacity signal. This leads to a nonhomogeneous Markov chain and we show almost sure convergence of this chain to the social optimum.

Motivation & Objective

• Address large-scale optimization in IoT and smart systems where agents have limited actuation, privacy constraints, and no common clock.
• Solve network utility maximization (NUM) problems with minimal communication, no inter-agent signaling, and no centralized feedback beyond a single capacity signal.
• Achieve convergence to the social optimum despite time-varying agent counts, private cost functions, and intermittent feedback.
• Design a scalable, decentralized algorithm resilient to network heterogeneity and asynchronous operation.

Proposed method

• Use additive-increase multiplicative-decrease (AIMD) dynamics where agents increase resource usage gradually and reduce it multiplicatively upon receiving a capacity signal.
• Introduce a probabilistic response rule where agents respond to the capacity signal with a probability λi that depends on their long-term average resource usage.
• Model the system as a nonhomogeneous place-dependent Markov chain to analyze convergence behavior under intermittent feedback.
• Apply iterated function systems and invariant measure theory to characterize the long-term behavior of the system.
• Use distance metrics (ℓ1 and Hausdorff) to track convergence to the optimal allocation vector w*.
• Prove almost sure convergence by analyzing the expected logarithmic drift of state distances, leveraging moment bounds and tail probability estimates.

Experimental results

Research questions

• RQ1 Can a decentralized, minimal-communication algorithm achieve convergence to the social optimum in large-scale network utility maximization?
• RQ2 Does the system converge almost surely to the optimal allocation when only one-bit intermittent feedback is available and no inter-agent communication is used?
• RQ3 How does the probabilistic response rule, dependent on long-term average usage, affect convergence and stability?
• RQ4 Can convergence be guaranteed independently of network size, depending only on the worst-case agent?
• RQ5 What role does the nonhomogeneous Markov chain structure play in ensuring convergence under asynchronous, unsynchronized agent updates?

Key findings

• The proposed algorithm achieves almost sure convergence to the optimal allocation vector w* without any inter-agent communication or common clock.
• Convergence is guaranteed under intermittent one-bit feedback indicating capacity violation, with no need to communicate the Lagrange multiplier or exact resource levels.
• The convergence rate depends only on the worst agent in the system, not on the total number of agents, ensuring scalability.
• The system's dynamics are modeled as a nonhomogeneous place-dependent Markov chain, and almost sure convergence is proven via almost-sure divergence of negative drift terms.
• The algorithm remains stable under small perturbations, as the state remains almost surely within a shrinking neighborhood of the optimal point after finite time.
• The proof relies on bounding the expected logarithmic drift of the distance to the optimal set, showing that the sum of log-drift terms diverges to −∞ almost surely.