[Paper Review] Asynchronous Algorithms for Solving Linear Programs

Tarek A. Lahlou, Thomas A. Baran|arXiv (Cornell University)|Feb 24, 2015

Sparse and Compressive Sensing Techniques11 references3 citations

TL;DR

This paper proposes asynchronous, distributed algorithms for solving linear programs using nonlinear signal-flow structures that converge to optimal solutions without global synchronization. By reformulating linear programs into an asynchronous form and leveraging fixed-point iterations via orthogonal matrices and memoryless nonlinearities, the method achieves convergence under arbitrary delays and computational heterogeneity, validated through Chebyshev center and basis pursuit problems with numerical simulations.

ABSTRACT

In this paper we design and analyze algorithms for asynchronously solving linear programs using nonlinear signal processing structures. In particular, we discuss a general procedure for generating these structures such that a fixed-point of the structure is within a change of basis the minimizer of an associated linear program. We discuss methods for organizing the computation into distributed implementations and provide a treatment of convergence. The presented algorithms are accompanied by numerical simulations of the Chebyshev center and basis pursuit problems.

Motivation & Objective

• To design distributed, asynchronous algorithms for solving linear programs that are robust to communication delays and processor heterogeneity.
• To address the limitations of traditional sequential and synchronized optimization methods in distributed settings, where synchronization and communication overheads degrade performance.
• To develop a general framework based on signal-flow structures that transform linear programs into fixed-point problems via stationarity conditions.
• To ensure convergence of the proposed algorithms using stability theory and dynamic systems analysis.
• To demonstrate the approach on practical problems such as the Chebyshev center and basis pursuit, showing robustness and scalability.

Proposed method

• Reformulate standard form linear programs into an 'asynchronous form' where all variables are constrained to be fixed, unconstrained, or non-negative, and all constraints are equality-based.
• Construct a system of stationarity conditions using an orthogonal matrix G and a memoryless nonlinearity m(·), such that fixed points of the system correspond to optimal solutions of the original linear program.
• Model asynchronous computation using randomly triggered sample-and-hold elements represented as independent Bernoulli processes, introducing arbitrary delays without disrupting convergence.
• Resolve delay-free loops via algebraic methods or insertion of asynchronous delays, ensuring the system remains well-posed and stable.
• Implement the algorithm using signal-flow structures where inputs and outputs are mapped to variables in the reformulated problem, with nonlinearities encoding cost and constraint terms.
• Use homotopy continuation in early iterations to improve convergence behavior, gradually transitioning to the full nonlinearity.

Experimental results

Research questions

• RQ1 Can linear programs be solved efficiently in a distributed, asynchronous manner without requiring global synchronization or precise timing?
• RQ2 What conditions ensure convergence of asynchronous signal-flow structures when solving linear programs under arbitrary delays and processor heterogeneity?
• RQ3 How can a linear program be reformulated into a form amenable to nonlinear signal processing with guaranteed fixed-point convergence to the optimal solution?
• RQ4 To what extent do the proposed algorithms maintain robustness and scalability in high-dimensional problems like basis pursuit and Chebyshev center computation?
• RQ5 How does the choice of nonlinearity and asynchronous firing probability affect convergence speed and accuracy in practice?

Key findings

• The proposed asynchronous algorithms converge to the optimal solution of the linear program under arbitrary communication and processing delays, as validated by theoretical stability analysis.
• Numerical simulations show convergence for the Chebyshev center problem in 100-dimensional space with 200 random hyperplanes, where the center is tracked via a blue line converging to the fixed-point.
• For the basis pursuit problem with a 16-sparse signal in 512 dimensions, the algorithm successfully recovers the true solution over 1000 trials, with convergence observed across varying delay probabilities (p = 0.2 to 0.8).
• The use of a homotopy method in early iterations significantly improves initial convergence, with the nonlinearity gradually transitioning from a smoothed to exact form.
• The framework achieves robust performance across diverse problem instances, demonstrating scalability and resilience to computational heterogeneity.
• The fixed-point of the signal-flow structure corresponds to the minimizer of the original linear program under a change of basis, confirming the correctness of the reformulation.