[Paper Review] Generalized Preconditioning and Network Flow Problems

Jonah Sherman|arXiv (Cornell University)|Jun 23, 2016

Complexity and Algorithms in Graphs17 references3 citations

TL;DR

This paper introduces a generalized preconditioning framework that enables nearly-linear time approximation algorithms for minimum-norm problems in non-Euclidean norms, particularly applying it to uncapacitated minimum-cost flow on undirected graphs. By recursively applying residual error reduction via a hierarchical routing scheme in ℓ₁, the method achieves (1+ε)-approximate solutions in ε⁻²m¹⁺ᵒ⁽¹⁾ time, with the key innovation being a generalized preconditioner based on lattice rounding and multigrid-style analysis.

ABSTRACT

We consider approximation algorithms for the problem of finding $x$ of minimal norm $\|x\|$ satisfying a linear system $\mathbf{A} x = \mathbf{b}$, where the norm $\|\cdot \|$ is arbitrary and generally non-Euclidean. We show a simple general technique for composing solvers, converting iterative solvers with residual error $\|\mathbf{A} x - \mathbf{b}\| \leq t^{-Ω(1)}$ into solvers with residual error $\exp(-Ω(t))$, at the cost of an increase in $\|x\|$, by recursively invoking the solver on the residual problem $ ilde{\mathbf{b}} = \mathbf{b} - \mathbf{A} x$. Convergence of the composed solvers depends strongly on a generalization of the classical condition number to general norms, reducing the task of designing algorithms for many such problems to that of designing a \emph{generalized preconditioner} for $\mathbf{A}$. The new ideas significantly generalize those introduced by the author's earlier work on maximum flow, making them more widely applicable. As an application of the new technique, we present a nearly-linear time approximation algorithm for uncapacitated minimum-cost flow on undirected graphs. Given an undirected graph with $m$ edges labelled with costs, and $n$ vertices labelled with demands, the algorithm takes $ε^{-2}m^{1+o(1)}$-time and outputs a flow routing the demands with total cost at most $(1+ε)$ times larger than minimal, along with a dual solution proving near-optimality. The generalized preconditioner is obtained by embedding the cost metric into $\ell_1$, and then considering a simple hierarchical routing scheme in $\ell_1$ where demands initially supported on a dense lattice are pulled from a sparser lattice by randomly rounding unaligned coordinates to their aligned neighbors. Analysis of the generalized condition number for the preconditioner follows that of the classical multigrid algorithm for lattice Laplacian systems.

Motivation & Objective

• To develop a general framework for solving minimum-norm problems in arbitrary, non-Euclidean norms, extending the success of preconditioning from ℓ₂ to broader settings.
• To address the lack of exponentially convergent iterative solvers for non-Euclidean norms, which currently force reliance on interior point methods with high iteration costs or polynomially convergent alternatives.
• To design a nearly-linear time approximation algorithm for uncapacitated minimum-cost flow on undirected graphs, achieving (1+ε)-approximation with dual certification.
• To generalize residual recursion techniques beyond ℓ₂ by introducing a generalized condition number and a modular composition framework for solvers.
• To construct a generalized preconditioner for ℓ₁ minimum-cost flow by embedding the cost metric into ℓ₁ and using hierarchical lattice rounding with random alignment.

Proposed method

• The method employs residual recursion: recursively solving the residual problem b̃ = b − Ax to reduce error, with convergence governed by a generalized condition number in arbitrary norms.
• A composition lemma is introduced to analyze the approximation parameters of recursively composed solvers, reducing algorithm design to constructing good generalized preconditioners.
• For minimum-cost flow, the preconditioner is built by embedding demands into ℓ₁ and using a hierarchical routing scheme that pulls demand from denser to sparser lattices via random rounding of unaligned coordinates.
• The routing scheme reduces demands on lattice Vₜ to Vₜ₋₁ by distributing each vertex’s demand to its closest neighbors on the coarser lattice, with path lengths ≤ k2⁻ᵗ.
• The cost of each routing level is bounded by k2⁻ᵗ‖bₜ‖₁, and the total cost is shown to be within a factor of 2k(t+1) of the optimal cost, establishing the preconditioner’s quality.
• The generalized condition number is analyzed using techniques analogous to classical multigrid, leveraging scale invariance and linearity of the reduction to bound cost increases.

Experimental results

Research questions

• RQ1 Can residual recursion be generalized beyond ℓ₂ to achieve exponential error decay in arbitrary norms?
• RQ2 What is the appropriate generalization of the condition number that governs convergence in non-Euclidean minimum-norm problems?
• RQ3 Can a nearly-linear time (1+ε)-approximation algorithm be constructed for uncapacitated minimum-cost flow on undirected graphs using generalized preconditioning?
• RQ4 How can a preconditioner be designed for ℓ₁ minimum-norm problems that ensures bounded generalized condition number?
• RQ5 Is there a modular framework that allows composing black-box solvers via residual recursion to achieve fast convergence in non-Euclidean settings?

Key findings

• The paper establishes a composition lemma showing that recursively composing two solvers with residual error t⁻Ω(1) yields a solver with residual error exp(−Ω(t)), at the cost of increasing the solution norm.
• The generalized preconditioner for ℓ₁ minimum-cost flow achieves a condition number bound of O(k), where k is the number of non-integer coordinates in the lattice, enabling nearly-linear time algorithms.
• The hierarchical routing scheme incurs total cost at most k2⁻ᵗ‖bₜ‖₁ per level, and the total cost is bounded by 2k(t+1)‖bₜ‖ₒₚₜ, proving the preconditioner is within a factor of O(k) of optimal.
• The algorithm computes a (1+ε)-approximate solution to the uncapacitated minimum-cost flow problem in ε⁻²m¹⁺ᵒ⁽¹⁾ time, with a dual solution proving near-optimality.
• The method recovers a simpler nearly-linear time max-flow algorithm as a special case, where the congestion approximator becomes a specific instance of the generalized preconditioner.
• The analysis shows that the residual reduction step does not increase the minimum cost, i.e., ‖bₜ₋₁‖ₒₚₜ ≤ ‖bₜ‖ₒₚₜ, which is essential for bounding the total cost.