[Paper Review] Faster Dynamic Matrix Inverse for Faster LPs

Shunhua Jiang, Zhao Song+2 more|arXiv (Cornell University)|Apr 16, 2020

Complexity and Algorithms in Graphs44 references54 citations

TL;DR

This paper designs cascading dynamic matrix inverse data structures using multi-level Woodbury identities to speed up low-rank updates, yielding faster interior-point LP solvers that run in near matrix-multiplication time.

ABSTRACT

Motivated by recent Linear Programming solvers, we design dynamic data structures for maintaining the inverse of an $n imes n$ real matrix under $ extit{low-rank}$ updates, with polynomially faster amortized running time. Our data structure is based on a recursive application of the Woodbury-Morrison identity for implementing $ extit{cascading}$ low-rank updates, combined with recent sketching technology. Our techniques and amortized analysis of multi-level partial updates, may be of broader interest to dynamic matrix problems. This data structure leads to the fastest known LP solver for general (dense) linear programs, improving the running time of the recent algorithms of (Cohen et al.'19, Lee et al.'19, Brand'20) from $O^*(n^{2+ \max\{\frac{1}{6}, ω-2, \frac{1-α}{2}\}})$ to $O^*(n^{2+\max\{\frac{1}{18}, ω-2, \frac{1-α}{2}\}})$, where $ω$ and $α$ are the fast matrix multiplication exponent and its dual. Hence, under the common belief that $ω\approx 2$ and $α\approx 1$, our LP solver runs in $O^*(n^{2.055})$ time instead of $O^*(n^{2.16})$.

Motivation & Objective

• Motivate the need for dynamic inverse maintenance in fast LP solvers and related applications.
• Develop a cascading, multi-level update framework to accelerate low-rank matrix updates and queries.
• Analyze amortized performance using a novel potential framework and martingale arguments.
• Show how the framework improves LP solvers to near matrix-multiplication time under standard conjectures.
• Provide a detailed description and analysis of the cascading lazy updates technique for dynamic inverses.

Proposed method

• Generalize Woodbury’s identity to K>1 levels to cascade low-rank updates.
• Partition updates into K epochs with shrinking thresholds and maintain an LU decomposition of the Woodbury-structured matrix.
• Use cascading lazy updates to amortize update and query costs, leveraging properties of slowly changing LP-related vectors.
• Combine randomized compression and sketching to accelerate matrix-vector products within the projection maintenance step.
• Balance per-level costs to achieve runtime n* (n^{ω} + n^{2.5−α/2} + n^{2+1/18}) log(n/δ) with high probability.

Experimental results

Research questions

• RQ1 Can dynamic inverse maintenance under low-rank updates be accelerated beyond previous O*(n^{2+1/6}) LP approaches?
• RQ2 How can multi-level (K-level) cascading updates be implemented to maintain inverses efficiently in the presence of slowly changing query vectors?
• RQ3 What is the achievable LP runtime under standard assumptions about the matrix multiplication exponent ω and its dual α?
• RQ4 To what extent can randomized sketching be integrated with cascading updates without derandomizing the process?
• RQ5 How does the proposed framework impact practical LP solvers in terms of iteration count and overall runtime?

Key findings

• The paper achieves an LP solver running in O*(n^{ω} + n^{2.5−α/2} + n^{2+1/18}) per problem, logarithmically amplified by the accuracy parameter.
• In the ideal regime (ω≈2, α≈1), the solver runs in about O*(n^{2.055}) time, improving over the previous O*(n^{2.166}) benchmarks.
• A cascading lazy-update framework with K=3 levels yields a per-iteration runtime of approximately n^{2+1/18} under the analysis.
• Maintaining LU decompositions of the update-structured matrix efficiently is central to achieving sub-quadratic amortized costs.
• Randomized compression and sketching techniques are integrated to reduce projection maintenance costs without compromising convergence guarantees.
• The approach provides a general technique for dynamic matrix problems beyond LPs, via multi-level Woodbury-based updates and amortized analysis.