[Paper Review] Efficient Structured Matrix Recovery and Nearly-Linear Time Algorithms for Solving Inverse Symmetric $M$-Matrices

Arun Jambulapati, Kirankumar Shiragur+1 more|arXiv (Cornell University)|Dec 15, 2018

Sparse and Compressive Sensing Techniques28 references5 citations

TL;DR

This paper presents nearly linear time algorithms for solving inverse symmetric M-matrices, a broad class including Laplacians and SDD matrices, by recovering spectral approximations using only polylogarithmic adaptive linear measurements. The key contribution is an $ ilde{O}(n^2)$-time algorithm for solving such systems, significantly improving over the $O(n^ ho)$ time of standard matrix multiplication-based methods.

ABSTRACT

In this paper we show how to recover a spectral approximations to broad classes of structured matrices using only a polylogarithmic number of adaptive linear measurements to either the matrix or its inverse. Leveraging this result we obtain faster algorithms for variety of linear algebraic problems. Key results include: $\bullet$ A nearly linear time algorithm for solving the inverse of symmetric $M$-matrices, a strict superset of Laplacians and SDD matrices. $\bullet$ An $ ilde{O}(n^2)$ time algorithm for solving $n imes n$ linear systems that are constant spectral approximations of Laplacians or more generally, SDD matrices. $\bullet$ An $ ilde{O}(n^2)$ algorithm to recover a spectral approximation of a $n$-vertex graph using only $ ilde{O}(1)$ matrix-vector multiplies with its Laplacian matrix. The previous best results for each problem either used a trivial number of queries to exactly recover the matrix or a trivial $O(n^ω)$ running time, where $ω$ is the matrix multiplication constant. We achieve these results by generalizing recent semidefinite programming based linear sized sparsifier results of Lee and Sun (2017) and providing iterative methods inspired by the semistreaming sparsification results of Kapralov, Lee, Musco, Musco and Sidford (2014) and input sparsity time linear system solving results of Li, Miller, and Peng (2013). We hope that by initiating study of these natural problems, expanding the robustness and scope of recent nearly linear time linear system solving research, and providing general matrix recovery machinery this work may serve as a stepping stone for faster algorithms.

Motivation & Objective

• To develop efficient algorithms for solving linear systems in inverse symmetric M-matrices, a strict superset of Laplacians and SDD matrices.
• To recover spectral approximations of $n$-vertex graphs using only $ ilde{O}(1)$ matrix-vector products with the Laplacian matrix.
• To overcome the limitation of existing methods that rely on fast matrix multiplication ($O(n^ ho)$ time) for such problems, even when graph structure is present.
• To generalize recent semidefinite programming and sparsification techniques to enable nearly linear time matrix recovery and system solving.
• To establish a general framework for structured matrix recovery with provable guarantees using few adaptive measurements.

Proposed method

• Leverages adaptive linear measurements to the matrix or its inverse to recover spectral approximations with polylogarithmic query complexity.
• Adapts and extends semidefinite programming-based sparsification techniques from Lee and Sun (2017) to structured matrices.
• Employs iterative methods inspired by semistreaming sparsification (Kapralov et al., 2014) and input-sparsity linear system solvers (Li et al., 2013).
• Uses Johnson-Lindenstrauss sketches to efficiently approximate matrix inner products and quadratic forms in the oracle implementation.
• Designs a specialized oracle ($\mathtt{MOracle}$) with speed $1/2$ and error $\varepsilon$ that runs in $\widetilde{O}\left(\frac{1}{\gamma^4\rho^{O(1)}}\left(\mathcal{T}_{\mathbf{B}} + \mathcal{T}_{\mathbf{B}^{-1}} + \mathcal{T}_{MV} + \mathcal{T}_{QF}\right)\right)$ time.
• Applies a packing SDP result (Theorem 5.1) to solve a transformed optimization problem with multiplicative approximations to objective coefficients and constraint vectors.

Experimental results

Research questions

• RQ1 Can spectral approximations of structured matrices like inverse symmetric M-matrices be recovered using only polylogarithmic adaptive linear measurements?
• RQ2 Is it possible to solve linear systems in inverse symmetric M-matrices in nearly linear time, even when the matrix is dense?
• RQ3 Can spectral sparsifiers of $n$-vertex graphs be recovered using only $\widetilde{O}(1)$ matrix-vector products with the Laplacian?
• RQ4 How can the robustness of nearly linear time linear system solving be extended beyond Laplacians to broader classes of matrices?
• RQ5 Can general matrix recovery machinery be developed that enables faster algorithms for problems previously reliant on fast matrix multiplication?

Key findings

• An $ ilde{O}(n^2)$-time algorithm is achieved for solving $n \times n$ linear systems that are constant spectral approximations of Laplacians or SDD matrices.
• A nearly linear time algorithm is developed for solving the inverse of symmetric $M$-matrices, which includes Laplacians and SDD matrices as special cases.
• The paper shows that the inverse of symmetric $M$-matrices is either block-separable or dense, enabling nearly linear time solution in both cases.
• A spectral approximation of an $n$-vertex graph can be recovered using only $ ilde{O}(1)$ matrix-vector products with its Laplacian matrix.
• The proposed oracle ($\mathtt{MOracle}$) runs in time $\widetilde{O}\left(\frac{1}{\gamma^4\rho^{O(1)}}\left(\mathcal{T}_{\mathbf{B}} + \mathcal{T}_{\mathbf{B}^{-1}} + \mathcal{T}_{MV} + \mathcal{T}_{QF}\right)\right)$, enabling efficient implementation.
• The framework achieves $(1 - O(\delta))$-approximate solutions to the optimization problem with high probability, using $O(\mathcal{W}_{L,\delta} \log m \cdot \log(nm/\delta)/\delta^3)$ time.