[Paper Review] Fast hierarchical solvers for sparse matrices

Hadi Pouransari, Pieter Coulier+1 more|arXiv (Cornell University)|Oct 26, 2015

Matrix Theory and Algorithms3 citations

TL;DR

This paper proposes a novel, fully algebraic sparse matrix solver with linear complexity and tunable accuracy by approximating LU factorization using hierarchical low-rank structures. It leverages fill-in approximation via low-rank compression in well-separated blocks of the adjacency graph, enabling efficient direct solves or use as a black-box preconditioner for iterative methods like GMRES.

ABSTRACT

Sparse linear systems are ubiquitous in various scientific computing applications. Inversion of sparse matrices with standard direct solve schemes are prohibitive for large systems due to their quadratic/cubic complexity. Iterative solvers, on the other hand, demonstrate better scalability. However, they suffer from poor convergence rates when used without a preconditioner. There are many preconditioners developed for different problems, such as ILU, AMG, Gauss-Seidel, etc. The choice of an effective preconditioner is highly problem dependent. We propose a novel fully algebraic sparse matrix solve algorithm, which has linear complexity with the problem size. Our scheme is based on the Gauss elimination. For a given matrix, we approximate the LU factorization with a tunable accuracy determined a priori. This method can be used as a stand-alone direct solver with linear complexity and tunable accuracy, or it can be used as a black-box preconditioner in conjunction with iterative methods such as GMRES. The proposed solver is based on the low-rank approximation of fill-ins generated during the elimination. Similar to H-matrices, fill-ins corresponding to blocks that are well-separated in the adjacency graph are represented via a hierarchical structure.

Motivation & Objective

• To address the high computational cost of direct solvers for large sparse linear systems, which scale quadratically or cubically with problem size.
• To overcome the poor convergence of iterative solvers by developing a robust, problem-agnostic preconditioner with tunable accuracy.
• To design a direct solver with linear complexity that maintains accuracy control through a priori error bounds.
• To enable efficient solution of sparse linear systems via hierarchical low-rank approximation of fill-ins during Gauss elimination.
• To provide a black-box preconditioner compatible with iterative methods such as GMRES without requiring problem-specific tuning.

Proposed method

• The method approximates the LU factorization of a sparse matrix using hierarchical low-rank compression of fill-in entries generated during Gaussian elimination.
• It exploits the adjacency graph structure to identify well-separated blocks, treating their fill-ins as low-rank matrices to reduce storage and computation.
• The algorithm applies a tunable accuracy threshold a priori to control the approximation error in the LU decomposition.
• The hierarchical structure mirrors H-matrix techniques, recursively partitioning blocks based on separation in the graph to apply low-rank compression.
• The solver can be used standalone as a direct solver or as a preconditioner in iterative Krylov subspace methods like GMRES.
• The approach maintains numerical stability by preserving sparsity patterns while compressing fill-ins via low-rank approximation.

Experimental results

Research questions

• RQ1 Can a fully algebraic sparse matrix solver achieve linear complexity while maintaining tunable accuracy?
• RQ2 How effectively can hierarchical low-rank approximation of fill-ins reduce computational cost in sparse LU factorization?
• RQ3 To what extent does the proposed solver serve as an effective black-box preconditioner for iterative solvers like GMRES?
• RQ4 Can the method maintain accuracy across diverse sparse matrix problems without problem-specific tuning?
• RQ5 What is the trade-off between accuracy and computational efficiency in the proposed approximation scheme?

Key findings

• The proposed solver achieves linear complexity with respect to problem size, significantly outperforming standard direct solvers in computational cost.
• The method provides tunable accuracy through a priori error control, allowing users to balance speed and precision.
• The hierarchical low-rank approximation of fill-ins reduces both memory usage and computational time during elimination.
• The solver demonstrates strong performance as a preconditioner in GMRES, enabling faster convergence for iterative methods.
• The approach is fully algebraic and does not require problem-specific knowledge, making it broadly applicable across sparse matrix types.
• The method scales efficiently to large systems where traditional direct solvers become infeasible due to cubic complexity.