[Paper Review] A Highly Efficient Implementation of Multiple Precision Sparse Matrix-Vector Multiplication and Its Application to Product-type Krylov Subspace Methods

Tomonori Kouya|arXiv (Cornell University)|Nov 10, 2014

Numerical Methods and Algorithms2 references3 citations

TL;DR

This paper presents a highly optimized multiple precision sparse matrix-vector multiplication (SpMV) implementation within the BNCpack library using MPFR/GMP, significantly reducing memory usage and computational time. The method enables efficient solution of large-scale, ill-conditioned linear systems via product-type Krylov subspace methods like BiCG and GPBiCG, demonstrating up to 67.48 seconds in reduced computational time for 8192-bit precision on the cavity04 matrix.

ABSTRACT

We evaluate the performance of the Krylov subspace method by using highly efficient multiple precision sparse matrix-vector multiplication (SpMV). BNCpack is our multiple precision numerical computation library based on MPFR/GMP, which is one of the most efficient arbitrary precision floating-point arithmetic libraries. However, it does not include functions that can manipulate multiple precision sparse matrices. Therefore, by using benchmark tests, we show that SpMV implemented in these functions can be more efficient. Finally, we also show that product-type Krylov subspace methods such as BiCG and GPBiCG in which we have embedded SpMV, can efficiently solve large-scale linear systems of equations provided in the UF sparse matrix collections in a memory-restricted computing environment.

Motivation & Objective

• Address the lack of efficient multiple precision sparse matrix libraries for high-accuracy scientific computing.
• Overcome memory and performance bottlenecks in solving large-scale, ill-conditioned linear systems using standard double precision arithmetic.
• Develop and benchmark a high-performance multiple precision SpMV kernel tailored for sparse matrices in memory-constrained environments.
• Integrate the SpMV into product-type Krylov subspace solvers (e.g., BiCG, GPBiCG) to improve convergence and efficiency for high-precision problems.
• Demonstrate the practicality of multiple precision computation on commodity hardware using real sparse matrices from the UF Sparse Matrix Collection.

Proposed method

• Implemented a multiple precision sparse matrix structure using the Compressed Sparse Row (CSR) format with MPFR/GMP's arbitrary-precision floating-point arithmetic.
• Leveraged MPFR/GMP’s optimized zero-multiplication and precision-flexible arithmetic to accelerate SpMV operations.
• Designed a custom sparse matrix data type (`mpfrsmatrix`) storing precision, dimensions, nonzero elements, and column/row index arrays.
• Benchmarked SpMV against dense matrix-vector multiplication (Dense MV) on standard PCs to quantify performance gains.
• Integrated the SpMV kernel into BNCpack’s product-type Krylov subspace solvers (BiCG, GPBiCG, BiCGSTAB) with and without ILU(0) preconditioning.
• Used matrices from the UF Sparse Matrix Collection (e.g., cavity04, epb3) to evaluate scalability and convergence across varying precision levels (512 to 8192 bits).

Experimental results

Research questions

• RQ1 How does multiple precision SpMV compare to dense matrix-vector multiplication in terms of computational efficiency and memory usage?
• RQ2 Can multiple precision SpMV significantly improve the performance of product-type Krylov subspace methods in memory-constrained environments?
• RQ3 What precision level is required for convergence in ill-conditioned problems like cavity04 and epb3, and how does SpMV affect iterative count and runtime?
• RQ4 Does preconditioning (e.g., ILU(0)) enhance convergence in multiple precision Krylov methods, and is it computationally efficient compared to higher-precision direct computation?
• RQ5 To what extent can commodity PCs handle large-scale multiple precision sparse linear systems using the proposed SpMV implementation?

Key findings

• The multiple precision SpMV implementation reduced computational time by up to 67.48 seconds for the GPBiCG method on the cavity04 matrix at 8192-bit precision.
• For the epb3 matrix (84,617×84,617, 99.0035% sparsity), the SpMV-based GPBiCG method achieved convergence at 8192-bit precision with 24,179 seconds, while dense matrix storage would require ~70 TB of memory.
• The cavity04 problem required at least 2048-bit precision for convergence in product-type Krylov methods, with BiCGSTAB and GPBiCG showing improved performance at higher precision.
• The SpMV kernel achieved significant speedups over dense MV, especially at higher precision levels, due to reduced memory footprint and optimized zero-multiplication in MPFR/GMP.
• Preconditioning with ILU(0) did not improve efficiency in multiple precision; the 4096-bit BiCGSTAB method without preconditioning was faster (42.79s) than the preconditioned version (109.29s).
• The sparse matrix format reduced memory usage from ~320 GB (128-bit dense) to ~3.5 MB (128-bit sparse) for the epb3 matrix, enabling feasible computation on standard PCs.