[Paper Review] On fast matrix-vector multiplication with a Hankel matrix in multiprecision arithmetics

Gleb Beliakov|arXiv (Cornell University)|Feb 21, 2014

Matrix Theory and Algorithms9 references3 citations

TL;DR

This paper presents two efficient algorithms for fast matrix-vector multiplication with Hankel matrices in multiprecision arithmetic, where standard FFT-based methods become inefficient due to high-precision overhead. The first method decomposes multiprecision numbers into sums for use with standard FFT routines; the second uses a Karatsuba-inspired recursive approach with O(n^log 3) complexity, offering better performance and accuracy in high-precision contexts.

ABSTRACT

We present two fast algorithms for matrix-vector multiplication $y=Ax$, where $A$ is a Hankel matrix. The current asymptotically fastest method is based on the Fast Fourier Transform (FFT), however in multiprecision arithmetics with very high accuracy FFT method is actually slower than schoolbook multiplication for matrix sizes up to $n=8000$. One method presented is based on a decomposition of multiprecision numbers into sums, and applying standard or double precision FFT. The second method, inspired by Karatsuba multiplication, is based on recursively performing multiplications with matrices of half-size of the original. Its complexity in terms of the matrix size $n$ is $Θ(n^{\log 3})$. Both methods are applicable to Toeplitz matrices and to circulant matrices.

Motivation & Objective

• Address the inefficiency of FFT-based matrix-vector multiplication in multiprecision arithmetic due to high computational overhead and nested FFT calls.
• Develop alternative algorithms that outperform standard FFT and schoolbook multiplication for large matrix sizes in high-precision environments.
• Ensure numerical accuracy by avoiding intermediate precision loss common in standard FFT when applied to multiprecision numbers.
• Design a method that is both simple to implement and amenable to parallelization for high-performance computing.
• Extend applicability to Toeplitz and circulant matrices through structural equivalence with Hankel matrices.

Proposed method

• Decompose each multiprecision number in the Hankel matrix and vector into a sum of double-precision components to enable use of highly optimized standard-precision FFT libraries.
• Reconstruct the final result from the component-wise FFT results, ensuring full-precision output without intermediate rounding.
• Propose a recursive algorithm inspired by Karatsuba multiplication that reduces four multiplications to three per recursive step, reducing the total number of multiplications.
• Structure the algorithm using recursive splitting of the input vector and Hankel matrix into smaller subproblems of size n/2, with auxiliary vectors formed via linear combinations.
• Use a divide-and-conquer approach where the product is computed as a combination of three recursive calls on smaller Hankel matrices, followed by addition/subtraction steps.
• Enable parallel execution by independently launching the three recursive calls on separate threads, with balanced work distribution and no load imbalance.

Experimental results

Research questions

• RQ1 Can FFT-based matrix-vector multiplication be effectively adapted to multiprecision arithmetic without incurring prohibitive overheads?
• RQ2 Does a Karatsuba-inspired recursive algorithm with O(n^log 3) complexity outperform both FFT and schoolbook methods in high-precision settings?
• RQ3 Can the proposed algorithms maintain full precision while avoiding cancellation errors common in standard FFT when applied to multiprecision numbers?
• RQ4 To what extent can the recursive algorithm be parallelized without introducing load imbalance or significant synchronization costs?
• RQ5 How do the computational costs of addition and multiplication scale in multiprecision arithmetic, and can they be leveraged to favor algorithms with fewer multiplications?

Key findings

• The FFT-based method using decomposition into double-precision components achieves O(n log n) complexity and leverages existing optimized FFT libraries, though it risks accuracy loss due to cancellation in intermediate steps.
• The Karatsuba-inspired recursive algorithm achieves O(n^log 3) complexity, with a theoretical bound of at most 3n^log 3 multiplications and 4n^log 3 additions for n > 2.
• The recursive algorithm is computationally more efficient than schoolbook multiplication for all n > 3, regardless of the relative cost of addition and multiplication in multiprecision arithmetic.
• The algorithm is inherently parallelizable, with three independent recursive calls that can be executed in parallel on separate threads or machines, each handling equal work.
• The method ensures full-precision computation throughout, avoiding the accuracy degradation associated with intermediate precision reduction in the FFT decomposition approach.
• Empirical evaluation is planned to compare the performance of both methods in practice, particularly for matrix sizes up to n = 8000 and precision levels of 10,000 decimal places.