[Paper Review] Computational linear algebra over finite fields

Jean‐Guillaume Dumas, Clément Pernet|arXiv (Cornell University)|Apr 17, 2012

Coding theory and cryptography57 references3 citations

TL;DR

This paper presents optimized algorithms for dense linear algebra over finite fields, focusing on efficient matrix multiplication, inversion, and decomposition using bitpacking, bit-slicing, and Kronecker substitution techniques. It achieves sub-cubic complexity for matrix multiplication over small fields like F₂, reducing the time complexity to O(n³/log n), and integrates these methods into high-performance libraries such as M4RI, Magma, and Sage.

ABSTRACT

We present here algorithms for efficient computation of linear algebra problems over finite fields.

Motivation & Objective

• Address the performance bottleneck in dense linear algebra over finite fields by optimizing low-level arithmetic and data representation.
• Develop algorithms that reduce the computational complexity of matrix multiplication below the standard O(n³) bound, especially for small finite fields.
• Enable high-performance implementations through compact representations (bitpacking, bit-slicing, Kronecker substitution) and recursive block algorithms.
• Generalize standard linear algebra operations (LU, inversion, triangular solves) to handle arbitrary rank profiles via PLE decomposition.
• Integrate the proposed algorithms into existing high-performance computing systems and libraries for practical use in cryptography and coding theory.

Proposed method

• Use bitpacking to store multiple field elements per machine word, enabling parallel operations on 64-bit words for F₂ and small extension fields.
• Apply bit-slicing to represent elements of F₃, F₅, F₇, and F_{2³}, F_{3²} using boolean operations on multiple bits simultaneously, minimizing arithmetic cost.
• Employ Kronecker substitution to represent polynomial elements in extension fields as integers, enabling efficient reduction via the REDQ algorithm.
• Design recursive block algorithms (e.g., PLE, trsm, trmm, trtri) that reduce linear algebra problems to matrix multiplication, achieving complexity bounds matching the best-known matrix multiplication algorithms.
• Use the method of the four Russians (Greasing) for F₂, achieving O(n³/log n) complexity by precomputing all 2ᵏ linear combinations of rows.
• Leverage recursive block decomposition with conformal partitioning to express triangular and matrix multiplication operations in terms of smaller recursive calls and GEMM subroutines.

Experimental results

Research questions

• RQ1 Can matrix multiplication over F₂ be accelerated beyond O(n³) using precomputation and bit-level parallelism?
• RQ2 How can compact representations (bitpacking, bit-slicing) reduce the cost of finite field arithmetic in dense linear algebra?
• RQ3 What is the achievable complexity of matrix multiplication over small finite fields using advanced data representations and recursive algorithms?
• RQ4 Can the PLE decomposition be computed in matrix multiplication time for arbitrary rank profiles, and how does it generalize PLU decomposition?
• RQ5 To what extent can recursive block algorithms for triangular solves and inversions reduce the number of field operations while maintaining numerical stability over finite fields?

Key findings

• Matrix multiplication over F₂ achieves O(n³/log n) complexity using the method of the four Russians (Greasing), with a bit operation count reduced from k·2ᵏ·n to 2ᵏ·n via Gray code ordering.
• Bit-slicing enables efficient arithmetic over F₃, F₅, and F₇ with only 6, 20, and 17 boolean operations per addition, respectively.
• The REDQ algorithm allows efficient reduction of coefficients modulo a prime using 64-bit integer arithmetic, enabling compact representation of field elements.
• The PLE decomposition algorithm reduces the complexity of rank-revealing Gaussian elimination to that of matrix multiplication, achieving O(M(n)) complexity where M(n) is the time to multiply two n×n matrices.
• Recursive triangular solve (trsm), matrix multiplication (trmm), and inversion (trtri) algorithms are implemented using block decomposition and GEMM subroutines, achieving optimal asymptotic complexity.
• The proposed algorithms are integrated into high-performance libraries including M4RI, Magma, Sage, NTL, FLINT, and IML, enabling practical use in computational algebra and cryptography.