[Paper Review] Simple exponential acceleration of the power iteration algorithm

Congzhou M. Sha, Nikolay V. Dokholyan|arXiv (Cornell University)|Sep 22, 2021

Matrix Theory and Algorithms10 references4 citations

TL;DR

This paper proposes a modified power iteration algorithm that achieves exponential convergence by leveraging matrix exponentiation through squaring, reducing the number of iterations from O(k) to O(log k) while maintaining polynomial computational cost. The method accelerates convergence without sacrificing robustness, making it a practical, drop-in replacement for traditional power iteration in eigenvalue computation.

ABSTRACT

Code to reproduce data in: https://arxiv.org/abs/2109.10884

Motivation & Objective

• To address the slow convergence of traditional power iteration in finding the largest eigenvalue and eigenvector of a matrix.
• To develop a method that maintains numerical stability while drastically reducing the number of iterations required for convergence.
• To provide a practical, efficient alternative to standard power iteration that can be easily integrated into existing computational pipelines.
• To extend the method to compute the k largest eigenvalues and eigenvectors for self-adjoint matrices using a modified Gram-Schmidt orthonormalization process.

Proposed method

• The algorithm combines power iteration with matrix exponentiation by squaring, computing high powers of matrix A via repeated squaring to accelerate convergence.
• At each step, the matrix is normalized to prevent numerical overflow, ensuring stability during iterative squaring.
• The final eigenvector is obtained by applying the computed high-power matrix to a randomly initialized vector x₀.
• For self-adjoint matrices, the method uses a deflation technique based on Gram-Schmidt orthonormalization to extract the k largest eigenpairs sequentially.
• The algorithm uses the Rayleigh quotient to estimate the largest eigenvalue from the final vector, ensuring high accuracy.
• The approach relies on the mathematical property that components along smaller eigenvalues decay exponentially with each squaring step.

Experimental results

Research questions

• RQ1 Can matrix exponentiation by squaring be used to accelerate the convergence of the power iteration algorithm?
• RQ2 Does the proposed method achieve exponential convergence while maintaining polynomial computational complexity?
• RQ3 How does the performance of the modified algorithm compare to standard power iteration in terms of iteration count and runtime?
• RQ4 Can the method be extended to compute multiple largest eigenpairs efficiently for self-adjoint matrices?
• RQ5 Is the proposed algorithm robust and practical for real-world matrices of moderate size?

Key findings

• The proposed algorithm achieves exponential convergence, reducing the number of iterations from O(k) in standard power iteration to O(log k).
• For 100×100 matrices, Algorithm 3 converged within 20 iterations in all tested cases, demonstrating exponential speedup.
• The method outperforms standard power iteration in both convergence rate and runtime on moderate-sized matrices (n=100).
• The algorithm maintains numerical stability through matrix normalization at each squaring step, preventing overflow.
• The computational cost is O(n³ log k), which is more efficient than the O(n²k) cost of standard power iteration for large k.
• The method is robust to random initialization of the starting vector and avoids degenerate cases with high probability.