[Paper Review] The eigenstructures of real (skew) circulant matrices with some applications

Zhongyun Liu, Siheng Chen+2 more|arXiv (Cornell University)|Jun 12, 2018

Matrix Theory and Algorithms12 references3 citations

TL;DR

This paper proposes a DCT-DST-based real Schur form for real circulant and skew-circulant matrices, enabling fast, real-arithmetic-only algorithms for matrix-vector multiplication and a novel CSCS iteration for solving real positive definite Toeplitz systems. The method reduces computational cost by half and saves 50% storage compared to FFT-based approaches, with numerical results showing superior efficiency for large-scale problems.

ABSTRACT

The circulant matrices and skew-circulant matrices are two special classes of Toeplitz matrices and play vital roles in the computation of Toeplitz matrices. In this paper, we focus on real circulant and skewcirculant matrices. We first investigate their real Schur forms, which are closely related to the family of discrete cosine transform (DCT) and discrete sine transform (DST). Using those real Schur forms, we then develop some fast algorithms for computing real circulant, skew-circulant and Toeplitz matrix-real vector multiplications. Also, we develop a DCT-DST version of circulant and skew-circulant splitting (CSCS) iteration for real positive definite Toeplitz systems. Compared with the fast Fourier transform (FFT) version of CSCS iteration, the DCTDST version is more efficient and saves a half storage. Numerical experiments are presented to illustrate the effectiveness of our method.

Motivation & Objective

• To develop real Schur forms for real circulant and skew-circulant matrices that avoid complex arithmetic.
• To enable fast matrix-vector multiplication for real Toeplitz systems using only real arithmetic.
• To design a DCT-DST version of the circulant and skew-circulant splitting (CSCS) iteration that is more efficient than the FFT-based version.
• To reduce computational cost and storage requirements in solving large-scale real positive definite Toeplitz systems.
• To demonstrate the superiority of the proposed method over existing FFT-based and AHSS iterative solvers in numerical experiments.

Proposed method

• Derives real Schur forms for real circulant and skew-circulant matrices using discrete cosine transform (DCT) and discrete sine transform (DST) matrices.
• Expresses the real Schur forms as $ C = U \Lambda U^T $ and $ S = \tilde{U} \tilde{\Lambda} \tilde{U}^T $, where $ U $ and $ \tilde{U} $ are orthogonal matrices based on DCT-II and DST-I.
• Develops fast algorithms for computing $ C\mathbf{x} $ and $ S\mathbf{x} $ using only real DCT and DST operations, avoiding complex FFTs.
• Reformulates the CSCS iteration for real Toeplitz systems using DCT and DST, resulting in a real-arithmetic-only iterative solver.
• Utilizes the structure of symmetric and skew-symmetric vectors to simplify the transformation and reduce computational complexity.
• Implements the new CSCS iteration (equation 20) using real DCT and DST, which is highly parallelizable and suitable for multiprocessor architectures.

Experimental results

Research questions

• RQ1 Can real Schur forms for real circulant and skew-circulant matrices be constructed using only real trigonometric transforms?
• RQ2 Can fast matrix-vector multiplication for real circulant and skew-circulant matrices be achieved using DCT and DST instead of FFT?
• RQ3 Can a DCT-DST-based CSCS iteration be developed that avoids complex arithmetic and reduces storage and operations?
• RQ4 How does the performance of the DCT-DST CSCS iteration compare to the FFT-based CSCS and AHSS iterations in terms of convergence and computational cost?
• RQ5 Does the proposed method achieve significant efficiency gains for large-scale real positive definite Toeplitz systems?

Key findings

• The proposed DCT-DST-based CSCS iteration (equation 20) achieves approximately 50% reduction in storage and computational operations compared to the FFT-based CSCS iteration.
• For $ n = 1024 $, the DCT-DST CSCS iteration requires 70.218 seconds, while the FFT-based version takes 11.418 seconds, indicating a significant speedup in practice despite theoretical expectations.
• The spectral radius of the iteration matrix in the DCT-DST CSCS method is much smaller than that of the AHSS iteration, leading to fewer iterations and faster convergence.
• Numerical experiments show that the DCT-DST CSCS method outperforms both the FFT-based CSCS and AHSS iterations, especially for large $ n $, due to reduced complexity and efficient use of real arithmetic.
• The method is highly parallelizable and well-suited for implementation on multiprocessor systems, enhancing its practical utility.
• Theoretical analysis shows that the new method reduces the number of operations by half compared to FFT-based approaches, with further gains expected if native real DCT/DST implementations are used in future MATLAB versions.