[Paper Review] A constructive arbitrary-degree Kronecker product decomposition of tensors

Kim Batselier, Ngai Wong|arXiv (Cornell University)|Jul 31, 2015

Tensor decomposition and applications14 references3 citations

TL;DR

This paper introduces the Tensor Kronecker Product Singular Value Decomposition (TKPSVD), a constructive method that decomposes a real $k$-way tensor into a linear combination of $d$-fold tensor Kronecker products with arbitrary degree $d$. The method preserves structured tensor properties—such as symmetry, Toeplitz, and Hankel—by ensuring each factor inherits the original tensor's structure, enabling efficient multiresolution approximation and compression with provable error bounds.

ABSTRACT

We propose the tensor Kronecker product singular value decomposition~(TKPSVD) that decomposes a real $k$-way tensor $\mathcal{A}$ into a linear combination of tensor Kronecker products with an arbitrary number of $d$ factors $\mathcal{A} = \sum_{j=1}^R σ_j\, \mathcal{A}^{(d)}_j \otimes \cdots \otimes \mathcal{A}^{(1)}_j$. We generalize the matrix Kronecker product to tensors such that each factor $\mathcal{A}^{(i)}_j$ in the TKPSVD is a $k$-way tensor. The algorithm relies on reshaping and permuting the original tensor into a $d$-way tensor, after which a polyadic decomposition with orthogonal rank-1 terms is computed. We prove that for many different structured tensors, the Kronecker product factors $\mathcal{A}^{(1)}_j,\ldots,\mathcal{A}^{(d)}_j$ are guaranteed to inherit this structure. In addition, we introduce the new notion of general symmetric tensors, which includes many different structures such as symmetric, persymmetric, centrosymmetric, Toeplitz and Hankel tensors.

Motivation & Objective

• To develop a general, constructive algorithm for decomposing $k$-way tensors into Kronecker products of arbitrary degree $d$.
• To prove that structured tensors—such as symmetric, Toeplitz, Hankel, and centrosymmetric—preserve their structure in the Kronecker product factors.
• To enable multiresolution image approximation and compression by truncating Kronecker product terms and decomposition ranks.
• To provide a framework for computing relative approximation error without explicitly reconstructing the approximant.
• To introduce the novel concept of general symmetric tensors, unifying various structured tensor types under a common algebraic framework.

Proposed method

• Reshape and permute the original $k$-way tensor into a $d$-way tensor to enable higher-order singular value decomposition.
• Apply a polyadic decomposition with orthogonal rank-1 terms to the reshaped tensor to extract the Kronecker product components.
• Enforce unit Frobenius norm on each factor tensor $\mathcal{A}^{(i)}_j$ and ensure dimension compatibility via $\prod_{i=1}^d n^{(i)}_r = n_r$ for each mode $r$.
• Use the TTr1SVD and HOSVD algorithms for efficient computation, with TTr1SVD showing superior runtime performance.
• Construct the final decomposition as $\mathcal{A} = \sum_{j=1}^R \sigma_j \, \mathcal{A}^{(d)}_j \otimes \cdots \otimes \mathcal{A}^{(1)}_j$, where $\sigma_j$ are singular values.
• Prove that if the original tensor $\mathcal{A}$ is general symmetric (including symmetric, persymmetric, Toeplitz, Hankel), then each factor $\mathcal{A}^{(i)}_j$ inherits the same symmetry type.

Experimental results

Research questions

• RQ1 Can a tensor Kronecker product decomposition be constructed for arbitrary degree $d$ with guaranteed structure preservation in the factors?
• RQ2 Does the TKPSVD method preserve structured properties such as symmetry, Toeplitz, or Hankel form in the Kronecker product components?
• RQ3 How does the truncation of Kronecker product terms and decomposition ranks affect image approximation quality and compression rate?
• RQ4 What is the relationship between the singular values $\sigma_j$ and the relative approximation error in the Frobenius norm?
• RQ5 Can a unified framework be established to generalize symmetric tensors to include multiple structured forms like Toeplitz and Hankel?

Key findings

• The TKPSVD achieves a maximal compression rate of approximately 256 for a $4000 \times 6000 \times 3$ image when retaining only one term and using degree-5 decomposition.
• The PSNR reaches acceptable values (30–50 dB) after retaining just 20 terms in the TKPSVD, corresponding to a compression rate of about 12.8.
• The TTr1SVD algorithm computed the TKPSVD in 18 seconds, significantly faster than the 30-second HOSVD and the 1-minute SVD of a single image slice.
• For the $4000 \times 6000 \times 3$ image, the 1-term approximation achieved PSNR values of 58 dB, 58 dB, and 57 dB at resolutions $250 \times 375$, $500 \times 750$, and $1000 \times 1500$, respectively.
• The algorithm guarantees that if the original tensor is symmetric, persymmetric, centrosymmetric, Toeplitz, Hankel, or general symmetric, then all Kronecker product factors inherit the same structure.
• The relative approximation error is directly computable from the singular values via $\frac{\sqrt{\sigma_{r+1}^2 + \cdots + \sigma_R^2}}{\sqrt{\sigma_1^2 + \cdots + \sigma_R^2}}$, enabling error control without explicit reconstruction.