[Paper Review] Note: low-rank tensor train completion with side information based on Riemannian optimization

Stanislav Budzinskiy, Н. Л. Замарашкин|arXiv (Cornell University)|Jun 23, 2020

Tensor decomposition and applications10 references4 citations

TL;DR

This paper proposes RTTC-SI, a Riemannian optimization algorithm for low-rank tensor train (TT) completion that incorporates side information in the form of subspace constraints on mode fibers. By leveraging a one-to-one correspondence between the original tensor and a smaller tensor in the side information subspace, the method achieves robust recovery with significantly fewer observed entries than standard TT completion, demonstrating an O(M log M) phase transition threshold similar to matrix completion theory.

ABSTRACT

We consider the low-rank tensor train completion problem when additional side information is available in the form of subspaces that contain the mode-$k$ fiber spans. We propose an algorithm based on Riemannian optimization to solve the problem. Numerical experiments show that the proposed algorithm requires far fewer known entries to recover the tensor compared to standard tensor train completion methods.

Motivation & Objective

• To address the low-rank tensor train completion problem when additional side information is available in the form of subspaces constraining mode- k fiber spans.
• To develop a Riemannian optimization-based algorithm that exploits this side information to reduce the number of required observed tensor entries.
• To analyze how side information affects the phase transition behavior in tensor completion, particularly the threshold of observed entries needed for successful recovery.
• To extend existing matrix completion results with side information to the tensor train format, particularly in terms of sample complexity.

Proposed method

• The method maps the original tensor completion problem to a smaller tensor space via orthogonal projections defined by side information matrices Q_k, transforming the feasible set to N_r(Q).
• It formulates the completion problem as minimizing a least-squares objective over the manifold of low-rank TT tensors in the reduced space, using the transformation X = QY.
• Riemannian optimization is applied on the manifold M_r of low-rank TT tensors in the smaller space, using Riemannian gradient descent with retraction and vector transport.
• The Riemannian gradient is computed via Euclidean gradient projection onto the tangent space of the TT manifold, adapted to the side information constraints.
• A retraction based on TT-SVD is used, which preserves the subspace constraints and ensures convergence within the feasible set N_r(Q).
• The algorithm, named RTTC-SI, inherits the computational efficiency of RTTC but is enhanced to respect the side information, enabling faster and more robust convergence.

Experimental results

Research questions

• RQ1 Can side information in the form of constrained mode- k fiber subspaces reduce the number of required observed entries for low-rank tensor train completion?
• RQ2 Does incorporating side information via Riemannian optimization lead to a phase transition threshold closer to the theoretical minimum, such as O(M log M)?
• RQ3 How does the performance of the RTTC-SI algorithm compare to standard RTTC when side information is present but not exploited?
• RQ4 What is the impact of partial side information—available only for a subset of modes—on the recovery threshold?

Key findings

• RTTC-SI achieves successful tensor recovery with far fewer observed entries than standard RTTC, even when the number of known entries is below the typical threshold for standard methods.
• The phase transition for RTTC-SI occurs at |Ω| = O(M log M), matching theoretical bounds observed in matrix completion with side information.
• Numerical experiments show that the required number of observed entries for convergence is independent of the original tensor dimension N when side information is used.
• When side information is available for only a subset of modes, the recovery threshold increases with the number of modes lacking constraints, but remains significantly lower than standard TT completion.
• The algorithm maintains robustness and convergence even with sparse data, as demonstrated by phase plots showing a sharp transition from failure to success at low sampling rates.
• The RTTC-SI algorithm outperforms standard RTTC even when the initial approximation respects the side information, proving that the optimization method effectively exploits the constraints.