[论文解读] Provable Tensor-Train Format Tensor Completion by Riemannian Optimization

Jian-Feng Cai, Jingyang Li+1 more|arXiv (Cornell University)|Aug 27, 2021

Tensor decomposition and applications参考文献 75被引用 15

一句话总结

本文首次为张量环（TT）格式张量补全中的黎曼梯度下降（RGrad）提供了理论收敛保证，证明了其具有与张量条件数无关的常数收缩率的线性收敛性。本文提出一种顺序二阶矩方法用于热初始化，并在子高斯噪声下建立了近似最优的统计恢复率，数值实验验证了其计算优势。

ABSTRACT

The tensor train (TT) format enjoys appealing advantages in handling structural high-order tensors. The recent decade has witnessed the wide applications of TT-format tensors from diverse disciplines, among which tensor completion has drawn considerable attention. Numerous fast algorithms, including the Riemannian gradient descent (RGrad), have been proposed for the TT-format tensor completion. However, the theoretical guarantees of these algorithms are largely missing or sub-optimal, partly due to the complicated and recursive algebraic operations in TT-format decomposition. Moreover, existing results established for the tensors of other formats, for example, Tucker and CP, are inapplicable because the algorithms treating TT-format tensors are substantially different and more involved. In this paper, we provide, to our best knowledge, the first theoretical guarantees of the convergence of RGrad algorithm for TT-format tensor completion, under a nearly optimal sample size condition. The RGrad algorithm converges linearly with a constant contraction rate that is free of tensor condition number without the necessity of re-conditioning. We also propose a novel approach, referred to as the sequential second-order moment method, to attain a warm initialization under a similar sample size requirement. As a byproduct, our result even significantly refines the prior investigation of RGrad algorithm for matrix completion. Lastly, statistically (near) optimal rate is derived for RGrad algorithm if the observed entries consist of random sub-Gaussian noise. Numerical experiments confirm our theoretical discovery and showcase the computational speedup gained by the TT-format decomposition.

研究动机与目标

• 填补黎曼优化在TT格式张量补全中收敛性分析的理论空白。
• 在近乎最优的样本量条件下，提供RGrad算法的可证明线性收敛性。
• 通过顺序二阶矩方法开发一种新型热初始化方法，其样本量要求相似。
• 在随机子高斯噪声下，推导RGrad的统计上近似最优的恢复率。
• 通过数值实验展示计算效率和鲁棒性。

提出的方法

• 在TT流形上提出黎曼梯度下降（RGrad）以优化低秩张量补全。
• 引入一种顺序二阶矩方法，在相同样本量条件下以高概率实现热初始化。
• 利用矩阵伯恩斯坦不等式和浓度不等式控制经验估计与期望之间的偏差。
• 采用谱初始化和矩阵浓度技术，控制TT格式中估计误差的范数。
• 通过收缩率分析研究收敛性，表明其独立于张量条件数。
• 在子高斯噪声假设下，推导RGrad迭代误差的高概率界。

实验结果

研究问题

• RQ1 TT格式张量补全的RGrad能否在理论上保证以常数收缩率实现线性收敛？
• RQ2 何种样本量足以确保TT格式张量补全的收敛性和初始化质量？
• RQ3 在随机子高斯噪声下，RGrad算法是否能实现统计上近似最优的恢复率？
• RQ4 在不重新条件化的情况下，收敛率能否独立于张量条件数？
• RQ5 所提出的热初始化方法在样本复杂度和鲁棒性方面与现有谱初始化相比如何？

主要发现

• RGrad以与张量条件数无关的常数收缩率实现线性收敛，消除了重新条件化的需要。
• 该算法在近乎最优的样本量条件下实现收敛，与信息论极限仅相差对数因子。
• 所提出的顺序二阶矩方法在相同样本量要求下，以高概率确保热初始化。
• 当观测条目受随机子高斯噪声污染时，该方法实现了统计上近似最优的恢复率。
• 数值实验验证了理论结果，并展示了由于TT格式结构带来的显著计算加速。
• 该分析改进了先前关于矩阵补全的研究，将其推广至张量情形并提供了更强的保证。