[论文解读] Towards Probabilistic Tensor Canonical Polyadic Decomposition 2.0: Automatic Tensor Rank Learning Using Generalized Hyperbolic Prior.

Lei Cheng, Zhongtao Chen+3 more|arXiv (Cornell University)|Sep 5, 2020

Tensor decomposition and applications参考文献 44被引用 6

一句话总结

本文提出了一种广义柯西（GH）先验，用于在概率CP分解（CPD）中实现自动张量秩学习，即使在低信噪比（SNR）条件下，也能在低秩和高秩张量中实现稳健的秩估计。通过在高斯-伽马模型基础上增强稀疏性灵活性，该方法采用具有闭式更新的变分推断，其在合成数据和真实世界数据实验中优于现有方法。

ABSTRACT

Tensor rank learning for canonical polyadic decomposition (CPD) has long been deemed as an essential but challenging problem. In particular, since the tensor rank controls the complexity of the CPD model, its inaccurate learning would cause overfitting to noise or underfitting to the signal sources, and even destroy the interpretability of model parameters. However, the optimal determination of a tensor rank is known to be a non-deterministic polynomial-time hard (NP-hard) task. Rather than exhaustively searching for the best tensor rank via trial-and-error experiments, Bayesian inference under the Gaussian-gamma prior was introduced in the context of probabilistic CPD modeling and it was shown to be an effective strategy for automatic tensor rank determination. This triggered flourishing research on other structured tensor CPDs with automatic tensor rank learning. As the other side of the coin, these research works also reveal that the Gaussian-gamma model does not perform well for high-rank tensors or/and low signal-to-noise ratios (SNRs). To overcome these drawbacks, in this paper, we introduce a more advanced generalized hyperbolic (GH) prior to the probabilistic CPD model, which not only includes the Gaussian-gamma model as a special case, but also provides more flexibilities to adapt to different levels of sparsity. Based on this novel probabilistic model, an algorithm is developed under the framework of variational inference, where each update is obtained in a closed-form. Extensive numerical results, using synthetic data and real-world datasets, demonstrate the excellent performance of the proposed method in learning both low as well as high tensor ranks even for low SNR cases.

研究动机与目标

• 为解决在典型多维阵列分解（CPD）中自动张量秩确定的挑战，该问题为NP难问题，且对模型可解释性和泛化能力至关重要。
• 克服高斯-伽马先验在处理高秩张量和低信噪比（SNR）条件下的局限性，后者在此类场景下表现欠佳。
• 开发一种更具灵活性的概率CPD模型，以适应张量分量中不同水平的稀疏性。
• 实现闭式变分推断更新，以确保高效且可扩展的秩学习。
• 在多种数据场景下展示优越性能，包括低SNR和高秩情形。

提出的方法

• 引入广义柯西（GH）先验作为CPD因子矩阵的层次先验，其可将高斯-伽马先验作为特例包含在内。
• 在层次贝叶斯框架下建模CPD参数，其中GH先验可实现对不同张量秩下自适应稀疏性的控制。
• 通过利用GH先验的共轭性质，推导出变分推断的闭式更新方程。
• 采用变分贝叶斯推断框架，以完全自动的方式联合估计张量秩和CPD参数。
• 利用GH分布的重尾特性，更好地捕捉高秩或低SNR数据中稀疏且复杂的结构。
• 通过迭代更新后验参数来优化变分下界（ELBO），确保收敛性和可扩展性。

实验结果

研究问题

• RQ1 相较于高斯-伽马模型，更灵活的先验是否能提升概率CPD中的自动张量秩学习性能？
• RQ2 与现有方法相比，广义柯西先验在估计高张量秩时表现如何？
• RQ3 所提出的方法在低信噪比（SNR）条件下是否仍保持鲁棒性？
• RQ4 GH先验在多大程度上能适应底层张量分量中不同水平的稀疏性？
• RQ5 能否在GH先验下实现闭式变分推断，以确保计算效率？

主要发现

• 所提出的广义柯西（GH）先验在性能上显著优于高斯-伽马先验，尤其在高秩和低SNR场景下。
• 该方法在合成数据和真实世界数据集上均实现了准确的秩估计，表现出对噪声和复杂性的鲁棒性。
• 大量数值结果证实，GH先验能有效适应张量分量中多样的稀疏水平。
• 闭式变分推断更新确保了计算效率和可扩展性，支持实际部署。
• 在秩估计准确性方面，该模型优于现有方法，尤其当真实秩较高或SNR较低时。
• GH先验推广了高斯-伽马模型，为概率CPD中的自动张量秩学习提供了更通用的框架。