[Paper Review] Towards Probabilistic Tensor Canonical Polyadic Decomposition 2.0: Automatic Tensor Rank Learning Using Generalized Hyperbolic Prior.

Lei Cheng, Zhongtao Chen+3 more|arXiv (Cornell University)|Sep 5, 2020

Tensor decomposition and applications44 references6 citations

TL;DR

This paper proposes a generalized hyperbolic (GH) prior for automatic tensor rank learning in probabilistic canonical polyadic decomposition (CPD), enabling robust rank estimation across low- and high-rank tensors even at low signal-to-noise ratios (SNRs). By extending the Gaussian-gamma model with greater flexibility for sparsity, the method uses variational inference with closed-form updates, outperforming existing approaches in synthetic and real-world data experiments.

ABSTRACT

Tensor rank learning for canonical polyadic decomposition (CPD) has long been deemed as an essential but challenging problem. In particular, since the tensor rank controls the complexity of the CPD model, its inaccurate learning would cause overfitting to noise or underfitting to the signal sources, and even destroy the interpretability of model parameters. However, the optimal determination of a tensor rank is known to be a non-deterministic polynomial-time hard (NP-hard) task. Rather than exhaustively searching for the best tensor rank via trial-and-error experiments, Bayesian inference under the Gaussian-gamma prior was introduced in the context of probabilistic CPD modeling and it was shown to be an effective strategy for automatic tensor rank determination. This triggered flourishing research on other structured tensor CPDs with automatic tensor rank learning. As the other side of the coin, these research works also reveal that the Gaussian-gamma model does not perform well for high-rank tensors or/and low signal-to-noise ratios (SNRs). To overcome these drawbacks, in this paper, we introduce a more advanced generalized hyperbolic (GH) prior to the probabilistic CPD model, which not only includes the Gaussian-gamma model as a special case, but also provides more flexibilities to adapt to different levels of sparsity. Based on this novel probabilistic model, an algorithm is developed under the framework of variational inference, where each update is obtained in a closed-form. Extensive numerical results, using synthetic data and real-world datasets, demonstrate the excellent performance of the proposed method in learning both low as well as high tensor ranks even for low SNR cases.

Motivation & Objective

• To address the challenge of automatic tensor rank determination in canonical polyadic decomposition (CPD), which is NP-hard and critical for model interpretability and generalization.
• To overcome the limitations of the Gaussian-gamma prior in handling high-rank tensors and low signal-to-noise ratios (SNRs), where it underperforms.
• To develop a more flexible probabilistic CPD model that adapts to varying levels of sparsity in tensor components.
• To enable closed-form variational inference updates for efficient and scalable rank learning.
• To demonstrate superior performance across diverse data regimes, including low-SNR and high-rank scenarios.

Proposed method

• Introduces a generalized hyperbolic (GH) prior as a hierarchical prior over CPD factor matrices, subsuming the Gaussian-gamma prior as a special case.
• Models the CPD parameters using a hierarchical Bayesian framework where the GH prior enables adaptive sparsity control across different tensor ranks.
• Derives closed-form update equations for variational inference by exploiting the conjugacy properties of the GH prior.
• Employs a variational Bayesian inference framework to jointly estimate the tensor rank and CPD parameters in a fully automatic manner.
• Leverages the heavy-tailed nature of the GH distribution to better capture sparse and complex structures in high-rank or low-SNR data.
• Optimizes the variational lower bound (ELBO) through iterative updates of the posterior parameters, ensuring convergence and scalability.

Experimental results

Research questions

• RQ1 Can a more flexible prior than the Gaussian-gamma model improve automatic tensor rank learning in probabilistic CPD?
• RQ2 How does the generalized hyperbolic prior perform in estimating high tensor ranks compared to existing methods?
• RQ3 Does the proposed method maintain robustness under low signal-to-noise ratio (SNR) conditions?
• RQ4 To what extent does the GH prior adapt to varying levels of sparsity in the underlying tensor components?
• RQ5 Can closed-form variational inference be achieved with the GH prior to ensure computational efficiency?

Key findings

• The proposed generalized hyperbolic (GH) prior significantly improves tensor rank learning performance over the Gaussian-gamma prior, especially in high-rank and low-SNR scenarios.
• The method achieves accurate rank estimation across both synthetic and real-world datasets, demonstrating robustness to noise and complexity.
• Extensive numerical results confirm that the GH prior enables effective adaptation to diverse sparsity levels in tensor components.
• The closed-form variational inference updates ensure computational efficiency and scalability, enabling practical deployment.
• The model outperforms existing approaches in terms of rank estimation accuracy, particularly when the true rank is high or the SNR is low.
• The GH prior generalizes the Gaussian-gamma model and provides a more versatile framework for automatic tensor rank learning in probabilistic CPD.