[Paper Review] Alternating minimization algorithms for graph regularized tensor completion

Yu Guan, Shuyu Dong+3 more|arXiv (Cornell University)|Aug 28, 2020

Tensor decomposition and applications44 references4 citations

TL;DR

This paper proposes alternating minimization and ADMM algorithms for graph-regularized tensor completion using CP decomposition, leveraging graph Laplacian regularization to improve low-rank tensor recovery. The methods achieve superior accuracy and efficiency, especially at low sampling rates, with global convergence proven via the Kurdyka-Łojasiewicz property.

ABSTRACT

We consider a Canonical Polyadic (CP) decomposition approach to low-rank tensor completion (LRTC) by incorporating external pairwise similarity relations through graph Laplacian regularization on the CP factor matrices. The usage of graph regularization entails benefits in the learning accuracy of LRTC, but at the same time, induces coupling graph Laplacian terms that hinder the optimization of the tensor completion model. In order to solve graph-regularized LRTC, we propose efficient alternating minimization algorithms by leveraging the block structure of the underlying CP decomposition-based model. For the subproblems of alternating minimization, a linear conjugate gradient subroutine is specifically adapted to graph-regularized LRTC. Alternatively, we circumvent the complicating coupling effects of graph Laplacian terms by using an alternating directions method of multipliers. Based on the Kurdyka-Łojasiewicz property, we show that the sequence generated by the proposed algorithms globally converges to a critical point of the objective function. Moreover, the complexity and convergence rate are also derived. In addition, numerical experiments including synthetic data and real data show that the graph regularized tensor completion model has improved recovery results compared to those without graph regularization, and that the proposed algorithms achieve gains in time efficiency over existing algorithms.

Motivation & Objective

• To address low-rank tensor completion under partial observations by incorporating external similarity information via graph regularization.
• To overcome the computational challenge of coupling graph Laplacian terms in optimization by exploiting block structure in CP decomposition.
• To develop efficient algorithms—alternating minimization with linear CG and ADMM—that handle graph-regularized tensor completion effectively.
• To establish global convergence and derive convergence rates using the Kurdyka-Łojasiewicz property.
• To empirically validate the proposed model and algorithms on synthetic and real-world datasets, showing improved recovery performance.

Proposed method

• Formulates a CP-based tensor completion model with graph Laplacian regularization on factor matrices to exploit pairwise similarity relations.
• Develops an alternating minimization algorithm where each subproblem is solved via a linear conjugate gradient (CG) method with efficient Hessian-vector products.
• Introduces an ADMM-based alternative to decouple graph regularization effects and improve optimization stability.
• Uses the Kurdyka-Łojasiewicz property to prove global convergence of the alternating minimization sequence to a critical point.
• Adapts the linear CG subroutine to the block-structured subproblems arising from CP decomposition.
• Employs graph Laplacian matrices derived from similarity graphs on tensor modes to regularize factor matrices.

Experimental results

Research questions

• RQ1 Can graph regularization improve tensor completion accuracy when only a small fraction of entries are observed?
• RQ2 How can the coupling effect of graph Laplacian terms be effectively mitigated in alternating minimization for tensor completion?
• RQ3 What optimization algorithms are most effective for solving graph-regularized CP tensor completion, and how do they compare in convergence and efficiency?
• RQ4 Does the proposed method achieve global convergence, and what is the theoretical convergence rate?
• RQ5 How does the performance of the proposed algorithms compare to state-of-the-art methods in terms of recovery accuracy and time efficiency?

Key findings

• The graph-regularized tensor completion model significantly improves recovery accuracy over non-regularized models, especially at low sampling rates (e.g., 0.3% and 0.5%).
• The proposed AltMin-CG and ADMM algorithms achieve the lowest test RMSEs on synthetic and real datasets, outperforming state-of-the-art methods like AirCP, TFAI, and BPTF.
• AltMin-CG and ADMM demonstrate superior time efficiency, ranking among the fastest algorithms in terms of training error reduction per iteration.
• The algorithms show robust performance on the MovieLens and FIA datasets, maintaining low RMSE even with sparse observations.
• Numerical experiments confirm global convergence of the alternating minimization algorithm, supported by theoretical analysis using the Kurdyka-Łojasiewicz property.
• The linear CG subroutine with efficient Hessian-vector multiplication enables fast subproblem solutions, enhancing overall algorithmic efficiency.