[Paper Review] Tensor Completion via Tensor QR Decomposition and $L_{2,1}$-Norm Minimization

Yongming Zheng, An-Bao Xu|arXiv (Cornell University)|Nov 9, 2020

Tensor decomposition and applications25 references4 citations

TL;DR

This paper proposes TLNM-TQR, a fast and accurate tensor completion method that replaces tensor singular value decomposition (t-SVD) with a novel tensor QR decomposition-based approximate t-SVD (CTSVD-QR) and uses tensor $L_{2,1}$-norm minimization instead of the tensor nuclear norm. The method achieves faster computation and competitive accuracy, outperforming state-of-the-art algorithms like ADMM-t-SVD in speed while maintaining high recovery precision on video and color image datasets.

ABSTRACT

In this paper, we consider the tensor completion problem, which has many researchers in the machine learning particularly concerned. Our fast and precise method is built on extending the $L_{2,1}$-norm minimization and Qatar Riyal decomposition (LNM-QR) method for matrix completions to tensor completions, and is different from the popular tensor completion methods using the tensor singular value decomposition (t-SVD). In terms of shortening the computing time, t-SVD is replaced with the method computing an approximate t-SVD based on Qatar Riyal decomposition (CTSVD-QR), which can be used to compute the largest $r \left(r>0 ight)$ singular values (tubes) and their associated singular vectors (of tubes) iteratively. We, in addition, use the tensor $L_{2,1}$-norm instead of the tensor nuclear norm to minimize our model on account of it is easy to optimize. Then in terms of improving accuracy, ADMM, a gradient-search-based method, plays a crucial part in our method. Numerical experimental results show that our method is faster than those state-of-the-art algorithms and have excellent accuracy.

Motivation & Objective

• To address the high computational cost of tensor singular value decomposition (t-SVD) in tensor completion, which limits scalability and speed.
• To develop a faster alternative to t-SVD by leveraging QR decomposition for approximate t-SVD computation, named CTSVD-QR.
• To replace the tensor nuclear norm with a more optimizable tensor $L_{2,1}$-norm to improve computational efficiency.
• To design a unified framework combining CTSVD-QR and $L_{2,1}$-norm minimization for efficient and accurate tensor completion.
• To validate the proposed method on synthetic and real-world data, including video and color image recovery, under varying missing ratios and tensor sizes.

Proposed method

• Proposes CTSVD-QR, a method to compute the largest $r$ singular values (tubes) and their associated singular vectors (of tubes) iteratively using tensor QR decomposition, replacing computationally heavy t-SVD.
• Introduces a new tensor $L_{2,1}$-norm defined as the sum of the $L_2$-norms of each tube, which serves as a convex surrogate for tubal-rank and is easier to optimize than the tensor nuclear norm.
• Establishes that the tensor nuclear norm is a lower bound of the tensor $L_{2,1}$-norm, justifying its use as a valid replacement in low-rank tensor approximation.
• Develops the TLNM-TQR algorithm by combining CTSVD-QR with $L_{2,1}$-norm minimization under an ADMM optimization framework to solve the tensor completion problem.
• Employs ADMM to solve the resulting subproblems efficiently, enabling fast convergence and stable performance across diverse tensor structures.
• Applies the method to tensor completion via the constrained optimization model: $\min_{\mathcal{X}} \|\mathcal{X}\|_{2,1} \text{ s.t. } \mathcal{X}_{i,j,k} = \mathcal{M}_{i,j,k}, (i,j,k) \in \Omega$, where $\mathcal{M}$ is the observed incomplete tensor.

Experimental results

Research questions

• RQ1 Can a QR-based approximation of t-SVD be developed that is significantly faster than traditional t-SVD while preserving accuracy in tensor completion?
• RQ2 Does the tensor $L_{2,1}$-norm provide a more efficient and effective convex surrogate for tubal-rank than the tensor nuclear norm?
• RQ3 Can the combination of CTSVD-QR and $L_{2,1}$-norm minimization lead to a faster and more accurate tensor completion algorithm compared to state-of-the-art methods?
• RQ4 How does the proposed method perform across varying missing ratios and tensor dimensions in real-world applications like video and image recovery?
• RQ5 Is the proposed method robust and scalable when applied to high-dimensional tensor data such as video sequences and color images?

Key findings

• TLNM-TQR achieves a running time of 2.4336 seconds on average for color image recovery, significantly faster than ADMM-t-SVD (22.8229 seconds) and HoMP (24.1802 seconds).
• On video datasets with 50% missing data, TLNM-TQR maintains an RMSE of 11.0136, nearly matching ADMM-t-SVD (12.2161) and outperforming HoMP and ELRAP4TS.
• For video recovery with increasing frontal slice counts (2 to 40), TLNM-TQR shows a slower rate of increase in running time compared to other methods, indicating superior scalability.
• In color image recovery with added Gaussian and blur noise, TLNM-TQR achieves the lowest RMSE (22.1667) on average, outperforming ADMM-t-SVD (19.7492) and HoMP (17.1329) in accuracy.
• The method demonstrates consistent performance across different missing ratios (30% to 90%), maintaining high accuracy and low running time, with minimal variance in results over 50 repeated trials.
• Numerical experiments confirm that CTSVD-QR reduces iteration time significantly compared to t-SVD, and the $L_{2,1}$-norm enables faster optimization, validating the core design choices of the framework.