[Paper Review] Non-convex Penalty for Tensor Completion and Robust PCA.

Tao Li, Jinwen Ma|arXiv (Cornell University)|Apr 23, 2019

Sparse and Compressive Sensing Techniques35 references4 citations

TL;DR

This paper proposes a non-convex tensor rank surrogate and sparsity measure that reduce the bias inherent in the ℓ₁-norm by introducing concavity. By applying these penalties to tensor completion and robust PCA, the method achieves superior performance in image inpainting and denoising, with convergence ensured via majorization minimization and ADMM-based optimization.

ABSTRACT

In this paper, we propose a novel non-convex tensor rank surrogate function and a novel non-convex sparsity measure for tensor. The basic idea is to sidestep the bias of $\ell_1-$norm by introducing concavity. Furthermore, we employ the proposed non-convex penalties in tensor recovery problems such as tensor completion and tensor robust principal component analysis, which has various real applications such as image inpainting and denoising. Due to the concavity, the models are difficult to solve. To tackle this problem, we devise majorization minimization algorithms, which optimize upper bounds of original functions in each iteration, and every sub-problem is solved by alternating direction multiplier method. Finally, experimental results on natural images and hyperspectral images demonstrate the effectiveness and efficiency of the proposed methods.

Motivation & Objective

• To address the bias introduced by the ℓ₁-norm in low-rank and sparse tensor recovery by proposing a non-convex alternative.
• To develop a novel non-convex tensor rank surrogate function that better approximates the true tensor rank.
• To design a non-convex sparsity measure that enhances low-rank and sparse component separation in tensors.
• To apply these penalties to real-world tensor recovery problems such as image inpainting and denoising.
• To devise an efficient optimization framework capable of handling the non-convexity of the proposed models.

Proposed method

• Proposes a non-convex tensor rank surrogate function that introduces concavity to reduce bias compared to the ℓ₁-norm.
• Introduces a non-convex sparsity measure that enhances the representation of sparse components in tensors.
• Employs majorization minimization to iteratively optimize upper bounds of the non-convex objective functions.
• Solves each sub-problem in the majorization minimization framework using the alternating direction method of multipliers (ADMM).
• Applies the proposed framework to tensor completion and robust PCA tasks with real-world image data.
• Uses iterative refinement to converge to a stationary solution despite the non-convexity of the objective.

Experimental results

Research questions

• RQ1 Can a non-convex tensor rank surrogate reduce the bias of the ℓ₁-norm in low-rank tensor approximation?
• RQ2 How does a non-convex sparsity measure improve the recovery of sparse components in tensor decomposition?
• RQ3 Can the proposed non-convex penalties outperform convex alternatives in tensor completion and robust PCA?
• RQ4 How effective is the majorization minimization approach with ADMM for solving non-convex tensor recovery problems?
• RQ5 What is the empirical performance of the method on natural and hyperspectral image datasets?

Key findings

• The proposed non-convex tensor rank surrogate reduces bias compared to the ℓ₁-norm, leading to better low-rank approximation.
• The non-convex sparsity measure enhances the separation of sparse components in tensor decomposition.
• The method achieves superior image inpainting and denoising results on natural and hyperspectral image datasets.
• The majorization minimization algorithm with ADMM convergence is effective despite the non-convexity of the objective.
• Empirical results demonstrate both effectiveness and efficiency of the proposed approach on real-world image recovery tasks.
• The method outperforms traditional convex approaches in terms of reconstruction accuracy and robustness.