[Paper Review] A Unified Convex Surrogate for the Schatten- p Norm

Chen Xu, Zhouchen Lin+1 more|arXiv (Cornell University)|Feb 12, 2017

Sparse and Compressive Sensing TechniquesEngineering28 citations

TL;DR

This paper proposes a unified convex surrogate for the Schatten-p norm that enables convex and smooth optimization for any p > 0, overcoming the non-convexity and non-smoothness of the original Schatten-p norm when 0 < p < 1. By establishing equivalence between the Schatten-p norm and the norms of its factor matrices, the method enables efficient matrix completion via accelerated proximal alternating linearized minimization with proven convergence and superior empirical performance.

ABSTRACT

The Schatten- p norm (0 0 satisfying 1/ p = 1/ p 1 + 1/ p 2 , there is an equivalence between the Schatten- p norm of one matrix and the Schatten- p 1 and the Schatten- p 2 norms of its two factor matrices. We further extend the equivalence to multiple factor matrices and show that all the factor norms can be convex and smooth for any p > 0. In contrast, the original Schatten- p norm for 0 < p < 1 is non-convex and non-smooth. As an example we conduct experiments on matrix completion. To utilize the convexity of the factor matrix norms, we adopt the accelerated proximal alternating linearized minimization algorithm and establish its sequence convergence. Experiments on both synthetic and real datasets exhibit its superior performance over the state-of-the-art methods. Its speed is also highly competitive.

Motivation & Objective

• To address the non-convexity and non-smoothness of the Schatten-p norm for 0 < p < 1, which hinders optimization in low-rank matrix recovery.
• To establish a convex and smooth surrogate for the Schatten-p norm through factor matrix decomposition.
• To enable efficient and convergent optimization for matrix completion by leveraging convexity in factor matrix norms.
• To demonstrate the superiority of the proposed method over state-of-the-art approaches in both accuracy and computational speed.

Proposed method

• Proposes a unified convex surrogate for the Schatten-p norm by relating it to the Schatten-p1 and Schatten-p2 norms of its factor matrices under the condition 1/p = 1/p1 + 1/p2.
• Extends the equivalence to multiple factor matrices, ensuring all factor norms remain convex and smooth for any p > 0.
• Employs the accelerated proximal alternating linearized minimization (APALM) algorithm to solve the optimization problem with established sequence convergence.
• Utilizes the equivalence between the Schatten-p norm and factor matrix norms to reformulate the original non-convex problem into a convex, smooth optimization framework.
• Applies the method to matrix completion by minimizing a convex surrogate objective over factorized matrices.

Experimental results

Research questions

• RQ1 Can a convex surrogate be constructed for the Schatten-p norm when 0 < p < 1, despite its inherent non-convexity?
• RQ2 Is there a mathematical equivalence between the Schatten-p norm of a matrix and the Schatten-p1 and p2 norms of its factor matrices under a specific parameter condition?
• RQ3 Can the convexity and smoothness of factor matrix norms be preserved across multiple factorizations for any p > 0?
• RQ4 Does the proposed convex surrogate enable faster and more accurate matrix completion compared to existing methods?

Key findings

• The proposed convex surrogate enables smooth and convex optimization for any p > 0, resolving the non-convexity of the original Schatten-p norm for 0 < p < 1.
• An equivalence is established between the Schatten-p norm of a matrix and the Schatten-p1 and p2 norms of its factor matrices when 1/p = 1/p1 + 1/p2.
• The method achieves state-of-the-art performance in matrix completion on both synthetic and real-world datasets.
• The accelerated proximal alternating linearized minimization algorithm ensures sequence convergence for the proposed formulation.
• The method demonstrates highly competitive computational speed alongside superior accuracy compared to existing approaches.