[Paper Review] Multi-View Clustering via Semi-non-negative Tensor Factorization

Jing Li, Quanxue Gao+3 more|arXiv (Cornell University)|Mar 29, 2023

Tensor decomposition and applications4 citations

TL;DR

This paper proposes a novel multi-view clustering method based on semi-non-negative tensor factorization (Semi-NTF) that models multi-view data as a 3rd-order tensor of anchor graphs, leveraging tensor Schatten p-norm regularization to capture both within-view structure and between-view complementary information. The method achieves state-of-the-art clustering performance with guaranteed convergence to a KKT stationary point.

ABSTRACT

Multi-view clustering (MVC) based on non-negative matrix factorization (NMF) and its variants have received a huge amount of attention in recent years due to their advantages in clustering interpretability. However, existing NMF-based multi-view clustering methods perform NMF on each view data respectively and ignore the impact of between-view. Thus, they can't well exploit the within-view spatial structure and between-view complementary information. To resolve this issue, we present semi-non-negative tensor factorization (Semi-NTF) and develop a novel multi-view clustering based on Semi-NTF with one-side orthogonal constraint. Our model directly performs Semi-NTF on the 3rd-order tensor which is composed of anchor graphs of views. Thus, our model directly considers the between-view relationship. Moreover, we use the tensor Schatten p-norm regularization as a rank approximation of the 3rd-order tensor which characterizes the cluster structure of multi-view data and exploits the between-view complementary information. In addition, we provide an optimization algorithm for the proposed method and prove mathematically that the algorithm always converges to the stationary KKT point. Extensive experiments on various benchmark datasets indicate that our proposed method is able to achieve satisfactory clustering performance.

Motivation & Objective

• To address the limitation of existing NMF-based multi-view clustering methods that treat views independently and ignore inter-view relationships.
• To preserve the spatial structure of multi-view data by modeling it as a 3rd-order tensor instead of reducing it to matrices.
• To exploit complementary information across views by using tensor Schatten p-norm regularization on the clustering indicator matrix.
• To develop an optimization algorithm with theoretical convergence guarantees to the KKT stationary point.

Proposed method

• Constructs anchor graphs for each view using a direct alternate sampling (DAS) strategy to reduce computational complexity.
• Forms a 3rd-order tensor from the anchor graphs of all views to preserve multi-view spatial structure.
• Applies semi-non-negative tensor factorization (Semi-NTF) with one-side orthogonal constraint to enforce solution uniqueness and improve interpretability.
• Introduces tensor Schatten p-norm regularization to approximate tensor rank and capture cluster structure across views.
• Develops an alternating direction method of multipliers (ADMM)-based optimization algorithm with convergence proof to the KKT point.
• Uses the clustering indicator matrix as input to the tensor factorization to directly model cluster assignments.

Experimental results

Research questions

• RQ1 Can a tensor-based formulation better capture the interplay between multiple views in multi-view clustering compared to matrix-based approaches?
• RQ2 How can the tensor Schatten p-norm effectively model the cluster structure and exploit complementary information across views?
• RQ3 Can a semi-non-negative tensor factorization model with one-side orthogonal constraint improve clustering interpretability and solution uniqueness?
• RQ4 Does the proposed optimization algorithm converge to a stationary point, and is this convergence guaranteed under the proposed formulation?
• RQ5 Can the use of anchor graphs instead of full affinity matrices significantly reduce computational cost without sacrificing clustering performance?

Key findings

• The proposed method achieves state-of-the-art clustering performance on multiple benchmark datasets, outperforming existing NMF-based multi-view clustering methods.
• The use of anchor graphs reduces the computational complexity significantly compared to full affinity matrix methods, improving efficiency.
• Tensor Schatten p-norm regularization effectively captures the low-rank cluster structure and enhances the exploitation of complementary information across views.
• The optimization algorithm converges to a KKT stationary point, as mathematically proven, ensuring algorithmic stability.
• The proposed method demonstrates superior robustness and clustering accuracy, especially in high-dimensional and multi-view settings.