[Paper Review] Ellipsoidal Rounding for Nonnegative Matrix Factorization Under Noisy Separability

Tomohiko Mizutani|arXiv (Cornell University)|Sep 23, 2013

Sparse and Compressive Sensing Techniques37 references18 citations

TL;DR

This paper proposes an ellipsoidal rounding algorithm for nonnegative matrix factorization (NMF) under noisy separability, leveraging the minimum-volume enclosing ellipsoid (MVEE) to identify basis vectors as points on the ellipsoid's boundary. The method achieves correctness on noiseless data and robustness to noise, outperforming SPA and XRAY in recovery rates and document clustering accuracy on real-world corpora.

ABSTRACT

We present a numerical algorithm for nonnegative matrix factorization (NMF) problems under noisy separability. An NMF problem under separability can be stated as one of finding all vertices of the convex hull of data points. The research interest of this paper is to find the vectors as close to the vertices as possible in a situation in which noise is added to the data points. Our algorithm is designed to capture the shape of the convex hull of data points by using its enclosing ellipsoid. We show that the algorithm has correctness and robustness properties from theoretical and practical perspectives; correctness here means that if the data points do not contain any noise, the algorithm can find the vertices of their convex hull; robustness means that if the data points contain noise, the algorithm can find the near-vertices. Finally, we apply the algorithm to document clustering, and report the experimental results.

Motivation & Objective

• Address the challenge of identifying basis vectors in nonnegative matrix factorization (NMF) when data points are corrupted by noise.
• Develop an algorithm that maintains correctness on noiseless separable data and robustness on noisy data.
• Improve upon existing NMF algorithms like SPA and XRAY in terms of recovery rate and clustering performance.
• Enable accurate document clustering by extracting recognizable topics from low-rank approximations of text data.
• Provide a geometric approach to NMF using the convex hull and enclosing ellipsoid to locate vertices corresponding to basis vectors.

Proposed method

• Compute the minimum-volume enclosing ellipsoid (MVEE) for the data points to capture the shape of their convex hull.
• Identify basis vectors as data points lying on the boundary of the MVEE, leveraging the geometric property that vertices of a simplex touch the MVEE.
• Use singular value decomposition (SVD) to reduce dimensionality before MVEE computation, preserving full-dimensionality of the convex hull.
• Apply a two-step process: SVD for dimensionality reduction, followed by MVEE computation on the reduced data to locate near-basis vectors.
• Formally establish correctness and robustness via theoretical analysis, showing that the algorithm recovers exact vertices under noiseless conditions and near-vertices under bounded noise.
• Implement the algorithm as Algorithm 2, using MATLAB’s svds for sparse SVD and MVEE computation via convex optimization.

Experimental results

Research questions

• RQ1 Can an NMF algorithm achieve correctness on noiseless separable data by identifying vertices of the convex hull via ellipsoidal rounding?
• RQ2 Does the proposed method maintain robustness when noise is present in the data, by identifying near-basis vectors?
• RQ3 How does the recovery rate of the proposed algorithm compare to SPA and XRAY under varying noise levels?
• RQ4 Can the algorithm extract more interpretable and accurate topics in document clustering compared to existing methods?
• RQ5 Is it possible to avoid SVD computation by using an alternative convex set for MVEE while preserving performance?

Key findings

• The proposed algorithm achieves correctness: on noiseless data, it recovers the exact vertices of the convex hull, corresponding to the true basis vectors.
• The algorithm demonstrates robustness: under bounded noise, it identifies near-basis vectors, with recovery rates improving over SPA and XRAY.
• On the Reuters-21578 dataset, ER-SPA (the implementation of the proposed algorithm) achieved higher Adjusted Rand Index (AC) and Normalized Mutual Information (NMI) than SPA.
• On the 20 Newsgroups dataset, ER-SPA outperformed SPA in both AC and NMI, indicating superior clustering performance.
• In topic extraction using the BBC corpus, ER-SPA produced more interpretable topics—e.g., correctly identifying 'politics' topics where SPA failed—due to better anchor word alignment.
• The algorithm efficiently handles large sparse document-word matrices, as demonstrated by successful execution on the 20 Newsgroups corpus (18,846×26,213 matrix) using sparse SVD.