[Paper Review] Robust Near-Separable Nonnegative Matrix Factorization Using Linear Optimization

Nicolas Gillis, Robert Luce|arXiv (Cornell University)|Feb 18, 2013

Face and Expression Recognition18 references80 citations

TL;DR

This paper proposes a robust linear programming model for near-separable nonnegative matrix factorization (NMF) that eliminates the need for matrix normalization and automatically detects the factorization rank. By reformulating the optimization problem with a novel sparsity-promoting objective, the method achieves superior noise tolerance and outlier resilience compared to Hottopixx, outperforming it on synthetic datasets while competing favorably with state-of-the-art methods under the separability assumption.

ABSTRACT

Nonnegative matrix factorization (NMF) has been shown recently to be tractable under the separability assumption, under which all the columns of the input data matrix belong to the convex cone generated by only a few of these columns. Bittorf, Recht, Ré and Tropp (`Factoring nonnegative matrices with linear programs', NIPS 2012) proposed a linear programming (LP) model, referred to as Hottopixx, which is robust under any small perturbation of the input matrix. However, Hottopixx has two important drawbacks: (i) the input matrix has to be normalized, and (ii) the factorization rank has to be known in advance. In this paper, we generalize Hottopixx in order to resolve these two drawbacks, that is, we propose a new LP model which does not require normalization and detects the factorization rank automatically. Moreover, the new LP model is more flexible, significantly more tolerant to noise, and can easily be adapted to handle outliers and other noise models. Finally, we show on several synthetic datasets that it outperforms Hottopixx while competing favorably with two state-of-the-art methods.

Motivation & Objective

• To address two key limitations of Hottopixx: the requirement for matrix normalization and the need to know the factorization rank in advance.
• To develop a more robust and flexible linear programming model for near-separable NMF under noisy conditions.
• To enable automatic detection of the factorization rank without prior knowledge.
• To enhance tolerance to noise and outliers by adapting the optimization framework to various noise models.
• To demonstrate superior performance on synthetic datasets compared to Hottopixx and two state-of-the-art NMF methods.

Proposed method

• Proposes a new linear programming model that replaces the normalization step in Hottopixx with a sparsity-promoting objective function.
• Introduces a modified optimization problem where the factorization rank is inferred automatically through a rank detection mechanism embedded in the objective.
• Uses a robustness condition based on conicality and minimum separation (κ, η, δ) to ensure stable recovery of the true basis matrix W.
• Employs an ℓ₁-regularized reconstruction error in the objective to promote sparsity and improve noise resilience.
• Adapts the model to handle outliers by distinguishing between pure components (W), outliers (T), and noisy columns (WH′), using separate bounds on their contributions.
• Applies theoretical guarantees via Theorem 4, showing that under bounded noise and conicality, the optimal solution X preserves high diagonal values for true components and low values for others.

Experimental results

Research questions

• RQ1 Can a linear programming model for near-separable NMF be designed without requiring input matrix normalization?
• RQ2 Can the factorization rank be detected automatically within the optimization framework rather than being pre-specified?
• RQ3 How does the proposed method perform in terms of noise tolerance and outlier handling compared to Hottopixx?
• RQ4 Can the new model maintain high accuracy under varying noise levels and different noise models?
• RQ5 Does the proposed method outperform existing state-of-the-art NMF algorithms on synthetic near-separable datasets?

Key findings

• The proposed method automatically detects the factorization rank without requiring prior knowledge, eliminating a key limitation of Hottopixx.
• The method achieves significantly higher noise tolerance than Hottopixx, as demonstrated by theoretical bounds involving κ, η, δ, and ϵ.
• The model successfully handles outliers by assigning minimal weight to them, as shown by the theoretical bound X(j,j) ≤ 8ϵ / (κ(1−β)(1−ϵ)) for non-basis columns.
• On synthetic datasets, the method outperforms Hottopixx in terms of reconstruction error and basis recovery accuracy.
• The method maintains robustness even when noise is bounded and the data is not normalized, as proven under the robust conicality condition.
• Theoretical analysis confirms that the optimal solution X assigns high diagonal values to true basis columns and low values to outliers and noisy columns, ensuring correct identification.