[Paper Review] Intersecting Faces: Non-negative Matrix Factorization With New Guarantees

Rong Ge, James Zou|arXiv (Cornell University)|Jul 8, 2015

Medical Image Segmentation Techniques23 references11 citations

TL;DR

This paper introduces subset-separable non-negative matrix factorization (NMF), a generalization of separable NMF that allows for unique and minimum-volume factorizations under milder conditions. The authors propose the Face-Intersect algorithm, which provably recovers the true factorization under subset-separability and is robust to small adversarial noise, outperforming state-of-the-art methods in simulations on both synthetic and real biological data.

ABSTRACT

Non-negative matrix factorization (NMF) is a natural model of admixture and is widely used in science and engineering. A plethora of algorithms have been developed to tackle NMF, but due to the non-convex nature of the problem, there is little guarantee on how well these methods work. Recently a surge of research have focused on a very restricted class of NMFs, called separable NMF, where provably correct algorithms have been developed. In this paper, we propose the notion of subset-separable NMF, which substantially generalizes the property of separability. We show that subset-separability is a natural necessary condition for the factorization to be unique or to have minimum volume. We developed the Face-Intersect algorithm which provably and efficiently solves subset-separable NMF under natural conditions, and we prove that our algorithm is robust to small noise. We explored the performance of Face-Intersect on simulations and discuss settings where it empirically outperformed the state-of-art methods. Our work is a step towards finding provably correct algorithms that solve large classes of NMF problems.

Motivation & Objective

• To generalize the concept of separable NMF to a broader class of problems by introducing subset-separability as a more natural and general condition.
• To develop a provably correct and robust algorithm for solving subset-separable NMF under natural geometric conditions.
• To demonstrate that subset-separability is a necessary condition for unique or minimum-volume NMF factorizations.
• To empirically evaluate the Face-Intersect algorithm against state-of-the-art NMF solvers in terms of accuracy and robustness to noise.
• To establish a theoretical foundation that enables provably correct algorithms for a significantly larger class of NMF problems beyond separable NMF.

Proposed method

• Introduce the notion of subset-separability, where each source (row of W) is represented by a unique combination of features (facets), rather than individual features as in separable NMF.
• Develop the Face-Intersect algorithm, which identifies latent components by intersecting faces (affine hulls) of data points corresponding to subsets of features.
• Use geometric properties of convex hulls and facets in non-negative data to locate the true components of W via intersection of supporting hyperplanes.
• Prove that under proper facet filling conditions, Face-Intersect recovers the true W and A matrices exactly in the noiseless case.
• Establish robustness to small adversarial noise by showing that the algorithm’s output remains close to the true factorization under bounded perturbations.
• Validate the method through simulations on synthetic data and real-world DNA methylation data, comparing against Anchor-Words and Projected Gradient methods.

Experimental results

Research questions

• RQ1 Can subset-separability serve as a generalization of separability that enables provable recovery of NMF factorizations?
• RQ2 Under what geometric conditions can the Face-Intersect algorithm provably recover the true factorization in subset-separable NMF?
• RQ3 How does the Face-Intersect algorithm perform in terms of accuracy and robustness compared to existing state-of-the-art NMF solvers?
• RQ4 Is subset-separability a necessary condition for unique or minimum-volume NMF factorizations?
• RQ5 Can the Face-Intersect algorithm outperform existing methods in practical settings with low noise levels?

Key findings

• Face-Intersect significantly outperformed Anchor-Words and Projected Gradient in reconstructing the W matrix (latent components) in synthetic simulations with low noise.
• In reconstructing the observed data matrix M̃, Face-Intersect slightly outperformed Anchor-Words at noise levels below 12.5%, with both methods substantially better than Projected Gradient.
• On real DNA methylation data with 5 cell types and 600 samples, Face-Intersect was significantly more accurate than Anchor-Words in recovering the true W matrix.
• Face-Intersect showed statistically superior performance over Anchor-Words in reconstructing M and M̃ when noise was below 8%, though the difference was small in magnitude.
• The algorithm’s performance degraded with increasing noise, and all methods converged at around 12.5% noise, indicating a threshold beyond which robustness is limited.
• Subset-separability was shown to be a necessary condition for unique or minimum-volume NMF factorizations, validating its theoretical significance.