[Paper Review] Manifold Denoising by Nonlinear Robust Principal Component Analysis

He Lyu, Ningyu Sha+4 more|arXiv (Cornell University)|Nov 10, 2019

Face and Expression Recognition23 references9 citations

TL;DR

This paper proposes Nonlinear Robust Principal Component Analysis (NRPCA), a novel method that extends Robust PCA to nonlinear manifolds by combining local tangent space analysis with curvature-informed optimization. It simultaneously denoises sparse outliers and recovers low-dimensional manifold structure, achieving state-of-the-art performance on synthetic and real datasets including MNIST digits and single-cell RNA-seq data, with theoretical error bounds under manifold incoherence conditions.

ABSTRACT

This paper extends robust principal component analysis (RPCA) to nonlinear manifolds. Suppose that the observed data matrix is the sum of a sparse component and a component drawn from some low dimensional manifold. Is it possible to separate them by using similar ideas as RPCA? Is there any benefit in treating the manifold as a whole as opposed to treating each local region independently? We answer these two questions affirmatively by proposing and analyzing an optimization framework that separates the sparse component from the manifold under noisy data. Theoretical error bounds are provided when the tangent spaces of the manifold satisfy certain incoherence conditions. We also provide a near optimal choice of the tuning parameters for the proposed optimization formulation with the help of a new curvature estimation method. The efficacy of our method is demonstrated on both synthetic and real datasets.

Motivation & Objective

• To address the limitation of standard Robust PCA, which assumes linear subspaces, by extending it to nonlinear manifolds.
• To correct sparse outliers in high-dimensional data while preserving the underlying low-dimensional manifold structure.
• To provide a unified framework that treats the manifold as a whole rather than local patches independently.
• To establish theoretical error bounds for the denoising performance under intrinsic manifold incoherence conditions.
• To enable practical outlier detection and correction in real-world data such as gene expression and image data.

Proposed method

• The method uses k-nearest neighbors (k-NN) to define overlapping local patches around each data point, forming local data matrices.
• For each patch, it solves a regularized optimization problem that separates low-rank manifold components from sparse noise using nuclear norm and L1 regularization.
• It incorporates manifold curvature estimates into the optimization weights via a new curvature estimation method to improve robustness.
• The algorithm employs an iterative patch-reassignment scheme to refine neighborhood assignments and enhance denoising accuracy.
• The tuning parameters, including the weight λi and β, are automatically selected based on curvature and data dimensionality.
• The overall formulation is solved via alternating minimization, with constraints enforcing consistency between global and local sparse components.

Experimental results

Research questions

• RQ1 Can Robust PCA be generalized to nonlinear manifolds while preserving theoretical guarantees?
• RQ2 Does treating the manifold as a whole, rather than local patches, improve denoising performance and stability?
• RQ3 Can curvature information be effectively integrated into the optimization framework to enhance robustness to sparse noise?
• RQ4 What theoretical error bounds can be established for the denoising error under intrinsic manifold incoherence conditions?
• RQ5 Is the proposed method effective in recovering true manifold structure and detecting sparse outliers in real-world nonlinear datasets?

Key findings

• NRPCA successfully recovers the underlying manifold structure in MNIST digit data, improving Isomap and Laplacian eigenmap embeddings by enabling clearer trajectory patterns between digits 4 and 9.
• On single-cell RNA-seq data, NRPCA denoising enabled Locally Linear Embedding (LLE) to reveal a biologically meaningful branching trajectory around day 9 of embryoid body differentiation.
• The method achieved near-optimal tuning parameter selection through a curvature-based automatic procedure, reducing reliance on manual hyperparameter tuning.
• Two iterations of patch-reassignment were sufficient to achieve stable and high-quality denoising results, minimizing computational cost without sacrificing accuracy.
• Theoretical error bounds were established under manifold incoherence conditions, validating the method’s robustness to sparse noise and manifold curvature.
• The support of the sparse component S^i identified biologically relevant genes in scRNA-seq data, enabling detection of differentially expressed genes linked to developmental transitions.