[Paper Review] Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise

Chenlu Qiu, Namrata Vaswani+2 more|arXiv (Cornell University)|Nov 15, 2012

Sparse and Compressive Sensing Techniques4 citations

TL;DR

This paper proposes a recursive robust PCA method, ReProCS-NS, for real-time recovery of sparse signals (e.g., moving objects in video) corrupted by large but structured noise (e.g., slowly changing background). By assuming a subspace change model and denseness of unestimated subspace components, it achieves exact support recovery of sparse signals and bounded reconstruction errors with high probability, even under challenging conditions where signal magnitudes are small compared to noise.

ABSTRACT

This work studies the recursive robust principal components' analysis(PCA) problem. Here, "robust" refers to robustness to both independent and correlated sparse outliers. If the outlier is the signal-of-interest, this problem can be interpreted as one of recursively recovering a time sequence of sparse vectors, St, in the presence of large but structured noise, Lt. The structure that we assume on Lt is that Lt is dense and lies in a low dimensional subspace that is either fixed or changes "slowly enough". A key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background (Lt) from moving foreground objects (St) on-the-fly. To solve the above problem, we introduce a novel solution called Recursive Projected CS (ReProCS). Under mild assumptions, we show that, with high probability (w.h.p.), ReProCS can exactly recover the support set of St at all times; and the reconstruction errors of both St and Lt are upper bounded by a time-invariant and small value at all times.

Motivation & Objective

• Address the challenge of recursive robust PCA in time-series data where the background (structured noise) changes slowly but is dense and low-rank.
• Recover sparse foreground signals (e.g., moving objects in video) in real time despite large-magnitude, structured background corruption.
• Enable online processing by leveraging a subspace change model and recursive estimation, avoiding batch processing.
• Ensure exact support recovery of sparse signals and bounded reconstruction errors under mild statistical assumptions.
• Provide theoretical guarantees for performance in the presence of slowly evolving low-rank noise and sparse outliers.

Proposed method

• Adapt the Recursive Projected Compressive Sensing (ReProCS) framework with a novel subspace change model assumption.
• Use recursive estimation of the low-rank background subspace $ L_t $, updating it incrementally as new data arrives.
• Apply a modified sparse recovery step that leverages the structured nature of $ L_t $, assuming it lies in a slowly changing low-dimensional subspace.
• Introduce a denseness assumption on the unestimated part of the subspace of $ L_t $, ensuring sufficient energy in the orthogonal complement for stable recovery.
• Employ concentration inequalities and matrix perturbation theory to bound estimation errors in the subspace and signal recovery.
• Use a time-invariant error bound framework to ensure stability and convergence of both $ S_t $ and $ L_t $ reconstruction.

Experimental results

Research questions

• RQ1 Can recursive robust PCA be achieved with exact support recovery of sparse signals under large but structured noise?
• RQ2 Under what conditions does the proposed method maintain bounded reconstruction errors for both $ S_t $ and $ L_t $ over time?
• RQ3 How does the denseness of the unestimated part of the $ L_t $ subspace affect recovery performance?
• RQ4 Can the method guarantee high-probability recovery when the subspace of $ L_t $ changes slowly over time?
• RQ5 What is the impact of support changes in $ S_t $ on the stability and accuracy of recursive recovery?

Key findings

• With high probability, the proposed method achieves exact support recovery of the sparse signal $ S_t $ at all times, provided the unestimated subspace components are sufficiently dense.
• The reconstruction errors for both $ S_t $ and $ L_t $ are upper bounded by a time-invariant and small value, ensuring long-term stability.
• The method maintains bounded error even when the signal-of-interest $ S_t $ has small magnitude relative to the background $ L_t $, overcoming limitations of prior outlier-detection-based methods.
• Simulation results confirm that the denseness assumption holds as long as there is some support change in $ S_t $ every few frames, validating the practical feasibility of the approach.
• Theoretical analysis shows that the error bounds depend on the subspace change rate, signal-to-noise ratio, and the spectral properties of the background subspace.
• The method outperforms prior recursive robust PCA techniques in scenarios with small-amplitude sparse signals and large structured noise, especially in online video surveillance applications.