[Paper Review] Nearly Optimal Robust Subspace Tracking

Praneeth Narayanamurthy, Namrata Vaswani|arXiv (Cornell University)|Dec 17, 2017

Sparse and Compressive Sensing Techniques15 citations

TL;DR

This paper proposes ReProCS-NORST, a recursive projected compressive sensing algorithm for robust subspace tracking (RST) that achieves nearly optimal tracking delay of O(r log n log(1/ϵ)) and optimal memory complexity. It provides the first provable guarantees for RST under weakened RPCA assumptions, with improved outlier tolerance due to two key assumptions: slow subspace change and lower-bounded outlier magnitudes.

ABSTRACT

In this work, we study the robust subspace tracking (RST) problem and obtain one of the first two provable guarantees for it. The goal of RST is to track sequentially arriving data vectors that lie in a slowly changing low-dimensional subspace, while being robust to corruption by additive sparse outliers. It can also be interpreted as a dynamic (time-varying) extension of robust PCA (RPCA), with the minor difference that RST also requires a short tracking delay. We develop a recursive projected compressive sensing algorithm that we call Nearly Optimal RST via ReProCS (ReProCS-NORST) because its tracking delay is nearly optimal. We prove that NORST solves both the RST and the dynamic RPCA problems under weakened standard RPCA assumptions, two simple extra assumptions (slow subspace change and most outlier magnitudes lower bounded), and a few minor assumptions. Our guarantee shows that NORST enjoys a near optimal tracking delay of $O(r \log n \log(1/ε))$. Its required delay between subspace change times is the same, and its memory complexity is $n$ times this value. Thus both these are also nearly optimal. Here $n$ is the ambient space dimension, $r$ is the subspaces' dimension, and $ε$ is the tracking accuracy. NORST also has the best outlier tolerance compared with all previous RPCA or RST methods, both theoretically and empirically (including for real videos), without requiring any model on how the outlier support is generated. This is possible because of the extra assumptions it uses.

Motivation & Objective

• To address the challenge of tracking a slowly changing low-dimensional subspace in the presence of sparse, arbitrary outliers with minimal delay.
• To develop a recursive algorithm that achieves provably near-optimal tracking delay and memory complexity for robust subspace tracking.
• To extend robust PCA (RPCA) to dynamic, time-varying settings with real-time estimation requirements.
• To improve outlier tolerance over prior RPCA and RST methods without requiring assumptions on outlier support generation.

Proposed method

• Proposes ReProCS-NORST, a recursive projected compressive sensing algorithm designed for real-time robust subspace tracking.
• Uses a recursive update strategy that leverages past estimates and compressive sensing principles to recover low-rank and sparse components.
• Employs a subspace error metric SE(P1, P2) = ∥(I − P1P1′)P2∥ to measure distance between subspaces.
• Introduces a delay model where tracking delay is O(r log n log(1/ϵ)), nearly optimal for the problem.
• Relies on two key assumptions: slow subspace change and lower-bounded outlier magnitudes to ensure recovery.
• Uses a basis matrix representation and orthogonal projection to maintain subspace estimates over time.

Experimental results

Research questions

• RQ1 Can robust subspace tracking be achieved with provably near-optimal delay and memory complexity?
• RQ2 How does the performance of ReProCS-NORST compare to existing RPCA and RST methods in terms of outlier tolerance and recovery accuracy?
• RQ3 What assumptions are necessary to achieve provable recovery in dynamic, time-varying subspace tracking?
• RQ4 Can the algorithm handle real-world video data with complex foreground dynamics and color similarity to backgrounds?
• RQ5 How does the lower bound on outlier magnitude affect recovery performance?

Key findings

• ReProCS-NORST achieves a tracking delay of O(r log n log(1/ϵ)), which is nearly optimal and matches the required delay between subspace changes.
• The algorithm achieves optimal memory complexity of n times the tracking delay, also nearly optimal.
• It provides the first provable guarantees for robust subspace tracking under weakened standard RPCA assumptions and two additional assumptions: slow subspace change and lower-bounded outlier magnitudes.
• NORST demonstrates significantly higher outlier tolerance than AltProj and other RPCA methods, especially when outlier magnitudes are bounded away from zero.
• In real video experiments on the MR and Lobby datasets, NORST achieved 43 fps and 16 fps processing rates, respectively, with superior foreground-background separation.
• Numerical results show that NORST fails only when outlier magnitudes are too small (e.g., xmin = 5), confirming the necessity of the lower-bound assumption.