[论文解读] Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees

Jinchun Zhan, Brian Lois+1 more|arXiv (Cornell University)|Jan 29, 2016

Sparse and Compressive Sensing Techniques被引用 29

一句话总结

本文提出了一种基于递归投影压缩感知（ReProCS）框架的新型在线鲁棒主成分分析（RPCA）算法，能够在实时流数据中准确分离低秩（背景）与稀疏（前景）分量。该工作在温和假设下首次提供了在线RPCA的完整性能保证，包括准确的初始子空间知识、缓慢的子空间变化以及特征值聚集，显著优于以往部分结果。

ABSTRACT

In this work, we study the online robust principal components' analysis (RPCA) problem. In recent work, RPCA has been defined as a problem of separating a low-rank matrix (true data), $L$, and a sparse matrix (outliers), $S$, from their sum, $M:=L + S$. A more general version of this problem is to recover $L$ and $S$ from $M:=L + S + W$ where $W$ is the matrix of unstructured small noise/corruptions. An important application where this problem occurs is in video analytics in trying to separate sparse foregrounds (e.g., moving objects) from slowly changing backgrounds. While there has been a large amount of recent work on solutions and guarantees for the batch RPCA problem, the online problem is largely open."Online" RPCA is the problem of doing the above on-the-fly with the extra assumptions that the initial subspace is accurately known and that the subspace from which $l_t$ is generated changes slowly over time. We develop and study a novel "online" RPCA algorithm based on the recently introduced Recursive Projected Compressive Sensing (ReProCS) framework. Our algorithm improves upon the original ReProCS algorithm and it also returns even more accurate offline estimates. The key contribution of this work is a correctness result (complete performance guarantee) for this algorithm under reasonably mild assumptions. By using extra assumptions -- accurate initial subspace knowledge, slow subspace change, and clustered eigenvalues -- we are able to remove one important limitation of batch RPCA results and two key limitations of a recent result for ReProCS for online RPCA. To our knowledge, this work is among the first few correctness results for online RPCA. Most earlier results were only partial results, i.e., they required an assumption on intermediate algorithm estimates.

研究动机与目标

• 解决在现实假设下在线鲁棒主成分分析的开放问题，确保可证明的正确性。
• 克服以往在线RPCA方法对中间算法估计值假设的局限性。
• 开发一种既能提供准确在线估计，又能改善离线重建质量的算法。
• 基于ReProCS框架，通过增强的理论分析，建立在线RPCA的完整性能保证。

提出的方法

• 提出一种基于ReProCS框架的新在线RPCA算法，结合递归子空间估计与稀疏分量检测。
• 使用递归投影机制，在新数据到达时持续维护和更新低秩子空间估计。
• 采用基于阈值的检测方法，利用残差信号与结构化噪声建模来检测和估计稀疏异常值。
• 引入一种新颖的子空间变化检测机制，利用特征值聚集与矩阵扰动理论。
• 通过Weyl不等式对矩阵扰动的概率界进行建模，确保对噪声与估计误差的鲁棒性。
• 通过一系列引理与定理推导性能保证，条件基于子空间稳定性与特征值分离。

实验结果

研究问题

• RQ1 能否在已知初始子空间与缓慢子空间变化等现实假设下，设计出可证明正确的在线RPCA算法？
• RQ2 所提出的算法是否在在线与离线估计中均优于现有ReProCS变体？
• RQ3 能否在不依赖对中间算法估计值假设的前提下建立性能保证，如以往工作所要求的那样？
• RQ4 特征值聚集与结构化噪声模型如何影响在线子空间估计的稳定性与正确性？
• RQ5 在小范围非结构化噪声与稀疏污染下，子空间追踪精度的理论极限是什么？

主要发现

• 所提算法在温和假设下实现了在线RPCA的完整性能保证，包括准确的初始子空间知识、缓慢的子空间变化以及特征值聚集。
• 该算法在所有数据处理完成后，提供的离线估计比原始ReProCS更准确，显著提升了重建保真度。
• 理论分析表明，在给定条件下，正确子空间检测与稀疏分量估计的概率下限由高置信水平 $ p_{ ext{det},1} $ 保证。
• 该方法成功检测到子空间变化，并以有界误差估计新方向，最终定理中形式化为估计误差的 $ ilde{O}(1) $-范数界。
• 性能保证无需依赖对中间算法估计值的假设，与以往结果不同，因此构成完整的正确性结果。
• 通过一系列引理证明了算法的正确性，这些引理建立了特征值分离、矩阵扰动界与递归子空间估计精度。