[论文解读] Side Information in Robust Principal Component Analysis: Algorithms and Applications

Niannan Xue, Yannis Panagakis+1 more|arXiv (Cornell University)|Feb 2, 2017

Sparse and Compressive Sensing Techniques参考文献 32被引用 3

一句话总结

该论文提出了一种新颖的凸优化框架用于鲁棒主成分分析（RPCA），通过引入噪声侧信息（如低秩分量的粗略近似或其列/行空间）来提高恢复精度和鲁棒性。通过使用可证明收敛的ADMM求解器将此类侧信息整合到PCP框架中，该方法在四个应用中（包括背景减除和人脸识别）优于六种先前方法，同时通过减少所需训练样本数量降低了计算成本。

ABSTRACT

Dimensionality reduction and noise removal are fundamental machine learning tasks that are vital to artificial intelligence applications. Principal component analysis has long been utilised in computer vision to achieve the above mentioned goals. Recently, it has been enhanced in terms of robustness to outliers in robust principal component analysis. Both convex and non-convex programs have been developed to solve this new formulation, some with exact convergence guarantees. Its effectiveness can be witnessed in image and video applications ranging from image denoising and alignment to background separation and face recognition. However, robust principal component analysis is by no means perfect. This dissertation identifies its limitations, explores various promising options for improvement and validates the proposed algorithms on both synthetic and real-world datasets. Common algorithms approximate the NP-hard formulation of robust principal component analysis with convex envelopes. Though under certain assumptions exact recovery can be guaranteed, the relaxation margin is too big to be squandered. In this work, we propose to apply gradient descent on the Burer-Monteiro bilinear matrix factorisation to squeeze this margin given available subspaces. This non-convex approach improves upon conventional convex approaches both in terms of accuracy and speed. On the other hand, oftentimes there is accompanying side information when an observation is made. The ability to assimilate such auxiliary sources of data can ameliorate the recovery process. In this work, we investigate in-depth such possibilities for incorporating side information in restoring the true underlining low-rank component from gross sparse noise. Lastly, tensors, also known as multi-dimensional arrays, represent real-world data more naturally than matrices. It is thus advantageous to adapt robust principal component analysis to tensors. Since there is no exact equivalence between tensor rank and matrix rank, we employ the notions of Tucker rank and CP rank as our optimisation objectives. Overall, this dissertation carefully defines the problems when facing real-world computer vision challenges, extensively and impartially evaluates the state-of-the-art approaches, proposes novel solutions and provides sufficient validations on both simulated data and popular real-world datasets for various mainstream computer vision tasks.

研究动机与目标

• 解决标准RPCA因缺乏领域特定先验知识而导致退化或次优解的局限性。
• 开发一种鲁棒的凸优化框架，利用低秩分量的噪声近似作为侧信息，集成到RPCA框架中。
• 将方法扩展至在统一的算法框架中同时利用低秩矩阵的列空间和行空间的先验知识。
• 在包括背景减除、去噪人脸和识别任务在内的多样化计算机视觉应用中，证明所提方法的有效性和通用性。
• 通过引入侧信息，减少RPCA的归纳约束，实现在更少样本下的有效训练。

提出的方法

• 提出一种新的凸优化模型，通过约束形式将低秩分量L0的噪声近似W引入RPCA问题。
• 采用增广拉格朗日法结合交替方向乘子法（ADMM）求解所得优化问题，确保收敛性。
• 将模型扩展以同时包含低秩矩阵列空间（通过X）和行空间（通过Y）的侧信息，实现更灵活且精确的恢复。
• 对低秩分量施加核范数惩罚，对稀疏分量施加l1-范数惩罚，同时强制执行数据一致性与侧信息约束。
• 推导出ADMM子问题的闭式解，包括l1-范数的软阈值化和核范数的奇异值阈值化。
• 采用两阶段优化流程：首先利用侧信息估计低秩矩阵H，然后优化稀疏分量S和残差。

实验结果

研究问题

• RQ1 噪声侧信息（如粗略背景估计或中性人脸）是否能提升RPCA中低秩矩阵恢复的准确性？
• RQ2 如何有效整合低秩矩阵列空间和行空间的侧信息，以增强RPCA框架的性能？
• RQ3 引入侧信息是否能减少所需训练样本数量，从而缓解RPCA的归纳约束？
• RQ4 所提方法在多样化的现实世界数据集上与六种现有RPCA方法相比，其定量表现如何？
• RQ5 使用噪声侧信息W与直接从数据中减去W相比有何影响？为何后一种方法次优？

主要发现

• 所提出的PCPS方法在五个真实世界数据集（包括用于背景减除的Airport和PETS）上显著优于六种基线方法，实现了更优的前景分割和更干净的背景恢复。
• 在Airport数据集上，PCPS处理60帧仅需20秒，优于PCP（52秒）和FRPCAG（11秒），同时保持更优的准确性。
• 在人脸图像去噪任务中，该方法能有效从噪声输入中恢复低秩分量，更好地保留身份和表情特征，优于对比方法。
• 在人脸与表情识别任务中，使用侧信息显著提升了分类准确率，尤其在训练数据有限时表现更优。
• 该方法通过引入侧信息，减轻了RPCA的归纳约束，实现在更少样本下的有效学习，降低计算成本而不损失性能。
• 消融研究证实，直接从数据中减去侧信息（W）会导致有用特征丢失，并引入虚假噪声，破坏低秩假设，从而降低性能。