[论文解读] Being Robust (in High Dimensions) Can Be Practical

Ilias Diakonikolas, Gautam Kamath+4 more|arXiv (Cornell University)|Mar 2, 2017

Statistical Methods and Inference参考文献 26被引用 72

一句话总结

本文提出近似样本最优、实用的算法，通过过滤方法对高维均值和协方差进行鲁棒估计，具有强大的经验表现。

ABSTRACT

Robust estimation is much more challenging in high dimensions than it is in one dimension: Most techniques either lead to intractable optimization problems or estimators that can tolerate only a tiny fraction of errors. Recent work in theoretical computer science has shown that, in appropriate distributional models, it is possible to robustly estimate the mean and covariance with polynomial time algorithms that can tolerate a constant fraction of corruptions, independent of the dimension. However, the sample and time complexity of these algorithms is prohibitively large for high-dimensional applications. In this work, we address both of these issues by establishing sample complexity bounds that are optimal, up to logarithmic factors, as well as giving various refinements that allow the algorithms to tolerate a much larger fraction of corruptions. Finally, we show on both synthetic and real data that our algorithms have state-of-the-art performance and suddenly make high-dimensional robust estimation a realistic possibility.

研究动机与目标

• 推动高维鲁棒统计的动机，并解决阻碍实际应用的计算限制。
• 提供鲁棒均值和协方差估计的近最优样本复杂度界限。
• 开发一种实用的基于过滤的算法，能够容忍一定比例的对照攻击污染。
• 证明鲁棒性保证可扩展到次高斯分布和有界矩分布。

提出的方法

• 使用过滤框架，基于经验协方差的谱特性迭代移除离群点。
• 沿着前几个特征向量执行一元尾部检验，以识别并修剪被污染的点。
• 通过自适应尾部界定优化阈值，以平衡去除坏点和好点。
• 通过监控高阶矩（如四阶矩）将过滤器扩展到鲁棒协方差估计。
• 通过使用鲁棒的一元均值（如中位数）来中心化，而不是经验均值，从而提升实际表现。

实验结果

研究问题

• RQ1 在高维中，基于过滤的鲁棒估计器是否能够达到近似最优的样本复杂度？
• RQ2 在均值和协方差估计中，可以容忍多大比例的对抗性污染而不牺牲鲁棒性？
• RQ3 在更弱的分布假设下，如有界矩、次高斯性，所提出的算法仍然有效吗？
• RQ4 哪些实际的调参策略（如自适应尾部）能在高维中提升经验表现？

主要发现

• The mean estimation algorithm achieves nearly optimal sample complexity ￿ormed as ￿or known covariance and ￿or unknown covariance under sub-Gaussian assumptions.
• Under bounded second moments, the mean estimator attains near-optimal error bounds with ￿ewer samples.
• The covariance estimator tolerates adversarial corruptions with error bounds in affine-invariant Mahalanobis distance.
• Adaptive tail bounding and empirical tuning significantly improve practical performance and dimension scalability.
• Empirical results show state-of-the-art performance on synthetic data and real data, with robustness extending to non-Gaussian settings.