[Paper Review] Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection

Yihe Dong, Samuel B. Hopkins+1 more|arXiv (Cornell University)|Jun 26, 2019

Fault Detection and Control Systems25 references33 citations

TL;DR

Introduces QUE-scoring via quantum entropy regularization to achieve nearly-linear time robust mean estimation and effective high-dimensional outlier detection, with theoretical guarantees and empirical validation.

ABSTRACT

We study two problems in high-dimensional robust statistics: \emph{robust mean estimation} and \emph{outlier detection}. In robust mean estimation the goal is to estimate the mean $μ$ of a distribution on $\mathbb{R}^d$ given $n$ independent samples, an $\varepsilon$-fraction of which have been corrupted by a malicious adversary. In outlier detection the goal is to assign an \emph{outlier score} to each element of a data set such that elements more likely to be outliers are assigned higher scores. Our algorithms for both problems are based on a new outlier scoring method we call QUE-scoring based on \emph{quantum entropy regularization}. For robust mean estimation, this yields the first algorithm with optimal error rates and nearly-linear running time $\widetilde{O}(nd)$ in all parameters, improving on the previous fastest running time $\widetilde{O}(\min(nd/\varepsilon^6, nd^2))$. For outlier detection, we evaluate the performance of QUE-scoring via extensive experiments on synthetic and real data, and demonstrate that it often performs better than previously proposed algorithms. Code for these experiments is available at https://github.com/twistedcubic/que-outlier-detection .

Motivation & Objective

• Motivate robust mean estimation and outlier detection in high dimensions under adversarial corruption.
• Develop QUE-scoring based on quantum entropy regularization to identify influential directions of outliers.
• Achieve near-linear time algorithms for robust mean estimation under bounded covariance and sub-Gaussian regimes.
• Provide an empirically validated outlier detection method leveraging QUE scores and spectral information.

Proposed method

• Define QUE scores using a density matrix U = exp(alpha * b1overSigma) / tr exp(alpha * b1overSigma) where overSigma is the empirical covariance and alpha is a tuning parameter.
• Integrate QUE into a matrix multiplicative weights framework to iteratively downweight suspected outliers.
• Use a DecreaseSpectralNorm subroutine to reduce the spectral norm of the weighted covariance with O(log d) rounds.
• Provide two algorithm variants: QUEScoreFilter for bounded covariance and s.g.-QUEScoreFilter for sub-Gaussian distributions.
• Show that QUE-based scoring improves on naive spectral scores by accounting for multiple large eigen-directions.
• Demonstrate nearly-linear running time by avoiding explicit matrix representations via low-rank sketches.

Experimental results

Research questions

• RQ1 Can QUE-scoring detect outliers more effectively in high-dimensional data with heterogeneous outlier directions?
• RQ2 Can robust mean estimation achieve near-optimal error with nearly-linear time under bounded covariance and sub-Gaussian assumptions?
• RQ3 How does QUE scoring compare to existing baselines (norm-based, top-eigenvector spectral, local methods) in practice?
• RQ4 What is the impact of the entropy-regularization parameter alpha on performance?

Key findings

• QUE scoring leads to near-linear time robust mean estimation with error bound 33sqrt(epsilon) + ~sqrt(d/n) under bounded covariance.
• Sub-Gaussian case achieves O(epsilon sqrt(log(1/epsilon)) + sqrt(d/n)) error bound with high probability.
• QUE-based outlier scoring often outperforms PCA-based, distance-based, and other baselines in high-dimensional experiments.
• Experiments on synthetic, word embeddings, and CIFAR-10 data show QUE improves ROCAUC scores across varied settings.
• A practical implementation computes approximate QUE scores in nearly-linear time using Johnson-Lindenstrauss sketches and matrix exponential approximations.