[Paper Review] Robust Covariance and Scatter Matrix Estimation under Huber's Contamination Model

Mengjie Chen, Chao Gao+1 more|arXiv (Cornell University)|Jun 1, 2015

Advanced Statistical Methods and Models63 references16 citations

TL;DR

This paper proposes a robust covariance matrix estimator based on a novel matrix depth function that maximizes empirical depth to ensure resistance to outliers under Huber’s ε-contamination model. The estimator achieves minimax optimal rates for structured covariance matrices—such as banded, sparse, and low-rank—under high-dimensional settings, combining statistical efficiency with high breakdown point.

ABSTRACT

Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber's $ε$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

Motivation & Objective

• Address the lack of robustness in classical covariance estimators under high-dimensional settings with outliers.
• Develop a framework that maintains statistical efficiency while being resistant to arbitrary outliers.
• Introduce matrix depth as a multivariate generalization of Tukey’s depth for covariance estimation.
• Establish minimax optimality of the proposed estimator under ε-contamination for structured covariance matrices.
• Unify the minimax rate expression across different structures, showing dependence on both classical and contamination-induced error components.

Proposed method

• Define matrix depth for a positive semi-definite matrix Γ as the infimum over unit vectors u of the minimum of P(|uᵀX|² ≤ uᵀΓu) and P(|uᵀX|² ≥ uᵀΓu), capturing robustness to directional deviations.
• Propose a robust estimator by maximizing the empirical matrix depth function over the space of positive semi-definite matrices.
• Use the resulting estimator Γ̂ to construct Σ̂ = Γ̂ / β, where β is a scaling constant ensuring consistency under the normal model.
• Establish theoretical guarantees via a bias-variance tradeoff, leveraging concentration inequalities and operator norm bounds.
• Adapt the depth framework to structured models (banded, sparse, low-rank) by restricting the unit sphere to relevant subspaces or sparsity patterns.
• Apply Weyl’s inequality and Davis-Kahan theorem to derive spectral and subspace estimation error bounds for low-rank structure.

Experimental results

Research questions

• RQ1 Can a depth-based estimator achieve minimax optimality in high-dimensional covariance estimation under Huber’s ε-contamination model?
• RQ2 How does the proposed matrix depth function compare to classical estimators in terms of robustness and efficiency?
• RQ3 What is the minimax rate for structured covariance matrices (e.g., banded, sparse) under ε-contamination, and can it be unified across models?
• RQ4 Can the matrix depth framework be adapted to incorporate structural constraints like sparsity or bandedness without sacrificing robustness?
• RQ5 Does the estimator maintain high breakdown point while achieving optimal convergence rates in operator norm?

Key findings

• The proposed matrix depth-based estimator achieves the minimax optimal rate for banded and bandable covariance matrices under the ε-contamination model.
• For sparse covariance matrices, the estimator achieves a minimax rate of order (s log(ep/s))/n ∨ ε², where s is the sparsity level.
• The estimator maintains minimax optimality for low-rank covariance matrices, with error bounds scaling as (s log(ep/s))/n ∨ ε².
• The minimax rate under ε-contamination is unified as M(ε) ≍ max{M(0), ω(ε, F)}, where M(0) is the classical minimax rate and ω(ε, F) captures contamination-induced error.
• The estimator achieves a high breakdown point due to its depth-based construction, ensuring resistance to arbitrary outliers.
• Theoretical analysis confirms that the estimator’s operator norm error is bounded by C(ε + √((k + log m)/n) + ||Σₖ − Σ||_op), with k = ⌈n^(1/(2α+1))⌉ ∧ p, for banded models.