[Paper Review] The Cellwise Minimum Covariance Determinant Estimator

Jakob Raymaekers, Peter J. Rousseeuw|arXiv (Cornell University)|Jul 27, 2022

Advanced Statistical Methods and Models36 references4 citations

TL;DR

This paper introduces cellMCD, a novel cellwise robust estimator for multivariate covariance matrices that combines observed likelihood with a penalty on flagged cellwise outliers. It uses concentration steps to iteratively minimize an objective function, achieving high breakdown and finite-sample efficiency, outperforming casewise methods in scenarios with pervasive, non-marginal cellwise outliers.

ABSTRACT

The usual Minimum Covariance Determinant (MCD) estimator of a covariance matrix is robust against casewise outliers. These are cases (that is, rows of the data matrix) that behave differently from the majority of cases, raising suspicion that they might belong to a different population. On the other hand, cellwise outliers are individual cells in the data matrix. When a row contains one or more outlying cells, the other cells in the same row still contain useful information that we wish to preserve. We propose a cellwise robust version of the MCD method, called cellMCD. Its main building blocks are observed likelihood and a penalty term on the number of flagged cellwise outliers. It possesses good breakdown properties. We construct a fast algorithm for cellMCD based on concentration steps (C-steps) that always lower the objective. The method performs well in simulations with cellwise outliers, and has high finite-sample efficiency on clean data. It is illustrated on real data with visualizations of the results.

Motivation & Objective

• Address the limitation of traditional casewise robust methods in handling cellwise outliers—individual outlying cells that are not marginally extreme but collectively contaminate data.
• Develop a robust covariance estimator that preserves useful information in rows with mixed inliers and outliers, avoiding row-wise deletion.
• Ensure high breakdown point and finite-sample efficiency by combining likelihood-based estimation with a penalty on the number of flagged outliers.
• Create a fast, convergent algorithm based on concentration steps (C-steps) that monotonically decreases the objective function.
• Provide a unified framework for robust covariance estimation that handles both cellwise outliers and clean data efficiently.

Proposed method

• Propose a cellwise MCD estimator (cellMCD) that minimizes a penalized objective function combining observed log-likelihood and a penalty term on the number of flagged cellwise outliers.
• Use a likelihood-based approach to model the data under a multivariate Gaussian assumption, while flagging cells that deviate significantly from the estimated model.
• Incorporate a penalty parameter $ q_j $ for each variable to control the number of flagged outliers, preventing overfitting and ensuring robustness.
• Implement a C-step algorithm that alternately updates the location and scatter estimates, guaranteeing monotonic decrease in the objective function.
• Ensure the resulting covariance matrix remains positive semidefinite through the use of a consistent estimator and penalty regularization.
• Integrate the method with initial estimators (e.g., DDCW) and use multiple starting points (W) and EM-like iterations (EM) to improve convergence and stability.

Experimental results

Research questions

• RQ1 Can a robust covariance estimator be designed to handle cellwise outliers—individual outlying cells that are not rowwise outliers—without discarding entire rows?
• RQ2 Does a single-objective formulation combining likelihood and outlier penalty yield better performance than existing pairwise or wrapped estimators?
• RQ3 Can a fast, convergent algorithm be constructed for the cellwise MCD that guarantees monotonic improvement and high breakdown?
• RQ4 How does cellMCD compare in finite-sample efficiency and robustness to existing methods like MCD, DDCW, and cellMCD without penalty?
• RQ5 What is the impact of tuning parameters $ \varepsilon $ and $ \gamma $ on the estimator’s performance in terms of MSE and Kullback-Leibler divergence?

Key findings

• cellMCD achieves high breakdown point and maintains low Kullback-Leibler divergence even under high contamination levels (e.g., $ \varepsilon = 0.2 $), outperforming DDCW and cellMCD without penalty.
• For $ n = 800 $, $ d = 40 $, and $ \varepsilon = 0.2 $, cellMCD with $ \gamma = 10 $ and 10W+10EM iterations yields a KL divergence of 108,819.65, significantly lower than cellMCD_q0 (108,819.46) and DDCW (108,819.48), indicating superior robustness.
• Under clean data ($ \varepsilon = 0 $), cellMCD maintains high finite-sample efficiency, with MSE of $ \widehat{\mu} $ at 0.00122 and KL divergence of 2.184 for $ d=20 $, comparable to classical estimators.
• The C-step algorithm consistently reduces the objective function, with convergence observed across all tested configurations (e.g., $ \varepsilon = 0.1 $, $ \gamma = 10 $, 10W+10EM), ensuring algorithmic stability.
• The method performs well in simulations with both A09 and ALYZ contamination models, showing bounded influence and robustness even when 25% of cells per variable are contaminated.
• Cellwise outliers that are not marginally outlying are effectively detected and handled, as demonstrated by the method’s ability to maintain low KL divergence despite complex contamination patterns.