[Paper Review] Outlier Detection Using Nonconvex Penalized Regression

Yiyuan She, Art B. Owen|arXiv (Cornell University)|Jun 14, 2010

Advanced Statistical Methods and Models15 references4 citations

TL;DR

This paper proposes Θ-IPOD, a nonconvex penalized regression method for robust outlier detection in linear models by introducing a mean shift parameter for each data point and applying a thresholding-based regularization. The method uses hard thresholding to identify outliers, outperforms traditional M-estimators and L1-penalized methods in masking/swamping scenarios, and achieves faster computation via O(np) per-iteration complexity, especially effective in high-dimensional settings with sparse outliers and coefficients.

ABSTRACT

This paper studies the outlier detection problem from the point of view of penalized regressions. Our regression model adds one mean shift parameter for each of the $n$ data points. We then apply a regularization favoring a sparse vector of mean shift parameters. The usual $L_1$ penalty yields a convex criterion, but we find that it fails to deliver a robust estimator. The $L_1$ penalty corresponds to soft thresholding. We introduce a thresholding (denoted by $Θ$) based iterative procedure for outlier detection ($Θ$-IPOD). A version based on hard thresholding correctly identifies outliers on some hard test problems. We find that $Θ$-IPOD is much faster than iteratively reweighted least squares for large data because each iteration costs at most $O(np)$ (and sometimes much less) avoiding an $O(np^2)$ least squares estimate. We describe the connection between $Θ$-IPOD and $M$-estimators. Our proposed method has one tuning parameter with which to both identify outliers and estimate regression coefficients. A data-dependent choice can be made based on BIC. The tuned $Θ$-IPOD shows outstanding performance in identifying outliers in various situations in comparison to other existing approaches. This methodology extends to high-dimensional modeling with $p\gg n$, if both the coefficient vector and the outlier pattern are sparse.

Motivation & Objective

• To address the limitations of traditional outlier detection methods, such as masking and swamping, in the presence of multiple outliers.
• To develop a robust regression framework that jointly identifies outliers and estimates coefficients using a single tuning parameter.
• To improve computational efficiency over iteratively reweighted least squares (IRLS) by reducing per-iteration cost from O(np²) to O(np).
• To extend the method to high-dimensional settings where p ≫ n, under sparsity assumptions on both regression coefficients and outlier patterns.
• To establish a theoretical connection between the proposed thresholding procedure and M-estimation, demonstrating its robustness properties.

Proposed method

• Formulates a mean-shift model where each observation has a dedicated mean shift parameter γ_i, allowing any subset of points to be outliers.
• Applies a nonconvex penalty via a thresholding operator Θ (e.g., hard thresholding) to induce sparsity in the γ vector, favoring zero estimates for non-outliers.
• Develops an iterative algorithm, Θ-IPOD, where β is updated via OLS on y - γ, and γ is updated via thresholding of residuals: γ ← Θ(Hy + (I - H)y; λ).
• Uses a data-dependent tuning parameter selection via BIC to balance model fit and sparsity, reducing reliance on manual tuning.
• Employs a preliminary robust regression step to initialize β and γ, improving stability and convergence, consistent with best practices in robust statistics.
• Theoretical analysis shows that Θ-IPOD estimates are equivalent to M-estimators under the ψ-function induced by the thresholding rule, linking the method to established robust regression theory.

Experimental results

Research questions

• RQ1 Can a nonconvex penalized regression framework effectively detect multiple outliers while avoiding the masking and swamping effects common in classical methods?
• RQ2 How does the performance of hard thresholding (Θ) compare to soft thresholding (L1) in identifying outliers under various contamination scenarios?
• RQ3 To what extent can the proposed Θ-IPOD method maintain robustness and computational efficiency in high-dimensional settings (p ≫ n) with sparse outliers and coefficients?
• RQ4 What is the theoretical relationship between the thresholding-based iterative procedure and classical M-estimation?
• RQ5 Can a single tuning parameter simultaneously control outlier detection and coefficient estimation, with data-driven selection via BIC?

Key findings

• Θ-IPOD with hard thresholding correctly identifies outliers on hard test problems where L1-penalized regression fails due to its convexity and soft thresholding behavior.
• The method significantly outperforms traditional M-estimators and L1-penalized regression in terms of outlier detection accuracy, particularly in scenarios with multiple outliers.
• Θ-IPOD achieves faster computation than IRLS, with each iteration costing at most O(np), compared to O(np²) for IRLS, due to avoiding full least squares solves.
• The method is theoretically connected to M-estimation: the fixed-point solution of Θ-IPOD corresponds to an M-estimate with a ψ-function derived from the thresholding rule.
• When both the coefficient vector β and the outlier pattern γ are sparse, the method remains effective even in high-dimensional settings (p ≫ n), maintaining robustness and efficiency.
• A data-driven BIC-based tuning parameter selection yields strong empirical performance, reducing the need for manual hyperparameter tuning while maintaining high detection accuracy.