[Paper Review] Anomaly detection using data depth: multivariate case

Pavlo Mozharovskyi, Valla, Romain|arXiv (Cornell University)|Oct 6, 2022

Advanced Statistical Methods and Models4 citations

TL;DR

This paper proposes data depth as a robust, affine-invariant method for multivariate anomaly detection, assigning anomalies to points with low depth values. It demonstrates that approximate depth computation—especially using optimized sampling like refined random search—enables efficient, scalable detection in high-dimensional data (up to d=50) with strong performance comparable to state-of-the-art methods like isolation forests and autoencoders.

ABSTRACT

Anomaly detection is a branch of data analysis and machine learning which aims at identifying observations that exhibit abnormal behaviour. Be it measurement errors, disease development, severe weather, production quality default(s) (items) or failed equipment, financial frauds or crisis events, their on-time identification, isolation and explanation constitute an important task in almost any branch of science and industry. By providing a robust ordering, data depth - statistical function that measures belongingness of any point of the space to a data set - becomes a particularly useful tool for detection of anomalies. Already known for its theoretical properties, data depth has undergone substantial computational developments in the last decade and particularly recent years, which has made it applicable for contemporary-sized problems of data analysis and machine learning. In this article, data depth is studied as an efficient anomaly detection tool, assigning abnormality labels to observations with lower depth values, in a multivariate setting. Practical questions of necessity and reasonability of invariances and shape of the depth function, its robustness and computational complexity, choice of the threshold are discussed. Illustrations include use-cases that underline advantageous behaviour of data depth in various settings.

Motivation & Objective

• To establish data depth as a practical tool for multivariate anomaly detection in industrial and scientific applications.
• To address computational challenges of data depth in high-dimensional and large-scale data settings.
• To evaluate the trade-offs between approximation accuracy, computational efficiency, and detection performance.
• To compare data depth-based anomaly detection with established methods such as autoencoders, one-class SVM, and isolation forests.
• To investigate the impact of sub-sampling and approximation algorithms on detection quality and runtime.

Proposed method

• Uses data depth as a non-parametric measure of centrality, assigning lower depth values to more anomalous points.
• Employs projection depth and halfspace depth as core depth functions, with robustness and affine invariance as key properties.
• Applies approximate computation via random search (RS) and refined random search (RRS) to reduce computational cost in high dimensions.
• Uses Nelder-Mead optimization in depth approximation to improve convergence and reduce variance compared to purely random methods.
• Implements sub-sampling of training data to reduce memory and computation load while maintaining detection quality.
• Evaluates performance using a measure of anomaly detection quality (equation 16) across 50 repetitions on synthetic data with 5% anomalies.

Experimental results

Research questions

• RQ1 How does data depth-based anomaly detection perform compared to established methods like isolation forests and one-class SVM in multivariate settings?
• RQ2 What is the impact of approximation methods (e.g., RS vs. RRS) on detection accuracy and computational efficiency in high-dimensional data?
• RQ3 How does sub-sampling the training set affect the quality of anomaly detection while reducing computational cost?
• RQ4 To what extent does data depth maintain robustness and affine invariance in real-world, non-elliptical data distributions?
• RQ5 Can data depth provide explainability in anomaly detection, and how does it compare to black-box models like autoencoders?

Key findings

• Data depth-based anomaly detection achieves strong performance comparable to state-of-the-art methods such as isolation forests and one-class SVM, even in complex, non-elliptical data distributions.
• With 200 directions, RRS approximation computed depth for 1,000 points in d=10 in under 20 seconds on a single-core Apple M1 Max chip, demonstrating high computational efficiency.
• Purely random search (RS) suffers from the curse of dimensionality, with computation time increasing to 500 seconds for 5,000 directions in d=10, while RRS maintains better scalability.
• In d=20, sub-sampling the training set reduces detection quality gradually rather than abruptly, suggesting a viable trade-off between speed and accuracy.
• At d=50, anomalies become unrecognizable in distance space, but data depth still enables effective detection, highlighting its utility in high-dimensional anomaly isolation.
• The use of Nelder-Mead optimization in depth approximation improves performance over random search, especially in higher dimensions, reducing variance and improving convergence.