[Paper Review] Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization

Selim F. Yilmaz, Süleyman S. Kozat|arXiv (Cornell University)|May 12, 2020

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes an end-to-end deep metric learning framework for unsupervised anomaly detection that jointly optimizes data projection and metric learning to improve separation of anomalies from normal data. By introducing self-supervised data distillation and hard normal mining, the method achieves state-of-the-art performance with a 4.44% to 11.74% absolute AUC improvement across 14 real-world datasets, while maintaining O(1) inference complexity.

ABSTRACT

We investigate unsupervised anomaly detection for high-dimensional data and introduce a deep metric learning (DML) based framework. In particular, we learn a distance metric through a deep neural network. Through this metric, we project the data into the metric space that better separates the anomalies from the normal data and reduces the effect of the curse of dimensionality for high-dimensional data. We present a novel data distillation method through self-supervision to remedy the conventional practice of assuming all data as normal. We also employ the hard mining technique from the DML literature. We show these components improve the performance of our model and significantly reduce the running time. Through an extensive set of experiments on the 14 real-world datasets, our method demonstrates significant performance gains compared to the state-of-the-art unsupervised anomaly detection methods, e.g., an absolute improvement between 4.44% and 11.74% on the average over the 14 datasets. Furthermore, we share the source code of our method on Github to facilitate further research.

Motivation & Objective

• To address the challenge of unsupervised anomaly detection in high-dimensional data where labeled anomalies are costly or unavailable.
• To overcome suboptimal performance from decoupled dimensionality reduction and modeling steps in prior two-stage approaches.
• To reduce false positives and improve robustness by learning a metric space that better separates anomalies from normal instances.
• To enable efficient inference with O(1) time and memory complexity through a distance-to-center scoring function.
• To provide a generic, adaptable framework that supports various data types via modular neural network architectures.

Proposed method

• The method employs a deep neural network to project input data into a learned metric space where normal instances are pulled closer together.
• It uses a similarity-based loss function that minimizes distances between similar instances while maximizing those between dissimilar ones, optimized end-to-end.
• A novel self-supervised data distillation technique selects only the most likely normal instances (based on model confidence) for training at each epoch, improving robustness to outliers.
• Hard normal mining is applied by focusing training updates on the most challenging normal instances—those farthest from the metric-space center—to enhance discrimination.
• Anomaly scores are computed as the L2 distance from each instance to the centroid of the distilled normal data, enabling O(1) inference time and memory.
• The framework is adaptable to different data modalities: CNNs for images, RNNs for time series, and other architectures via modular design.

Experimental results

Research questions

• RQ1 Can end-to-end deep metric learning improve unsupervised anomaly detection performance compared to conventional two-stage approaches?
• RQ2 How does self-supervised data distillation enhance model robustness when training data contains unknown anomalies?
• RQ3 What is the impact of hard normal mining on model generalization and performance in the absence of labels?
• RQ4 Does the proposed method maintain high efficiency with O(1) inference complexity across diverse data types?
• RQ5 To what extent does the method generalize across datasets with varying anomaly rates and data distributions?

Key findings

• The proposed method achieves a 4.44% to 11.74% absolute improvement in average AUC across 14 real-world datasets compared to state-of-the-art unsupervised anomaly detection methods.
• The method demonstrates robust performance across all settings (seen, unseen, one-class), with minimal sensitivity to the data distillation ratio ρⁿ when ρⁿ > 0.5, indicating low hyperparameter sensitivity.
• Hard normal mining with ρʰ < 0.35 yields a slight performance gain in seen and unseen settings and reduces running time, making it both efficient and effective.
• The model maintains high performance even when the true normal rate is 0.9375, with optimal performance achieved at ρⁿ ≈ 0.5 due to self-supervised distillation.
• The method outperforms all baselines on the Letter dataset across all ρⁿ and ρʰ values, confirming consistent superiority in diverse settings.
• The O(1) scoring function enables real-time inference, making the method suitable for deployment in time-sensitive anomaly detection applications.