[Paper Review] Generative Adversarial Active Learning for Unsupervised Outlier Detection

Yezheng Liu, Zhe Li+5 more|arXiv (Cornell University)|Sep 28, 2018

Anomaly Detection Techniques and ApplicationsComputer Science46 references59 citations

TL;DR

This paper proposes SO-GAAL and MO-GAAL, GAN-based frameworks to generate informative potential outliers for unsupervised outlier detection, addressing high-dimensional sparsity and mode collapse.

ABSTRACT

Outlier detection is an important topic in machine learning and has been used in a wide range of applications. In this paper, we approach outlier detection as a binary-classification issue by sampling potential outliers from a uniform reference distribution. However, due to the sparsity of data in high-dimensional space, a limited number of potential outliers may fail to provide sufficient information to assist the classifier in describing a boundary that can separate outliers from normal data effectively. To address this, we propose a novel Single-Objective Generative Adversarial Active Learning (SO-GAAL) method for outlier detection, which can directly generate informative potential outliers based on the mini-max game between a generator and a discriminator. Moreover, to prevent the generator from falling into the mode collapsing problem, the stop node of training should be determined when SO-GAAL is able to provide sufficient information. But without any prior information, it is extremely difficult for SO-GAAL. Therefore, we expand the network structure of SO-GAAL from a single generator to multiple generators with different objectives (MO-GAAL), which can generate a reasonable reference distribution for the whole dataset. We empirically compare the proposed approach with several state-of-the-art outlier detection methods on both synthetic and real-world datasets. The results show that MO-GAAL outperforms its competitors in the majority of cases, especially for datasets with various cluster types or high irrelevant variable ratio.

Motivation & Objective

• Frame outlier detection as a binary classification problem by artificially generating potential outliers.
• Develop SO-GAAL to generate informative outliers via a mini-max game between generator and discriminator.
• Extend to MO-GAAL with multiple generators to mitigate mode collapse and provide a comprehensive reference distribution.

Proposed method

• Formulate outlier detection as a classification task using a uniform reference distribution and a discriminator.
• Introduce SO-GAAL: a generator produces informative outliers and a discriminator defines the decision boundary.
• Analyze mode collapse and propose MO-GAAL with multiple generators learning distinct data subsets to create a mixture reference distribution.
• Define an objective where the discriminator S(x) serves as the outlier score OS(x) = 1 - D(x).
• Provide training procedure and stopping criteria to prevent collapse by transitioning from single to multiple generators.

Experimental results

Research questions

• RQ1 Can adversarial active learning generate informative potential outliers that help a classifier separate outliers from normal data in unsupervised settings?
• RQ2 Does expanding from a single generator to multiple generators (MO-GAAL) mitigate mode collapse and improve robustness across diverse data structures?
• RQ3 How do SO-GAAL and MO-GAAL perform relative to state-of-the-art outlier detection methods on synthetic and real-world high-dimensional datasets?

Key findings

• MO-GAAL outperforms competitors in the majority of cases, especially with varied cluster types or high irrelevant variable ratios.
• SO-GAAL shows enhanced boundary learning as informative outliers are generated, but can suffer from mode collapse without stopping criteria.
• MO-GAAL uses k sub-generators to learn distinct data subsets and create a comprehensive reference distribution for better detection.
• Empirical evaluation includes synthetic datasets with different cluster types and real-world datasets, using ROC-AUC as the performance metric.
• The approach is implemented in Keras and compared against density-based, distance-based, cluster-based, and one-class classifiers.