[Paper Review] OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations

Pramuditha Perera, Ramesh Nallapati+1 more|arXiv (Cornell University)|Mar 20, 2019

Anomaly Detection Techniques and Applications14 references56 citations

TL;DR

OCGAN learns a denoising auto-encoder with a constrained latent space and dual discriminators to force every latent sample to decode to the known class, enabling effective one-class novelty detection with adversarial training and informative-negative mining.

ABSTRACT

We present a novel model called OCGAN for the classical problem of one-class novelty detection, where, given a set of examples from a particular class, the goal is to determine if a query example is from the same class. Our solution is based on learning latent representations of in-class examples using a denoising auto-encoder network. The key contribution of our work is our proposal to explicitly constrain the latent space to exclusively represent the given class. In order to accomplish this goal, firstly, we force the latent space to have bounded support by introducing a tanh activation in the encoder's output layer. Secondly, using a discriminator in the latent space that is trained adversarially, we ensure that encoded representations of in-class examples resemble uniform random samples drawn from the same bounded space. Thirdly, using a second adversarial discriminator in the input space, we ensure all randomly drawn latent samples generate examples that look real. Finally, we introduce a gradient-descent based sampling technique that explores points in the latent space that generate potential out-of-class examples, which are fed back to the network to further train it to generate in-class examples from those points. The effectiveness of the proposed method is measured across four publicly available datasets using two one-class novelty detection protocols where we achieve state-of-the-art results.

Motivation & Objective

• Motivate and address limitations of latent-space representations where out-of-class objects are sometimes well-reconstructed by models trained on a single class.
• Propose a latent-space constraint and adversarial training to ensure all latent samples reproduce only the known class.
• Develop an end-to-end training framework combining denoising auto-encoder, two discriminators (latent and visual), and a classifier.
• Incorporate informative-negative mining to improve robustness against latent-space regions yielding out-of-class-like outputs.

Proposed method

• Use a denoising auto-encoder (En, De) with a tanh-activated encoder output to bound latent space to (-1,1)^d.
• Train a latent-space discriminator Dl to force latent representations of in-class data to follow a Uniform(-1,1)^d distribution.
• Introduce a visual discriminator Dv to ensure that images generated from random latent samples resemble the in-class data distribution.
• Add a classifier trained on reconstructions as positives and random-latent-generated images as negatives to identify informative-negative samples.
• Implement an informative-negative mining procedure that uses classifier loss gradients to move in latent space toward regions likely to produce out-of-class images.
• Train in two iterative steps: (i) train classifier with fixed auto-encoder and discriminators, (ii) train auto-encoder and discriminators with informative-negative mining.
• Leverage a composite loss: l_latent + l_visual for discriminators and l_mse + l_visual + l_latent for the generator, with a strong reconstruction term.

Experimental results

Research questions

• RQ1 Can constraining the latent space to uniformly cover a bounded region force the model to represent only the given class?
• RQ2 Does adversarial training with latent and visual discriminators prevent generation of out-of-class-like images from random latent samples?
• RQ3 Does informative-negative mining improve detection by exposing the model to latent regions that produce poor in-class representations?

Key findings

• OCGAN achieved high AUC performance across four public datasets, including a 0.995 AUC on COIL100 under Protocol 1.
• On MNIST-like data, the method outperformed several baselines, with notable gains over prior work on MNIST and fMNIST when evaluated under Protocol 1.
• For CIFAR-10 under Protocol 2, OC GAN achieved a mean AUC of 0.9750, competitive with or superior to several baselines.
• Ablation study on MNIST showed progressive improvements from adding latent and visual discriminators and informative-negative mining, culminating in the full model.
• Qualitative analysis showed that informative-negative mining reduces generation of out-of-class-like samples and yields more consistent in-class outputs across latent space.