[Paper Review] CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances

Jihoon Tack, Sangwoo Mo+2 more|arXiv (Cornell University)|Jul 16, 2020

Anomaly Detection Techniques and Applications79 references230 citations

TL;DR

CSI introduces contrasting shifted instances in contrastive learning to enhance novelty (OOD) detection and confidence calibration, achieving state-of-the-art results across one-class, multi-class, and supervised settings.

ABSTRACT

Novelty detection, i.e., identifying whether a given sample is drawn from outside the training distribution, is essential for reliable machine learning. To this end, there have been many attempts at learning a representation well-suited for novelty detection and designing a score based on such representation. In this paper, we propose a simple, yet effective method named contrasting shifted instances (CSI), inspired by the recent success on contrastive learning of visual representations. Specifically, in addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Based on this, we propose a new detection score that is specific to the proposed training scheme. Our experiments demonstrate the superiority of our method under various novelty detection scenarios, including unlabeled one-class, unlabeled multi-class and labeled multi-class settings, with various image benchmark datasets. Code and pre-trained models are available at https://github.com/alinlab/CSI.

Motivation & Objective

• Motivate robust OOD/novelty detection when only in-distribution data is available.
• Learn discriminative representations tailored for distinguishing in-distribution vs. out-of-distribution.
• Develop a detection score that leverages the CSI-trained representation.
• Extend CSI to train confidence-calibrated classifiers in supervised settings.

Proposed method

• Use SimCLR-style contrastive learning as a base.
• Introduce contrasting shifted instances (con-SI) by treating distribution-shifting augmentations as negatives.
• Add an auxiliary task to classify which shifting transformation was applied (cls-SI).
• Train with the combined CSI objective: L_CSI = L_con-SI + lambda * L_cls-SI (lambda = 1).
• Define OOD detection scores that combine representation-based features (cosine similarity, norm) and shifted-instance information (s_con-SI, s_cls-SI).
• Optionally ensemble scores over random augmentations T(x) to obtain s_CSI-ens.

Experimental results

Research questions

• RQ1 How to learn representations that improve detection of out-of-distribution samples using contrastive learning?
• RQ2 Can distribution-shifting augmentations serve as informative negatives for OOD detection?
• RQ3 What score functions best exploit CSI representations for OOD detection and calibration?
• RQ4 Does CSI extend to confidence-calibrated classifiers in supervised settings?
• RQ5 Which shifting transformations (e.g., rotation, Gaussian noise) maximize OOD detection performance across datasets?

Key findings

• CSI outperforms baselines on one-class CIFAR-10, CIFAR-100, and ImageNet-30 across multiple OOD setups.
• Using distribution-shifting transformations as negatives (con-SI) significantly improves OOD detection even when standard representations are unchanged for classification.
• Auxiliary shifting-instance classifier (cls-SI) and combined CSI score (s_CSI) yield stronger OOD scores than base contrastive methods.
• Ensembling CSI scores over shifts and augmentations further improves AUROC and calibration metrics.
• In supervised settings, CSI-ens improves AUROC and reduces expected calibration error (ECE) while maintaining or boosting accuracy.