[論文レビュー] Hard Negative Mixing for Contrastive Learning

Yannis Kalantidis, Mert Bülent Sarıyıldız+3 more|arXiv (Cornell University)|Oct 2, 2020

Domain Adaptation and Few-Shot Learning参考文献 99被引用数 270

ひとこと要約

本論文はMoCHiを導入し、コントラスト学習の自己教師あり表現学習を改善するための特徴空間でのハードネガティブ混合戦略を提案し、オンザフライで難しいネガティブを合成して表現学習と転移性能を最小限のオーバーヘッドで向上させる。線形分類と転移タスクで一貫した利得を示し、特に前学習を短くした場合に顕著。

ABSTRACT

Contrastive learning has become a key component of self-supervised learning approaches for computer vision. By learning to embed two augmented versions of the same image close to each other and to push the embeddings of different images apart, one can train highly transferable visual representations. As revealed by recent studies, heavy data augmentation and large sets of negatives are both crucial in learning such representations. At the same time, data mixing strategies either at the image or the feature level improve both supervised and semi-supervised learning by synthesizing novel examples, forcing networks to learn more robust features. In this paper, we argue that an important aspect of contrastive learning, i.e., the effect of hard negatives, has so far been neglected. To get more meaningful negative samples, current top contrastive self-supervised learning approaches either substantially increase the batch sizes, or keep very large memory banks; increasing the memory size, however, leads to diminishing returns in terms of performance. We therefore start by delving deeper into a top-performing framework and show evidence that harder negatives are needed to facilitate better and faster learning. Based on these observations, and motivated by the success of data mixing, we propose hard negative mixing strategies at the feature level, that can be computed on-the-fly with a minimal computational overhead. We exhaustively ablate our approach on linear classification, object detection and instance segmentation and show that employing our hard negative mixing procedure improves the quality of visual representations learned by a state-of-the-art self-supervised learning method.

研究の動機と目的

• Motivate the importance of hard negatives in contrastive learning beyond large batch/memory sizes.
• Propose a feature-space hard negative mixing mechanism that creates synthetic hard negatives per query.
• Show that hard negative mixing improves transfer learning and representation utilization across tasks and epochs.
• Demonstrate that MoCHi yields competitive improvements on ImageNet-100 and transfer tasks like PASCAL VOC and COCO, especially with shorter training regimes.

提案手法

• Work within a MoCo-style momentum contrastive framework with a memory bank of negatives.
• Identify the hardest negatives for each query based on similarity to the query.
• Generate synthetic hard negatives by convex combinations of the closest negatives (hard negative mixing).
• Optionally mix the query with the hardest negatives to obtain even harder synthetic negatives.
• Append the synthetic negatives to the loss computation with minimal computational overhead (s + s' extra dot products).
• Use an MLP head as in related work and evaluate using standard linear evaluation and transfer tasks.

実験結果

リサーチクエスチョン

• RQ1 Can synthesizing hard negatives in embedding space lead to faster and more robust learning in contrastive self-supervised frameworks?
• RQ2 Does hard negative mixing improve transfer performance and embedding space utilization across vision tasks and datasets?
• RQ3 How do hyperparameters N, s, and s' affect proxy-task hardness and final representations?
• RQ4 What is the impact of MoCHi on alignment and uniformity of representations compared to MoCo-v2 and supervised baselines?

主な発見

• Hard negative mixing (MoCHi) yields consistent gains over MoCo-v2 on ImageNet-100 linear classification.
• Mixing both hardest negatives and the query produces stronger improvements and better space utilization than mixing negatives alone.
• MoCHi increases embedding space uniformity and improves transfer performance to PASCAL VOC and COCO, sometimes reaching supervised-like levels with shorter pre-training.
• On ImageNet-100, MoCHi variants achieve about +0.7% to +1.0% top-1 accuracy over MoCo-v2 in 200 epochs, with further gains in transfer tasks.
• MoCHi accelerates learning, enabling competitive performance after fewer pre-training epochs than baseline methods.
• Class oracle analysis suggests that removing same-class negatives from the memory can recover part of supervised performance, illustrating the impact of hard negatives on learning dynamics.