[Paper Review] Negative Margin Matters: Understanding Margin in Few-shot Classification

Bin Liu, Yue Cao+5 more|arXiv (Cornell University)|Mar 26, 2020

Domain Adaptation and Few-Shot LearningComputer Science54 references33 citations

TL;DR

The paper shows that using a negative margin in softmax losses improves few-shot classification by balancing base-class discriminability with transferability to novel classes, achieving state-of-the-art results on standard benchmarks.

ABSTRACT

This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and achieves state-of-the-art accuracy on three standard few-shot classification benchmarks with few bells and whistles. These results are contrary to the common practice in the metric learning field, that the margin is zero or positive. To understand why the negative margin loss performs well for the few-shot classification, we analyze the discriminability of learned features w.r.t different margins for training and novel classes, both empirically and theoretically. We find that although negative margin reduces the feature discriminability for training classes, it may also avoid falsely mapping samples of the same novel class to multiple peaks or clusters, and thus benefit the discrimination of novel classes. Code is available at https://github.com/bl0/negative-margin.few-shot.

Motivation & Objective

• Motivate why margin should be negative in open-set few-shot settings.
• Introduce negative-margin softmax and negative-margin cosine softmax losses for metric learning in few-shot tasks.
• Analyze how margins affect discriminability of base vs. novel classes.
• Demonstrate state-of-the-art performance on standard few-shot benchmarks across multiple backbones.

Proposed method

• Formulate a negative-margin softmax loss (Neg-Softmax) and a negative-margin cosine softmax loss (Neg-Cosine).
• Integrate the margin into the softmax objective with inner-product or cosine similarity.
• Define and compute class centers, inter-class and intra-class variances, and a discriminability function phi to analyze margin effects.
• Provide a theoretical analysis showing how negative margins can improve discriminability for novel classes under certain conditions.
• Adopt a two-stage training framework: pre-training on base classes with negative-margin losses, then fine-tuning a classifier for novel classes.

Experimental results

Research questions

• RQ1 Does a negative margin in softmax learning improve transferability to novel classes in few-shot settings?
• RQ2 How do margins affect discriminability for base vs. novel classes, both empirically and theoretically?
• RQ3 Can negative-margin losses achieve state-of-the-art performance on standard few-shot benchmarks across different backbones and datasets?
• RQ4 What is the practical effect of the margin sign on open-set few-shot classification under cross-domain and fine-grained settings.

Key findings

• Negative-margin softmax losses can outperform regular softmax losses in few-shot classification.
• Neg-Cosine often achieves state-of-the-art results across multiple backbones on mini-ImageNet for both 1-shot and 5-shot settings.
• Increasing positive margins improves base-class discriminability but hurts novel-class discrimination, while negative margins provide a better trade-off for novel-class discrimination.
• Empirical and theoretical analyses show that negative margins reduce the risk of mapping novel-class samples to multiple peaks across base classes, aiding transferability.
• Regularization techniques (weight decay, DropBlock, etc.) further boost performance, achieving strong results on multiple benchmarks.