[Paper Review] Circle Loss: A Unified Perspective of Pair Similarity Optimization

Yifan Sun, Changmao Cheng+5 more|arXiv (Cornell University)|Feb 25, 2020

Face recognition and analysisComputer Science58 references102 citations

TL;DR

Circle Loss introduces a unified, flexible loss for deep feature learning by re-weighting similarity pairs with self-paced factors, yielding a circular decision boundary that improves optimization and convergence for both class-level and pair-wise supervision.

ABSTRACT

This paper provides a pair similarity optimization viewpoint on deep feature learning, aiming to maximize the within-class similarity $s_p$ and minimize the between-class similarity $s_n$. We find a majority of loss functions, including the triplet loss and the softmax plus cross-entropy loss, embed $s_n$ and $s_p$ into similarity pairs and seek to reduce $(s_n-s_p)$. Such an optimization manner is inflexible, because the penalty strength on every single similarity score is restricted to be equal. Our intuition is that if a similarity score deviates far from the optimum, it should be emphasized. To this end, we simply re-weight each similarity to highlight the less-optimized similarity scores. It results in a Circle loss, which is named due to its circular decision boundary. The Circle loss has a unified formula for two elemental deep feature learning approaches, i.e. learning with class-level labels and pair-wise labels. Analytically, we show that the Circle loss offers a more flexible optimization approach towards a more definite convergence target, compared with the loss functions optimizing $(s_n-s_p)$. Experimentally, we demonstrate the superiority of the Circle loss on a variety of deep feature learning tasks. On face recognition, person re-identification, as well as several fine-grained image retrieval datasets, the achieved performance is on par with the state of the art.

Motivation & Objective

• Explain the limitations of optimizing (s_n - s_p) in existing losses across classification and metric-learning paradigms.
• Propose Circle loss with self-paced weighting to adapt gradients based on optimization status.
• Show that Circle loss provides a unified formulation compatible with class-level and pair-wise labels.
• Demonstrate empirical improvements on face recognition, person re-identification, and fine-grained image retrieval.

Proposed method

• Reformulate pair similarity optimization as (α_n s_n − α_p s_p) with adaptive, non-negative weights α_n and α_p.
• Define α_p and α_n as self-paced terms: α_p^i = [O_p − s_p^i]_+ and α_n^j = [s_n^j − O_n]_+, where O_p and O_n are optimal similarities.
• Derive a circular decision boundary from the margins and relaxations, yielding a two-sided margin formulation with Δ_n and Δ_p.
• Show that Circle loss degenerates to triplet loss or classification losses under special cases, establishing a unified perspective.
• Provide analytical gradient expressions that reflect balanced, gradually attenuated updates to s_p and s_n.
• Empirically validate Circle loss on face recognition, re-ID, and fine-grained retrieval with robust hyper-parameter behavior.

Experimental results

Research questions

• RQ1 Can a weighted similarity-pair optimization improve convergence over traditional (s_n − s_p) based losses?
• RQ2 How can self-paced weighting be integrated into a unified Circle loss to support both class-level and pair-wise supervision?
• RQ3 Does Circle loss provide a more definite convergence target and better separability across tasks like face recognition and re-ID?
• RQ4 What are the empirical gains of Circle loss across standard benchmarks compared to softmax-based and triplet-based losses?

Key findings

• Circle loss achieves competitive or superior performance to state-of-the-art losses on face recognition, with CircleLoss (ours) attaining 97.81% rank-1 and 98.50% verification on MegaFace with ResNet34, and 98.12% rank-1 and 98.73% verification with ResNet100.
• In 1:1 IJB-C verification, Circle loss with ResNet34 reaches 96.04% TAR@1e-4 and 93.44% TAR@1e-5, outperforming ArcFace.
• Circle loss improves re-ID metrics, achieving 94.2% R-1 and 84.9% mAP on Market-1501 (ResNet50 + CircleLoss) and 96.1% R-1, 87.4% mAP on MSMT17.
• On fine-grained retrieval, Circle loss yields competitive results across CUB-200-2011, Cars196, and Stanford Online Products without elaborate hard-mining techniques.
• The method shows robustness to hyper-parameters γ and m, with Circle loss consistently surpassing competitive baselines across a broad range of settings.
• The gradient dynamics of Circle loss provide balanced, gradually attenuated updates and a definite convergence target due to the circular decision boundary.