[Paper Review] CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition

Yuge Huang, Yuhan Wang+6 more|arXiv (Cornell University)|Apr 1, 2020

Face recognition and analysis38 references30 citations

TL;DR

CurricularFace introduces an adaptive curriculum learning loss that emphasizes easy samples early and hard samples later by adaptively modulating negative cosine similarities during training, improving state-of-the-art face recognition performance.

ABSTRACT

As an emerging topic in face recognition, designing margin-based loss functions can increase the feature margin between different classes for enhanced discriminability. More recently, the idea of mining-based strategies is adopted to emphasize the misclassified samples, achieving promising results. However, during the entire training process, the prior methods either do not explicitly emphasize the sample based on its importance that renders the hard samples not fully exploited; or explicitly emphasize the effects of semi-hard/hard samples even at the early training stage that may lead to convergence issue. In this work, we propose a novel Adaptive Curriculum Learning loss (CurricularFace) that embeds the idea of curriculum learning into the loss function to achieve a novel training strategy for deep face recognition, which mainly addresses easy samples in the early training stage and hard ones in the later stage. Specifically, our CurricularFace adaptively adjusts the relative importance of easy and hard samples during different training stages. In each stage, different samples are assigned with different importance according to their corresponding difficultness. Extensive experimental results on popular benchmarks demonstrate the superiority of our CurricularFace over the state-of-the-art competitors.

Motivation & Objective

• Motivate margin- and mining-based losses in face recognition and address their convergence and sampling issues.
• Introduce an adaptive curriculum learning loss that selects and weights samples during training.
• Design a modulation function for negative cosine similarities that adjusts with training stage without manual hyper-parameter tuning.
• Demonstrate improved verification/identification performance on common face benchmarks compared to SOTA methods.

Proposed method

• Embed curriculum learning into the loss by adaptively weighting hard samples through a novel modulation I(t, cos theta_j).
• Define N(t, cos theta_j) = cos theta_j for easy/hard boundary cases, and N(t, cos theta_j) = (t + cos theta_j) cos theta_j for hard samples.
• Estimate t online via Exponential Moving Average of the positive cosine similarities to track training stages (t^{(k)} = alpha r^{(k)} + (1-alpha) t^{(k-1)}).
• Adopt ArcFace as the margin-based positive similarity T(cos theta_y_i) = cos(theta_y_i + m) and apply N(t, cos theta_j) to negative similarities in the softmax loss.
• Provide SGD-compatible gradients for x_i and W_j under three cases (y_i, easy negative, hard negative).
• Compare CurricularFace with ArcFace and MV-Arc-Softmax, highlighting adaptive weighting of hard samples and staged emphasis during training.

Experimental results

Research questions

• RQ1 Can adaptive curriculum learning improve discriminability in deep face recognition beyond fixed-margin and fixed mining losses?
• RQ2 Does online adaptive weighting of hard samples facilitate convergence on smaller backbones and under various benchmarks?
• RQ3 How does CurricularFace perform relative to ArcFace and MV-Arc-Softmax across standard verification/identification benchmarks?
• RQ4 Is the proposed t-estimation mechanism robust across datasets and backbone models?

Key findings

• CurricularFace achieves competitive to state-of-the-art results on LFW, CFP-FP, CPLFW, AgeDB, CALFW, IJB-B, IJB-C, and MegaFace with MS1MV2 / ResNet backbones.
• On LFW and CFP-FP/CPLFW benchmarks with MS1MV2-ResNet100, CurricularFace reaches 99.80% (LFW) and 98.37% (CFP-FP) and 93.13% (CPLFW), outperforming ArcFace and MV-Arc-Softmax in several metrics.
• Adaptive t estimation improves verification accuracy over fixed t values; adaptive mean of cos theta_y_i yields best results among strategies tested.
• CurricularFace demonstrates robustness to convergence issues on small backbones (e.g., MobileFaceNet) where ArcFace can diverge, while CurricularFace converges with m=0.5 or 0.45.
• Across MegaFace and IJB-B/IJB-C, CurricularFace achieves leading single-model identification/verification scores compared to strong baselines.
• The method adds minimal computational overhead relative to ArcFace (training time per iteration: 0.378s vs 0.370s on P40 GPUs).