[Paper Review] Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective

Jialun Liu, Yifan Sun+3 more|arXiv (Cornell University)|Feb 25, 2020

Video Surveillance and Tracking Methods52 references38 citations

TL;DR

LEAP transfers intra-class angular diversity from head to tail classes by constructing a feature cloud around tail samples, improving discriminability in long-tailed data for person re-identification and face recognition. It achieves notable gains over strong baselines on multiple benchmarks.

ABSTRACT

This paper considers learning deep features from long-tailed data. We observe that in the deep feature space, the head classes and the tail classes present different distribution patterns. The head classes have a relatively large spatial span, while the tail classes have significantly small spatial span, due to the lack of intra-class diversity. This uneven distribution between head and tail classes distorts the overall feature space, which compromises the discriminative ability of the learned features. Intuitively, we seek to expand the distribution of the tail classes by transferring from the head classes, so as to alleviate the distortion of the feature space. To this end, we propose to construct each feature into a "feature cloud". If a sample belongs to a tail class, the corresponding feature cloud will have relatively large distribution range, in compensation to its lack of diversity. It allows each tail sample to push the samples from other classes far away, recovering the intra-class diversity of tail classes. Extensive experimental evaluations on person re-identification and face recognition tasks confirm the effectiveness of our method.

Motivation & Objective

• Motivate and address the challenge of learning discriminative deep features under long-tailed class distributions.
• Propose a learnable embedding augmentation framework that enlarges tail-class intra-class diversity by transferring from head classes.
• Model intra-class angular distributions and construct feature clouds to compensate tail samples during training.
• Evaluate the method on person re-identification and face recognition benchmarks to demonstrate effectiveness over strong baselines.

Proposed method

• Model intra-class feature distribution via angles between features and class centers.
• Maintain and update an angle memory per class to estimate Gaussian angular distributions.
• Compute head-class angular variance and transfer it to tail classes by constructing a feature cloud around each tail instance.
• Define tail-feature augmentation as sampling from a distribution with angle offset alpha ~ N(0, sigma_h^2 - sigma_t^2) to enlarge tail space.
• Integrate the augmentation with CosFace and ArcFace losses, yielding modified losses L3/L4 (full version) or L1/L2 as baselines; include clipping of angles to [0, pi].
• Provide vanilla (head/tail labels) and full (no explicit head/tail labeling) versions to adjust tail diversity without human interference.

Experimental results

Research questions

• RQ1 How does long-tailed data distort the feature space in deep embedding learning?
• RQ2 Can tail classes be compensated by transferring intra-class angular diversity learned from head classes?
• RQ3 Does feature-cloud based augmentation improve tail-class discrimination without requiring explicit head-tail labeling?
• RQ4 How does LEAP perform on person re-identification and face recognition benchmarks with varying head-tail ratios?

Key findings

• LEAP improves discriminability on long-tailed data, achieving higher Rank-1 and mAP than strong baselines on Market-1501 and DukeMTMC-reID.
• On Market-1501 and DukeMTMC-reID, LEAP-CF reaches 94.4% Rank-1 and 84.2% mAP (Market-1501) and 87.8% Rank-1 and 74.2% mAP (DukeMTMC-reID).
• On MSMT17, LEAP-CF attains 50.8 mAP and 76.7 Rank-1; LEAP-AF attains 51.3 mAP and 76.3 Rank-1.
• Compared to vanilla baselines, LEAP consistently boosts performance across varying head-tail ratios, especially under very severe long-tailed settings (e.g., H20/S3 or H20/S4).
• The full version (no explicit head/tail labeling) achieves results close to or better than the vanilla version, demonstrating robustness to dataset distribution without manual labeling.