[Paper Review] Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets

Tong Wu, Qingqiu Huang+3 more|arXiv (Cornell University)|Jul 19, 2020

Machine Learning and Data Classification43 references36 citations

TL;DR

Introduces Distribution-Balanced Loss for multi-label recognition on long-tailed data, combining re-balanced weighting that accounts for label co-occurrence with negative-tolerant regularization to reduce over-suppression of negatives.

ABSTRACT

We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions. Compared to conventional single-label classification problem, multi-label recognition problems are often more challenging due to two significant issues, namely the co-occurrence of labels and the dominance of negative labels (when treated as multiple binary classification problems). The Distribution-Balanced Loss tackles these issues through two key modifications to the standard binary cross-entropy loss: 1) a new way to re-balance the weights that takes into account the impact caused by label co-occurrence, and 2) a negative tolerant regularization to mitigate the over-suppression of negative labels. Experiments on both Pascal VOC and COCO show that the models trained with this new loss function achieve significant performance gains over existing methods. Code and models are available at: https://github.com/wutong16/DistributionBalancedLoss .

Motivation & Objective

• Motivate robust multi-label learning under long-tailed distributions typical of real-world data.
• Address co-occurrence of labels and the dominance of negative labels in BCE-based training.
• Propose a loss that combines re-balanced weighting with negative-tolerant regularization.
• Demonstrate performance gains on standard long-tailed multi-label benchmarks (VOC-MLT and COCO-MLT).

Proposed method

• Introduce re-balanced weighting that accounts for label co-occurrence when adjusting sampling probabilities (Eq. 3–5).
• Compute class-aware and instance-level sampling probabilities to derive per-instance, per-class weights that mitigate inner-class imbalance (Eq. 2–3).
• Develop negative-tolerant regularization (NTR) to prevent over-suppression of negative labels by adapting BCE with class-specific bias and scaling (Eq. 7).
• Integrate re-balanced weighting and NTR into a unified Distribution-Balanced loss (DB loss) for end-to-end training (Eq. 10).
• Provide guidance on hyper-parameters for smoothing in re-balancing and the NTR terms, with empirical study across VOC-MLT and COCO-MLT.

Experimental results

Research questions

• RQ1 How can we re-balance losses in multi-label long-tailed settings while considering label co-occurrence?
• RQ2 How does negative-tolerant regularization affect learning dynamics for tail classes in BCE-based multi-label training?
• RQ3 What performance gains can be achieved by combining re-balanced weighting with negative-tolerant regularization on standard benchmarks?
• RQ4 Do the proposed methods maintain or improve head and medium class performance while boosting tail classes?

Key findings

• DB loss yields consistent improvements over baselines on VOC-MLT and COCO-MLT across total and tail subclasses.
• Re-balanced weighting alone improves overall mAP and particularly aids tail classes when combined with sampling.
• Negative-tolerant regularization provides notable gains for tail classes by mitigating over-suppression of negatives.
• DB-Focal further boosts performance, achieving the best overall results among compared methods.
• Tail classes benefit most from NTR, while head classes remain competitive with careful hyper-parameter tuning.