[Paper Review] On Multi-Domain Long-Tailed Recognition, Imbalanced Domain Generalization and Beyond

Yuzhe Yang, Hao Wang+1 more|arXiv (Cornell University)|Mar 17, 2022

Artificial Intelligence in Healthcare4 citations

TL;DR

This paper introduces Multi-Domain Long-Tailed Recognition (MDLT), a novel task addressing label imbalance across multiple domains with divergent class distributions and domain shift. It proposes BoDA, a theoretically grounded loss function that aligns features across domains while accounting for class imbalance, achieving state-of-the-art performance on MDLT and Domain Generalization benchmarks by optimizing an upper bound of transferability statistics.

ABSTRACT

Real-world data often exhibit imbalanced label distributions. Existing studies on data imbalance focus on single-domain settings, i.e., samples are from the same data distribution. However, natural data can originate from distinct domains, where a minority class in one domain could have abundant instances from other domains. We formalize the task of Multi-Domain Long-Tailed Recognition (MDLT), which learns from multi-domain imbalanced data, addresses label imbalance, domain shift, and divergent label distributions across domains, and generalizes to all domain-class pairs. We first develop the domain-class transferability graph, and show that such transferability governs the success of learning in MDLT. We then propose BoDA, a theoretically grounded learning strategy that tracks the upper bound of transferability statistics, and ensures balanced alignment and calibration across imbalanced domain-class distributions. We curate five MDLT benchmarks based on widely-used multi-domain datasets, and compare BoDA to twenty algorithms that span different learning strategies. Extensive and rigorous experiments verify the superior performance of BoDA. Further, as a byproduct, BoDA establishes new state-of-the-art on Domain Generalization benchmarks, highlighting the importance of addressing data imbalance across domains, which can be crucial for improving generalization to unseen domains. Code and data are available at: https://github.com/YyzHarry/multi-domain-imbalance.

Motivation & Objective

• To formalize Multi-Domain Long-Tailed Recognition (MDLT) as a new learning paradigm that generalizes across imbalanced domains with divergent label distributions.
• To address the limitations of single-domain long-tailed learning in real-world scenarios where data originate from multiple domains with varying class frequencies.
• To develop a unified framework—domain-class transferability graph—that models transferability between domain-class pairs under data imbalance.
• To design BoDA, a loss function that ensures balanced alignment and calibration across imbalanced domain-class distributions.
• To evaluate BoDA on five curated MDLT benchmarks and demonstrate its superiority over 20 existing methods, including state-of-the-art gains in domain generalization.

Proposed method

• Proposes the domain-class transferability graph, where nodes represent domain-class pairs and edges represent embedding space distances, to model inter-domain and inter-class transferability under imbalance.
• Introduces BoDA (Balanced Domain-Class Distribution Alignment), a loss function that encourages feature similarity for the same class across domains and penalizes similarity between different classes.
• BoDA explicitly accounts for class imbalance by optimizing an upper bound of balanced transferability statistics, ensuring stable learning despite skewed sample counts.
• The method uses a contrastive-style loss that penalizes inter-class and inter-domain feature similarity while promoting intra-class alignment, with regularization tuned to class frequency.
• Theoretical analysis proves that minimizing BoDA optimizes an upper bound of transferability statistics, justifying its effectiveness in MDLT.
• Empirically evaluates BoDA on five MDLT benchmarks derived from standard domain generalization datasets, with controlled label imbalance across domains.

Experimental results

Research questions

• RQ1 How does data imbalance across multiple domains affect model generalization, especially when label distributions differ between domains?
• RQ2 Can a unified representation learning framework effectively handle both in-domain label imbalance and cross-domain distribution shift in multi-domain recognition?
• RQ3 To what extent does the transferability between domain-class pairs govern the success of long-tailed learning in multi-domain settings?
• RQ4 Can a loss function be designed to balance alignment and calibration across imbalanced domain-class distributions while maintaining generalization to unseen domains?
• RQ5 Does addressing data imbalance across domains improve performance in domain generalization, even when the primary task is not long-tailed?

Key findings

• BoDA achieves consistent and significant accuracy gains over all 20 baselines across five MDLT benchmarks, with average improvements of up to 6.4% under extreme label imbalance.
• On the Digits-MLT dataset, BoDA maintains consistent transferability across all label configurations, while ERM fails under divergent and imbalanced distributions, showing a 6.4% average gain.
• The transferability statistics (α, β, γ) derived from the domain-class transferability graph show a strong correlation (R² > 0.9) with test accuracy across all MDLT datasets, validating their role as performance predictors.
• BoDA reduces feature discrepancy between training and test data—especially for tail classes—by up to 40% compared to ERM, leading to better domain generalization.
• When combined with state-of-the-art domain generalization methods, BoDA sets a new SOTA on DG benchmarks, demonstrating that imbalance-aware design enhances out-of-distribution generalization.
• Visualization confirms that BoDA learns more aligned and robust representations for minority classes (red regions in feature space), particularly under high domain shift and imbalance.