[Paper Review] Long-tailed Recognition by Routing Diverse Distribution-Aware Experts

Xudong Wang, Long Lian+3 more|arXiv (Cornell University)|Oct 5, 2020

Domain Adaptation and Few-Shot Learning38 references30 citations

TL;DR

RIDE introduces a multi-expert model with a shared backbone and a dynamic routing module to reduce bias and variance in long-tailed recognition, achieving 5–7% gains on CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.

ABSTRACT

Natural data are often long-tail distributed over semantic classes. Existing recognition methods tackle this imbalanced classification by placing more emphasis on the tail data, through class re-balancing/re-weighting or ensembling over different data groups, resulting in increased tail accuracies but reduced head accuracies. We take a dynamic view of the training data and provide a principled model bias and variance analysis as the training data fluctuates: Existing long-tail classifiers invariably increase the model variance and the head-tail model bias gap remains large, due to more and larger confusion with hard negatives for the tail. We propose a new long-tailed classifier called RoutIng Diverse Experts (RIDE). It reduces the model variance with multiple experts, reduces the model bias with a distribution-aware diversity loss, reduces the computational cost with a dynamic expert routing module. RIDE outperforms the state-of-the-art by 5% to 7% on CIFAR100-LT, ImageNet-LT and iNaturalist 2018 benchmarks. It is also a universal framework that is applicable to various backbone networks, long-tailed algorithms, and training mechanisms for consistent performance gains. Our code is available at: https://github.com/frank-xwang/RIDE-LongTailRecognition.

Motivation & Objective

• Motivate the study of long-tailed recognition via bias-variance analysis under fluctuating training data distributions.
• Propose a principled model to reduce both bias (especially tail bias) and variance in long-tail classifiers.
• Develop RoutIng Diverse Experts (RIDE) with shared early features, diverse later-expert branches, and a dynamic router.
• Show that RIDE is universal across backbones and compatible with multiple long-tail loss functions and training schemes.

Proposed method

• Model a multi-expert architecture with shared early layers and n reduced-channel late branches.
• Train experts with a combination of per-expert classification losses and a distribution-aware diversity loss to promote complementary decisions.
• Introduce a router that dynamically activates additional experts only when needed, to control computation.
• Use softmax averaging of active experts’ logits at inference to produce final predictions.
• Optionally apply self-distillation from a higher-expert model to improve performance.
• Ground diversity and routing with a class-wise temperature mechanism that scales with class frequency to balance head/tail learning.

Experimental results

Research questions

• RQ1 How can model bias and variance be analyzed in long-tailed recognition as training data fluctuates?
• RQ2 Can a multi-expert architecture with distribution-aware diversity loss reduce tail bias and overall variance without prohibitive computation?
• RQ3 Does dynamic routing of experts maintain or improve accuracy across many-shot, medium-shot, and few-shot classes across diverse backbones?
• RQ4 Is RIDE a universal framework that can improve existing long-tail losses and training schemes across different architectures?
• RQ5 What are the empirical gains of RIDE on CIFAR100-LT, ImageNet-LT, and iNaturalist 2018 compared to state-of-the-art methods?

Key findings

• RIDE reduces model variance across all class splits compared to CE and existing long-tail methods.
• RIDE reduces tail bias via a distribution-aware diversity loss, tightening head-tail bias gaps.
• RIDE achieves 5–7% higher top-1 accuracy than SOTA on CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
• RIDE is compatible with various backbones (ResNet, ResNeXt, Swin Transformer) and loss functions (CE, LDAM, focal, etc.).
• A dynamic router enables multiple experts to be used only when needed, reducing computational cost to levels comparable with a single expert.
• Ablations show architecture, individual loss, diversity loss, routing, and distillation each contribute to performance, with the full setup delivering the largest gains.