[Paper Review] Deep Long-Tailed Learning: A Survey

Yifan Zhang, Bingyi Kang+3 more|arXiv (Cornell University)|Oct 9, 2021

Domain Adaptation and Few-Shot LearningComputer Science166 references79 citations

TL;DR

A comprehensive survey of deep long-tailed learning, organizing methods into class re-balancing, information augmentation, and module improvement, with a new relative accuracy metric for evaluation.

ABSTRACT

Deep long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing deep models from a large number of images that follow a long-tailed class distribution. In the last decade, deep learning has emerged as a powerful recognition model for learning high-quality image representations and has led to remarkable breakthroughs in generic visual recognition. However, long-tailed class imbalance, a common problem in practical visual recognition tasks, often limits the practicality of deep network based recognition models in real-world applications, since they can be easily biased towards dominant classes and perform poorly on tail classes. To address this problem, a large number of studies have been conducted in recent years, making promising progress in the field of deep long-tailed learning. Considering the rapid evolution of this field, this paper aims to provide a comprehensive survey on recent advances in deep long-tailed learning. To be specific, we group existing deep long-tailed learning studies into three main categories (i.e., class re-balancing, information augmentation and module improvement), and review these methods following this taxonomy in detail. Afterward, we empirically analyze several state-of-the-art methods by evaluating to what extent they address the issue of class imbalance via a newly proposed evaluation metric, i.e., relative accuracy. We conclude the survey by highlighting important applications of deep long-tailed learning and identifying several promising directions for future research.

Motivation & Objective

• Summarize the challenges of training deep models under long-tailed class distributions.
• Categorize existing deep long-tailed learning methods into three main categories and nine sub-categories.
• Provide an empirical analysis framework using a new evaluation metric (relative accuracy) to assess how methods address imbalance.
• Highlight real-world applications and identify promising directions for future research.

Proposed method

• Organizes and reviews existing methods into three main categories: class re-balancing, information augmentation, and module improvement.
• Within class re-balancing, discusses re-sampling, class-sensitive learning, and logit adjustment.
• Reviews information augmentation including transfer learning and data augmentation.
• Covers module improvement approaches such as representation learning, classifier design, decoupled training, and ensemble learning.
• Proposes and applies a new evaluation metric, relative accuracy, to analyze state-of-the-art methods.

Experimental results

Research questions

• RQ1 How do existing deep long-tailed learning methods mitigate imbalanced class distributions across head, middle, and tail classes?
• RQ2 What are the core techniques within the three broad categories (re-balancing, augmentation, module improvement) that yield the best improvements for tail classes?
• RQ3 How effective are current methods when evaluated with the proposed relative accuracy metric?
• RQ4 What datasets and applications illustrate the practical impact and limitations of these methods?
• RQ5 What future directions and new task settings are most promising for advancing deep long-tailed learning?

Key findings

• Provides the first comprehensive survey of deep long-tailed learning up to mid-2021.
• Offers an in-depth review of advanced methods and an empirical analysis using a relative accuracy metric to assess imbalance handling.
• Identifies four potential directions for method innovation and eight new task settings for future research.
• Discusses real-world applications and the limitations of existing approaches in fine-grained, imbalanced visual tasks.