[Paper Review] Recent Advances in Hierarchical Multi-label Text Classification: A Survey

Rundong Liu, Wenhan Liang+6 more|arXiv (Cornell University)|Jul 30, 2023

Text and Document Classification Technologies4 citations

TL;DR

This survey reviews recent advances in hierarchical multi-label text classification, covering datasets, models, learning strategies, evaluation metrics, and challenges. It highlights deep learning-based approaches—especially graph neural networks and ensemble methods—as state-of-the-art, while identifying label imbalance, data sparsity, and deep-level prediction errors as key challenges, with future directions including instruction tuning and continual learning for evolving hierarchies.

ABSTRACT

Hierarchical multi-label text classification aims to classify the input text into multiple labels, among which the labels are structured and hierarchical. It is a vital task in many real world applications, e.g. scientific literature archiving. In this paper, we survey the recent progress of hierarchical multi-label text classification, including the open sourced data sets, the main methods, evaluation metrics, learning strategies and the current challenges. A few future research directions are also listed for community to further improve this field.

Motivation & Objective

• To systematically review recent progress in hierarchical multi-label text classification (HMLTC), a critical task in NLP for organizing scientific literature, product reviews, and news.
• To analyze existing datasets, model architectures, learning strategies, and evaluation metrics used in HMLTC research.
• To identify key challenges such as label imbalance, data sparsity, and error propagation in deep hierarchies.
• To propose future research directions, including instruction learning, domain knowledge injection, and incremental learning for evolving label hierarchies.
• To provide a comprehensive reference for researchers aiming to improve HMLTC systems in real-world applications.

Proposed method

• Categorizes HMLTC methods into four main approaches: tree-based, embedding-based, graph-based, and ensemble methods.
• Reviews the use of deep learning encoders with binary cross-entropy loss for end-to-end training, especially BERT and Transformer-based models.
• Examines graph neural networks (GNNs) that jointly model text and label hierarchy in a unified graph structure.
• Analyzes ensemble strategies that combine multiple hierarchical classifiers with a global predictor to improve performance.
• Discusses techniques for handling label imbalance, such as adaptive sampling (ARS2), reinforcement learning (HiLAP), and meta-learning (Meta-LMTC).
• Explores zero-shot and few-shot learning methods, including label hierarchy-aware models and contrastive representation learning.

Experimental results

Research questions

• RQ1 How do different model architectures—tree-based, embedding-based, graph-based, and ensemble—compare in performance and scalability for hierarchical multi-label text classification?
• RQ2 What are the most effective strategies for mitigating label imbalance and data sparsity in HMLTC, especially for low-frequency or rare labels?
• RQ3 To what extent can pre-trained language models and instruction tuning improve zero-shot and few-shot generalization in HMLTC?
• RQ4 How does error propagation from shallow to deeper levels affect classification accuracy, and what methods can reduce this effect?
• RQ5 What are the key challenges in enabling continual and incremental learning for evolving label hierarchies in real-world systems?

Key findings

• Graph neural network-based approaches have become the mainstream, effectively modeling both text and hierarchical label structures in a unified framework.
• Ensemble methods combining hierarchical and global classifiers achieve state-of-the-art performance, though formal academic validation remains limited.
• Label imbalance significantly degrades model performance on low-frequency labels, with methods like ARS2 and Meta-LMTC showing improved robustness through adaptive sampling and meta-learning.
• Zero-shot and few-shot learning remain challenging due to sparse label data and complex label-text relationships, though recent contrastive learning and hierarchy-aware models show promise.
• Deep-level labels suffer from lower accuracy due to sparsity and error propagation, highlighting the need for better hierarchical feature learning.
• Recent large language models like GPT-4 show strong zero-shot capabilities, but their reliability and alignment in HMLTC settings remain open research questions.