[Paper Review] Few-Shot Text Classification with Induction Network.

Ruiying Geng, Binhua Li+4 more|arXiv (Cornell University)|Feb 27, 2019

Topic Modeling19 citations

TL;DR

This paper proposes an Induction Network that improves few-shot text classification by learning generalized, class-wise representations from support sets, using dynamic routing within a meta-learning framework. The model achieves over 3% higher average accuracy than state-of-the-art methods on both English sentiment and Chinese dialogue intent classification datasets.

ABSTRACT

Text classification tends to struggle when data is deficient or when it needs to adapt to unseen classes. In such challenging scenarios, recent studies often use meta learning to simulate the few-shot task, in which new queries are compared to a small support set on a sample-wise level. However, this sample-wise comparison may be severely disturbed by the various expressions in the same class. Therefore, we should be able to learn a general representation of each class in the support set and then compare it to new queries. In this paper, we propose a novel Induction Network to learn such generalized class-wise representations, innovatively combining the dynamic routing algorithm with the typical meta learning framework. In this way, our model is able to induce from particularity to university, which is a more human-like learning approach. We evaluate our model on a well-studied sentiment classification dataset (English) and a real-world dialogue intent classification dataset (Chinese). Experiment results show that, on both datasets, our model significantly outperforms existing state-of-the-art models and improves the average accuracy by more than 3%, which proves the effectiveness of class-wise generalization in few-shot text classification.

Motivation & Objective

• Address the challenge of few-shot text classification when training data is scarce or new classes emerge.
• Overcome limitations of sample-wise comparison in meta-learning, which is sensitive to intra-class expression variations.
• Develop a method that generalizes across samples within the same class to form robust, class-level representations.
• Enable meta-learning models to mimic human-like induction—from specific examples to general concepts—through a novel architecture.
• Improve performance on low-resource text classification tasks by focusing on class-wise generalization rather than instance-level matching.

Proposed method

• Introduce an Induction Network that learns a generalized, class-level representation from a small support set using dynamic routing.
• Integrate dynamic routing with a meta-learning framework to allow attention-based aggregation of support samples into a unified class prototype.
• Use a differentiable routing mechanism to adaptively weight contributions from individual support samples based on their relevance to the class.
• Train the model end-to-end using episodic training, where each episode simulates a few-shot classification task with support and query sets.
• Apply attention mechanisms to align query samples with the induced class representations, enabling accurate prediction in low-shot scenarios.
• Leverage the inductive bias of dynamic routing to generalize from specific examples to abstract class-level concepts, improving robustness to expression variation.

Experimental results

Research questions

• RQ1 Can learning a generalized class representation from a few support samples improve few-shot text classification beyond instance-level matching?
• RQ2 How does class-wise generalization via dynamic routing compare to traditional sample-wise comparison in meta-learning for text classification?
• RQ3 To what extent does the proposed Induction Network generalize across diverse text classification tasks with limited labeled data?
• RQ4 Does the model maintain performance gains when applied to low-resource, real-world scenarios such as dialogue intent classification?
• RQ5 Can the model achieve human-like inductive reasoning by inferring general concepts from specific examples in text?

Key findings

• The Induction Network significantly outperforms existing state-of-the-art models on both the English sentiment classification and Chinese dialogue intent classification datasets.
• The model improves average accuracy by more than 3% compared to prior methods, demonstrating the effectiveness of class-wise generalization.
• The performance gain is consistent across both low-resource and real-world text classification benchmarks, indicating robustness.
• The dynamic routing mechanism enables better representation learning by aggregating diverse expressions within the same class into a unified prototype.
• The model shows superior generalization to unseen classes due to its ability to induce general class-level features from limited support examples.
• The results validate that class-wise generalization is a critical factor in improving few-shot text classification beyond instance-level matching.