[Paper Review] LST: Lexicon-Guided Self-Training for Few-Shot Text Classification

Hazel Kim, Jaeman Son+1 more|arXiv (Cornell University)|Feb 5, 2022

Text and Document Classification Technologies4 citations

TL;DR

LST proposes a lexicon-guided self-training method for few-shot text classification that enhances pseudo-labeling reliability by integrating linguistically enriched lexical knowledge. By refining a lexicon using attention-weighted frequent words and applying lexical data augmentation, LST achieves state-of-the-art performance, outperforming prior methods by 1.0–2.0% on five benchmark datasets with only 30 labeled samples per class.

ABSTRACT

Self-training provides an effective means of using an extremely small amount of labeled data to create pseudo-labels for unlabeled data. Many state-of-the-art self-training approaches hinge on different regularization methods to prevent overfitting and improve generalization. Yet they still rely heavily on predictions initially trained with the limited labeled data as pseudo-labels and are likely to put overconfident label belief on erroneous classes depending on the first prediction. To tackle this issue in text classification, we introduce LST, a simple self-training method that uses a lexicon to guide the pseudo-labeling mechanism in a linguistically-enriched manner. We consistently refine the lexicon by predicting confidence of the unseen data to teach pseudo-labels better in the training iterations. We demonstrate that this simple yet well-crafted lexical knowledge achieves 1.0-2.0% better performance on 30 labeled samples per class for five benchmark datasets than the current state-of-the-art approaches.

Motivation & Objective

• Address the overconfidence and error propagation in self-training due to unreliable pseudo-labels from limited labeled data.
• Improve pseudo-labeling reliability in few-shot text classification by incorporating structured lexical knowledge beyond model confidence.
• Develop a simple yet effective method that leverages lexicons for confidence regularization, selective sampling, and noise injection in self-training.
• Demonstrate that a refined lexicon with semantically meaningful word hierarchies (synonyms, hypernyms, etc.) enhances model generalization with minimal labeled data.
• Show that lexical knowledge injection through data augmentation improves model robustness and performance on unseen examples.

Proposed method

• Construct an initial lexicon from golden labeled data using top attention-weighted words from each class to capture representative terms.
• Refine the lexicon by selecting the top k% of frequent words from the most confident attention-weighted terms to balance coverage and reliability.
• Use the refined lexicon as a confidence regulator to filter and guide pseudo-labeling, reducing overconfidence in incorrect predictions.
• Implement selective sampling by prioritizing unlabeled examples whose words align with the lexicon, increasing informativeness of training data.
• Apply data augmentation using WordNet to generate semantically similar variants of input texts, enriching the lexicon and improving generalization.
• Iteratively update the lexicon and retrain the model in a teacher-student framework, where the student learns from both labeled data and lexicon-informed pseudo-labels.

Experimental results

Research questions

• RQ1 Can a linguistically enriched lexicon improve the reliability of pseudo-labels in few-shot text classification beyond model confidence alone?
• RQ2 How does selective sampling based on lexicon alignment affect the performance of self-training in low-resource settings?
• RQ3 To what extent does lexical data augmentation enhance model generalization and reduce error accumulation in self-training?
• RQ4 What is the optimal balance between lexicon size and word confidence for maximizing performance in few-shot learning?
• RQ5 Does integrating lexical knowledge as a confidence regulator outperform complex regularization techniques in self-training for text classification?

Key findings

• LST achieves state-of-the-art performance across five benchmark datasets, improving accuracy by 1.0–2.0% over existing self-training methods with only 30 labeled samples per class.
• The best performance is achieved with a lexicon refined using the top 10% of frequent words from high-attention terms, outperforming both smaller (1%) and larger (50%, 100%) lexicon sizes.
• LST without data augmentation still outperforms standard self-training by 3 percentage points on average, confirming the lexicon's critical role in improving pseudo-label quality.
• Lexical data augmentation contributes approximately 1% accuracy gain by enriching the lexicon and improving the model’s ability to identify meaningful unlabeled examples.
• The refined lexicon acts as an effective confidence regulator and selective sampler, reducing error propagation and improving model generalization.
• The method consistently outperforms UDA, a strong data augmentation baseline, by over 3 percentage points on average, demonstrating the superiority of lexicon-guided learning.