[Paper Review] Knowledge Guided Metric Learning for Few-Shot Text Classification

Dianbo Sui, Yubo Chen+4 more|arXiv (Cornell University)|Apr 4, 2020

Domain Adaptation and Few-Shot Learning17 references4 citations

TL;DR

This paper proposes a knowledge-guided metric learning framework for few-shot text classification that leverages external knowledge from a knowledge base to generate task-specific relation network parameters. By using a novel parameter generator network to adapt metrics to diverse tasks, the model achieves state-of-the-art performance of 87.93% accuracy on the ARSC dataset, outperforming existing methods including P-MAML.

ABSTRACT

The training of deep-learning-based text classification models relies heavily on a huge amount of annotation data, which is difficult to obtain. When the labeled data is scarce, models tend to struggle to achieve satisfactory performance. However, human beings can distinguish new categories very efficiently with few examples. This is mainly due to the fact that human beings can leverage knowledge obtained from relevant tasks. Inspired by human intelligence, we propose to introduce external knowledge into few-shot learning to imitate human knowledge. A novel parameter generator network is investigated to this end, which is able to use the external knowledge to generate relation network parameters. Metrics can be transferred among tasks when equipped with these generated parameters, so that similar tasks use similar metrics while different tasks use different metrics. Through experiments, we demonstrate that our method outperforms the state-of-the-art few-shot text classification models.

Motivation & Objective

• To address the challenge of low performance in few-shot text classification due to limited labeled data and task diversity.
• To imitate human-like few-shot learning by incorporating external knowledge into metric learning.
• To develop a parameter generator network that produces task-specific relation network parameters using knowledge base embeddings.
• To enable diverse metrics for diverse tasks, ensuring similar tasks use similar metrics while different tasks use distinct metrics.
• To achieve state-of-the-art performance in few-shot text classification without requiring backpropagation during inference.

Proposed method

• Uses BERT to encode input sentences into dense vector representations.
• Employs knowledge base embeddings (from NELL via BILINEAR model) to represent external knowledge.
• Designs a parameter generator network that takes task-specific knowledge embeddings and generates parameters for the relation network.
• Applies the generated parameters to tailor the metric function in the relation network for each task.
• Trains the model end-to-end using episode-based meta-learning, simulating few-shot scenarios during training.
• Uses mean-pooled sentence embeddings as class prototypes and computes similarity via the task-specific metric.

Experimental results

Research questions

• RQ1 Can external knowledge from a knowledge base improve few-shot text classification performance?
• RQ2 How can task-specific metrics be dynamically generated to adapt to diverse few-shot tasks?
• RQ3 Does knowledge-guided parameter generation lead to better generalization than fixed or shared metrics?
• RQ4 Can the model outperform existing state-of-the-art few-shot learning methods in text classification?
• RQ5 Is the performance gain due to knowledge integration or merely increased model capacity?

Key findings

• The proposed model achieves 87.93% mean accuracy on the ARSC dataset, surpassing the previous SOTA method P-MAML (86.65%).
• The ablation study shows that removing the task-relevant relation network reduces performance to 86.09%, confirming the importance of the proposed component.
• The replacement experiment, which increases model parameters but uses a task-agnostic network, yields only 86.40% accuracy, indicating that performance gain stems from knowledge integration, not parameter count.
• The model outperforms both metric-based models (e.g., Relation Network, Prototypical Network) and optimization-based methods (e.g., MAML) on the benchmark.
• The method does not require backpropagation during inference, making it more efficient than gradient-based meta-learning approaches.
• The results demonstrate that knowledge-guided metric learning enables better adaptation to diverse few-shot tasks by learning task-specific metrics.