[论文解读] Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks

Yufeng Zhang, Xueli Yu+4 more|arXiv (Cornell University)|Apr 22, 2020

Topic Modeling参考文献 21被引用 30

一句话总结

TextING 构建了一个独立的文档图，以学习细粒度的单词交互，采用门控图神经网络，使归纳文本分类和未见词处理成为可能。

ABSTRACT

Text classification is fundamental in natural language processing (NLP), and Graph Neural Networks (GNN) are recently applied in this task. However, the existing graph-based works can neither capture the contextual word relationships within each document nor fulfil the inductive learning of new words. In this work, to overcome such problems, we propose TextING for inductive text classification via GNN. We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures, which can also effectively produce embeddings for unseen words in the new document. Finally, the word nodes are aggregated as the document embedding. Extensive experiments on four benchmark datasets show that our method outperforms state-of-the-art text classification methods.

研究动机与目标

• Motivate overcoming limitations of global-structure text graphs in capturing within-document word interactions.
• Develop an inductive, per-document graph neural network to learn local word relations.
• Enable embeddings for unseen words in test documents.
• Demonstrate that per-document graphs improve classification accuracy over state-of-the-art baselines.

提出的方法

• Construct an individual word graph for each document using sliding window co-occurrences.
• Apply gated graph neural networks to propagate and update word node representations within each document graph.
• Use a readout function that combines attention-weighted word features and max-pooling to form a document representation.
• Classify by feeding the graph representation into a softmax layer and training with cross-entropy loss.
• Optionally extend with a multichannel TextING-M variant combining local and global graph signals.

实验结果

研究问题

• RQ1 Can inductive text classification be improved by learning local word-word interactions within per-document graphs?
• RQ2 Do per-document graphs enable effective embeddings for unseen words in test documents?
• RQ3 How does TextING compare to existing graph-based and non-graph-based text classification methods on benchmark datasets?
• RQ4 What is the impact of graph depth (number of layers) and window size on performance?
• RQ5 Does a multichannel variant leveraging both local and global structures offer complementary gains?

主要发现

模型	MR	R8	R52	Ohsumed
CNN (Non-static)	77.75 ± 0.72	95.71 ± 0.52	87.59 ± 0.48	58.44 ± 1.06
RNN (Bi-LSTM)	77.68 ± 0.86	96.31 ± 0.33	90.54 ± 0.91	49.27 ± 1.07
fastText	75.14 ± 0.20	96.13 ± 0.21	92.81 ± 0.09	57.70 ± 0.49
SWEM	76.65 ± 0.63	95.32 ± 0.26	92.94 ± 0.24	63.12 ± 0.55
TextGCN	76.74 ± 0.20	97.07 ± 0.10	93.56 ± 0.18	68.36 ± 0.56
Huang et al. (2019)	-	97.80 ± 0.20	94.60 ± 0.30	69.40 ± 0.60
TextING	79.82 ± 0.20	98.04 ± 0.25	95.48 ± 0.19	70.42 ± 0.39
TextING-M	80.19 ± 0.31	98.13 ± 0.12	95.68 ± 0.35	70.84 ± 0.52

• TextING outperforms state-of-the-art baselines on four benchmark datasets (MR, R8, R52, Ohsumed).
• The method achieves strong gains in inductive settings with high unseen-word rates (e.g., ~21.06% average gain over TextGCN under inductive conditions).
• The per-document graphs capture fine-grained word relations that improve classification accuracy compared to global-structure graphs.
• TextING-M (multichannel) further improves performance across datasets.
• Attention visualization shows the readout emphasizes words correlating with the sentiment label in MR.