[Paper Review] Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks

Yufeng Zhang, Xueli Yu+4 more|arXiv (Cornell University)|Apr 22, 2020

Topic Modeling21 references30 citations

TL;DR

TextING builds a separate document graph to learn fine-grained word interactions with a gated graph neural network, enabling inductive text classification and unseen word handling.

ABSTRACT

Text classification is fundamental in natural language processing (NLP), and Graph Neural Networks (GNN) are recently applied in this task. However, the existing graph-based works can neither capture the contextual word relationships within each document nor fulfil the inductive learning of new words. In this work, to overcome such problems, we propose TextING for inductive text classification via GNN. We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures, which can also effectively produce embeddings for unseen words in the new document. Finally, the word nodes are aggregated as the document embedding. Extensive experiments on four benchmark datasets show that our method outperforms state-of-the-art text classification methods.

Motivation & Objective

• Motivate overcoming limitations of global-structure text graphs in capturing within-document word interactions.
• Develop an inductive, per-document graph neural network to learn local word relations.
• Enable embeddings for unseen words in test documents.
• Demonstrate that per-document graphs improve classification accuracy over state-of-the-art baselines.

Proposed method

• Construct an individual word graph for each document using sliding window co-occurrences.
• Apply gated graph neural networks to propagate and update word node representations within each document graph.
• Use a readout function that combines attention-weighted word features and max-pooling to form a document representation.
• Classify by feeding the graph representation into a softmax layer and training with cross-entropy loss.
• Optionally extend with a multichannel TextING-M variant combining local and global graph signals.

Experimental results

Research questions

• RQ1 Can inductive text classification be improved by learning local word-word interactions within per-document graphs?
• RQ2 Do per-document graphs enable effective embeddings for unseen words in test documents?
• RQ3 How does TextING compare to existing graph-based and non-graph-based text classification methods on benchmark datasets?
• RQ4 What is the impact of graph depth (number of layers) and window size on performance?
• RQ5 Does a multichannel variant leveraging both local and global structures offer complementary gains?

Key findings

• TextING outperforms state-of-the-art baselines on four benchmark datasets (MR, R8, R52, Ohsumed).
• The method achieves strong gains in inductive settings with high unseen-word rates (e.g., ~21.06% average gain over TextGCN under inductive conditions).
• The per-document graphs capture fine-grained word relations that improve classification accuracy compared to global-structure graphs.
• TextING-M (multichannel) further improves performance across datasets.
• Attention visualization shows the readout emphasizes words correlating with the sentiment label in MR.