[Paper Review] Matching Long Text Documents via Graph Convolutional Networks

Bang Liu, Ting Zhang+4 more|arXiv (Cornell University)|Feb 21, 2018

Topic Modeling21 references24 citations

TL;DR

This paper proposes a graph convolutional network approach for matching long text documents by modeling semantic interactions between concepts via a Concept Interaction Graph. The Siamese GCN architecture learns document representations through graph convolution, significantly outperforming state-of-the-art methods on two Tencent-constructed news article datasets for event alignment tasks.

ABSTRACT

Identifying the relationship between two text objects is a core research problem underlying many natural language processing tasks. A wide range of deep learning schemes have been proposed for text matching, mainly focusing on sentence matching, question answering or query document matching. We point out that existing approaches do not perform well at matching long documents, which is critical, for example, to AI-based news article understanding and event or story formation. The reason is that these methods either omit or fail to fully utilize complicated semantic structures in long documents. In this paper, we propose a graph approach to text matching, especially targeting long document matching, such as identifying whether two news articles report the same event in the real world, possibly with different narratives. We propose the Concept Interaction Graph to yield a graph representation for a document, with vertices representing different concepts, each being one or a group of coherent keywords in the document, and with edges representing the interactions between different concepts, connected by sentences in the document. Based on the graph representation of document pairs, we further propose a Siamese Encoded Graph Convolutional Network that learns vertex representations through a Siamese neural network and aggregates the vertex features though Graph Convolutional Networks to generate the matching result. Extensive evaluation of the proposed approach based on two labeled news article datasets created at Tencent for its intelligent news products show that the proposed graph approach to long document matching significantly outperforms a wide range of state-of-the-art methods.

Motivation & Objective

• Address the challenge of matching long-form news articles that report the same real-world event but with different narratives.
• Overcome limitations of existing deep learning methods that fail to capture complex semantic structures in long documents.
• Develop a graph-based representation that models interactions between key concepts across sentences in long texts.
• Design a Siamese GCN framework to effectively learn and compare document-level representations for matching.
• Demonstrate superior performance on long document matching tasks compared to state-of-the-art approaches.

Proposed method

• Construct a Concept Interaction Graph where vertices represent coherent keyword groups (concepts) and edges represent semantic interactions between concepts via shared sentences.
• Use a Siamese neural network architecture to encode paired documents into shared embedding spaces for comparison.
• Apply graph convolutional networks (GCNs) to aggregate and update vertex features through message passing over the graph structure.
• Leverage the learned graph representations to produce a final matching score between document pairs.
• Train the end-to-end model using labeled pairs of news articles annotated for event equivalence.
• Utilize attention mechanisms implicitly through graph edge construction to emphasize salient concept interactions.

Experimental results

Research questions

• RQ1 Can a graph-based representation effectively model complex semantic structures in long documents for matching tasks?
• RQ2 How does the proposed Concept Interaction Graph compare to traditional sequence-based or attention-based models in capturing long-range dependencies?
• RQ3 To what extent does the Siamese GCN architecture improve matching performance on long document pairs compared to non-graph baselines?
• RQ4 Does the graph-based approach generalize well to real-world news article matching where narratives differ but events are identical?
• RQ5 What is the contribution of graph convolution to learning robust document-level representations for matching?

Key findings

• The proposed graph-based method significantly outperforms a wide range of state-of-the-art models on two large-scale, real-world news article datasets from Tencent.
• The model achieves superior performance in identifying whether two long-form news articles report the same underlying event despite differing narratives.
• The use of Concept Interaction Graphs enables better modeling of semantic interactions across long documents than conventional sequence or attention-based models.
• The Siamese GCN architecture effectively learns discriminative representations that enhance matching accuracy.
• Extensive ablation studies confirm the importance of graph structure and GCN aggregation in achieving high performance.
• The approach demonstrates strong generalization on real-world news data, highlighting its practical relevance for AI-powered news understanding systems.