[Paper Review] GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state

Junyi Bian, Xiaodi Huang+2 more|arXiv (Cornell University)|Nov 18, 2022

Topic Modeling4 citations

TL;DR

GoSum proposes a novel extractive summarization model for long scientific documents by integrating reinforcement learning with a heterogeneous graph neural network to encode hierarchical discourse structures. By modeling sentences and sections as nodes with discourse-aware edges, GoSum achieves state-of-the-art ROUGE scores on PubMed and arXiv, demonstrating that discourse hierarchy significantly improves summarization quality.

ABSTRACT

Extracting summaries from long documents can be regarded as sentence classification using the structural information of the documents. How to use such structural information to summarize a document is challenging. In this paper, we propose GoSum, a novel graph and reinforcement learning based extractive model for long-paper summarization. In particular, GoSum encodes sentence states in reinforcement learning by building a heterogeneous graph for each input document at different discourse levels. An edge in the graph reflects the discourse hierarchy of a document for restraining the semantic drifts across section boundaries. We evaluate GoSum on two datasets of scientific articles summarization: PubMed and arXiv. The experimental results have demonstrated that GoSum achieve state-of-the-art results compared with strong baselines of both extractive and abstractive models. The ablation studies further validate that the performance of our GoSum benefits from the use of discourse information.

Motivation & Objective

• Address the challenge of extractive summarization in long scientific documents, where traditional models struggle with memory cost and complex discourse structures.
• Overcome the limitations of extractive models that rely on suboptimal, non-deterministic labels from greedy search by leveraging reinforcement learning for better training signal.
• Integrate hierarchical discourse information—such as section boundaries and titles—into sentence representation to reduce semantic drift across document sections.
• Develop a graph-based method that captures both local sentence context and global document structure efficiently, enabling linear scalability for long inputs.
• Demonstrate that discourse-aware modeling significantly improves extractive summarization performance over baselines on scientific literature datasets.

Proposed method

• Construct a heterogeneous graph where each sentence and section is a node, and edges represent discourse hierarchy to encode structural relationships.
• Use graph neural networks (GNNs) to encode sentence states that include local sentence features, global document context, and extraction history.
• Frame summarization as a sequential decision-making process in reinforcement learning, where selecting a sentence is an action based on the current state.
• Train the model using reinforcement learning with a reward function based on ROUGE scores, enabling data augmentation through multiple sampled label sequences.
• Apply curriculum learning by corrupting section information incrementally to evaluate robustness and the importance of discourse structure.
• Use a policy network to predict sentence selection probabilities, optimized via policy gradient methods with rewards from ROUGE scores on gold summaries.

Experimental results

Research questions

• RQ1 Can a graph-structured representation of discourse hierarchy improve extractive summarization of long scientific documents?
• RQ2 How does the integration of reinforcement learning with GNNs enhance performance compared to supervised or non-graph baselines?
• RQ3 To what extent does discourse information—especially section titles and hierarchical structure—contribute to summarization quality?
• RQ4 How sensitive is the model to inaccuracies or corruption in discourse structure, such as missing or misattributed section titles?
• RQ5 Does increasing the number of sampled label sequences during reinforcement learning training lead to better generalization and performance?

Key findings

• GoSum achieves state-of-the-art performance on both PubMed and arXiv datasets, with ROUGE-1, ROUGE-2, and ROUGE-L scores of 49.83, 23.56, and 45.10 on PubMed, and 48.61, 20.53, and 42.80 on arXiv.
• Removing section titles (replaced with "section #id") leads to a slight but measurable drop in performance—0.20, 0.11, and 0.12 in ROUGE-1, ROUGE-2, and ROUGE-L on PubMed—indicating that section title semantics provide marginal but useful signal.
• Performance degrades significantly when section attribution is corrupted, especially at higher corruption rates, confirming that discourse hierarchy is critical for GoSum’s success.
• The ablation study shows that the model’s performance is most sensitive to accurate discourse structure, not just section titles, as the drop in performance is steeper when section boundaries are disrupted.
• Reinforcement learning with multiple sampled label sequences (top-k) significantly improves performance: top-4 sampling yields 49.64 ROUGE-1, close to the complete RL model’s 49.83.
• Removing the reward mechanism (setting all rewards to 1) reduces performance slightly, confirming that reward-based feedback is essential for distinguishing high-quality summaries during training.