[Paper Review] Salience Allocation as Guidance for Abstractive Summarization

Fei Wang, Kaiqiang Song+7 more|arXiv (Cornell University)|Oct 22, 2022

Topic Modeling4 citations

TL;DR

This paper proposes Season, a novel abstractive summarization framework that uses salience allocation—estimating the degree of importance for each sentence in a document—as flexible, adaptive guidance for summarization models. By incorporating salience-aware cross-attention and using the expectation of salience scores, Season improves performance across multiple benchmarks, achieving +2.06/1.41/1.91 ROUGE-1/2/L on CNNDM and +0.33/0.32/0.60 on Newsroom, while revealing a natural 15-50% salience split in news articles.

ABSTRACT

Abstractive summarization models typically learn to capture the salient information from scratch implicitly. Recent literature adds extractive summaries as guidance for abstractive summarization models to provide hints of salient content and achieves better performance. However, extractive summaries as guidance could be over strict, leading to information loss or noisy signals. Furthermore, it cannot easily adapt to documents with various abstractiveness. As the number and allocation of salience content pieces vary, it is hard to find a fixed threshold deciding which content should be included in the guidance. In this paper, we propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON). SEASON utilizes the allocation of salience expectation to guide abstractive summarization and adapts well to articles in different abstractiveness. Automatic and human evaluations on two benchmark datasets show that the proposed method is effective and reliable. Empirical results on more than one million news articles demonstrate a natural fifteen-fifty salience split for news article sentences, providing a useful insight for composing news articles.

Motivation & Objective

• To address the limitations of extractive summaries as rigid, noisy, and inflexible guidance in abstractive summarization.
• To develop a more adaptive and reliable salience guidance mechanism that adjusts to documents with varying degrees of abstractiveness.
• To improve abstractive summarization performance by incorporating salience expectation into the decoder attention mechanism.
• To empirically investigate the natural distribution of salience in news articles using a large-scale dataset of over one million articles.

Proposed method

• A linear classifier is trained on top of the encoder to estimate the salience of each sentence based on its similarity to the ground-truth summary.
• Salience scores are mapped to discrete degrees with thresholds determined by corpus statistics to balance informativeness and accuracy.
• Salience-Aware Cross-Attention (SACA) dynamically controls how much attention the decoder pays to the salience-guided encoder states.
• Label smoothing is applied between adjacent salience degrees during training to improve robustness.
• The expectation of salience probabilities is used as a more stable and reliable salience estimation, with a sharpening coefficient τ to control distribution confidence.
• The method is evaluated on BART-based models using maximum likelihood estimation and is compatible with other backbones and objectives.

Experimental results

Research questions

• RQ1 Can salience allocation provide a more flexible and reliable alternative to extractive summaries as guidance in abstractive summarization?
• RQ2 How does salience-aware cross-attention improve summarization performance across documents with varying abstractiveness?
• RQ3 What is the natural distribution of salience in news articles, and does it follow a consistent pattern across large-scale datasets?
• RQ4 How does label smoothing and salience expectation improve model robustness and generalization?

Key findings

• Season achieves a 2.06/1.41/1.91 ROUGE-1/2/L performance gain over BART on the CNNDM dataset, demonstrating significant improvement in automatic evaluation.
• On the Newsroom dataset, Season improves ROUGE-1/2/L by 0.33/0.32/0.60 points, confirming effectiveness in diverse settings.
• Human evaluation confirms that Season generates more accurate and coherent summaries compared to baselines, especially in handling complex or ambiguous documents.
• Empirical analysis of over one million news articles reveals a consistent 15-50% salience split, where approximately 15% of sentences are highly salient and 50% are moderately to highly salient, offering insights for article composition.
• The use of salience expectation with optimal sharpening (τ=0.5) improves ROUGE-1 by 0.50 points, showing the benefit of refined probability calibration.
• Case studies demonstrate that Season adapts well to both concise and information-rich documents, avoiding over-generation and missing key facts better than extractive-guided models.