[Paper Review] Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation

Dayiheng Liu, Yeyun Gong+7 more|arXiv (Cornell University)|Apr 8, 2020

Advanced Text Analysis Techniques28 references4 citations

TL;DR

This paper proposes a keyphrase-aware, multi-headline generation method for news articles using a multi-source Transformer decoder that integrates keyphrases, filtered articles, and original content to produce diverse, high-quality, and relevant headlines. The authors introduce a large-scale (180K triple) keyphrase-aware news headline corpus mined from real user search and click data, achieving state-of-the-art performance in headline quality, relevance, and diversity on real-world benchmarks.

ABSTRACT

News headline generation aims to produce a short sentence to attract readers to read the news. One news article often contains multiple keyphrases that are of interest to different users, which can naturally have multiple reasonable headlines. However, most existing methods focus on the single headline generation. In this paper, we propose generating multiple headlines with keyphrases of user interests, whose main idea is to generate multiple keyphrases of interest to users for the news first, and then generate multiple keyphrase-relevant headlines. We propose a multi-source Transformer decoder, which takes three sources as inputs: (a) keyphrase, (b) keyphrase-filtered article, and (c) original article to generate keyphrase-relevant, high-quality, and diverse headlines. Furthermore, we propose a simple and effective method to mine the keyphrases of interest in the news article and build a first large-scale keyphrase-aware news headline corpus, which contains over 180K aligned triples of $$. Extensive experimental comparisons on the real-world dataset show that the proposed method achieves state-of-the-art results in terms of quality and diversity

Motivation & Objective

• To address the limitation of single-headline generation by enabling multiple, controllable, and keyphrase-relevant headlines for news articles.
• To build a large-scale, real-world keyphrase-aware news headline corpus using user search and click data to reflect user interests.
• To design a multi-source Transformer decoder that improves headline quality and keyphrase sensitivity through multi-source attention.
• To enable controlled, diverse, and high-quality headline generation by incorporating user-interest keyphrases as explicit guidance.
• To provide a controllable generation framework that supports personalized news recommendation and editorial assistance.

Proposed method

• A keyphrase mining method is proposed that identifies user-interest keyphrases by analyzing common phrases in user search queries and click-through data across news articles.
• A large-scale keyphrase-aware news headline corpus (KeyAware News) is constructed with 180,000 aligned triples of (news article, headline, keyphrase) from real-world Bing search logs.
• A multi-source Transformer decoder is designed with three inputs: the keyphrase, the keyphrase-filtered article, and the original article to enhance keyphrase relevance and diversity.
• Three variants of attention-based fusing mechanisms are explored and compared to effectively integrate multi-source information in the decoder.
• The headline generation model is trained and evaluated using a two-stage pipeline: keyphrase generation followed by keyphrase-aware headline generation.
• Human evaluation is conducted with expert judges to assess headline quality, with a qualified rate metric used to validate model outputs.

Experimental results

Research questions

• RQ1 Can keyphrase-aware headline generation improve the diversity and relevance of news headlines compared to single-headline baselines?
• RQ2 How effective is a multi-source Transformer decoder in integrating keyphrase, filtered article, and original article inputs for headline generation?
• RQ3 To what extent can mined user search and click data reliably identify keyphrases of user interest in news articles?
• RQ4 Does incorporating keyphrase guidance lead to higher-quality headlines that are more acceptable to human editors?
• RQ5 Can the proposed method generate multiple high-quality headlines per article that are both diverse and contextually relevant?

Key findings

• The proposed method achieves state-of-the-art performance in headline quality, relevance, and diversity on a real-world dataset, outperforming strong baselines.
• Human evaluation shows a 91.8% qualified rate for golden headlines, compared to 62.6% for the BASE + Stack model, indicating superior quality.
• The qualified rate for the proposed method (BASE + StackFuse) is significantly higher than for diverse baselines (36.2%) and random headlines (0.0%), confirming improved quality and acceptability.
• The model generates diverse headlines that are highly relevant to specified keyphrases, as validated by human annotators and automatic metrics.
• The keyphrase-aware corpus construction method successfully identifies user-interest keyphrases using search and click data, forming a reliable training and evaluation resource.
• The multi-source attention mechanism in the Transformer decoder effectively fuses keyphrase, filtered article, and original article inputs, enhancing both relevance and diversity.