[Paper Review] Generating Diverse Numbers of Diverse Keyphrases.

Xingdi Yuan, Tong Wang+4 more|arXiv (Cornell University)|Oct 11, 2018

Advanced Text Analysis Techniques32 references14 citations

TL;DR

This paper proposes a recurrent generative model that produces diverse, variable-length sets of keyphrases from text by incorporating dedicated modules to enhance diversity during sequential generation. Evaluated on three datasets—including two new ones, StackExchange and TextWorld ACG—the model outperforms baselines using both standard and proposed variable-length evaluation metrics, demonstrating superior performance in generating diverse, contextually relevant keyphrases without fixed output length constraints.

ABSTRACT

Existing keyphrase generation studies suffer from the problems of generating duplicate phrases and deficient evaluation based on a fixed number of predicted phrases. We propose a recurrent generative model that generates multiple keyphrases sequentially from a text, with specific modules that promote generation diversity. We further propose two new metrics that consider a variable number of phrases. With both existing and proposed evaluation setups, our model demonstrates superior performance to baselines on three types of keyphrase generation datasets, including two newly introduced in this work: StackExchange and TextWorld ACG. In contrast to previous keyphrase generation approaches, our model generates sets of diverse keyphrases of a variable number.

Motivation & Objective

• To address the limitations of existing keyphrase generation models that produce duplicate phrases and rely on fixed-length output evaluation.
• To develop a generative model capable of producing sets of diverse keyphrases with a variable number of outputs.
• To propose new evaluation metrics that account for variable-length keyphrase sets, improving assessment beyond fixed-length benchmarks.
• To evaluate the model on both existing and newly introduced datasets, including StackExchange and TextWorld ACG, to demonstrate robustness and generalization.

Proposed method

• A recurrent neural network architecture is used to generate keyphrases sequentially, allowing dynamic control over the number of generated phrases.
• Specific modules are integrated into the model to actively promote diversity among generated keyphrases, reducing repetition.
• The model is trained end-to-end using a sequence-to-sequence framework with attention mechanisms to attend to relevant parts of the input text.
• Two new evaluation metrics are proposed that consider the variable number of predicted keyphrases, enabling fairer comparison across models with differing output lengths.
• The model is fine-tuned and evaluated on three datasets: two existing benchmarks and two newly introduced datasets—StackExchange and TextWorld ACG.
• The training objective incorporates diversity regularization to discourage the model from generating redundant or overlapping keyphrases.

Experimental results

Research questions

• RQ1 Can a generative model produce diverse keyphrases without being constrained to a fixed output length?
• RQ2 How does the proposed model compare to existing baselines in terms of keyphrase diversity and relevance across different datasets?
• RQ3 To what extent do the proposed variable-length evaluation metrics improve the fairness and accuracy of model comparison in keyphrase generation?
• RQ4 Does the model generalize well to new domains, as demonstrated on the newly introduced StackExchange and TextWorld ACG datasets?
• RQ5 How effective are the proposed diversity-promoting modules in reducing duplicate keyphrase generation?

Key findings

• The proposed model achieves superior performance compared to baselines across all three datasets, including the two newly introduced ones: StackExchange and TextWorld ACG.
• The model generates keyphrase sets with significantly higher diversity, as measured by reduced repetition and improved coverage of distinct concepts.
• The proposed evaluation metrics effectively capture performance differences across models with variable output lengths, offering a more accurate assessment than fixed-length benchmarks.
• On the new StackExchange and TextWorld ACG datasets, the model demonstrates strong generalization and robustness, outperforming existing methods in both diversity and relevance.
• The integration of diversity-promoting modules leads to a measurable reduction in duplicate keyphrases, confirming their effectiveness in the generation process.