[Paper Review] Assessing Factoid Question-Answer Generation for Portuguese (Short Paper)

Ferreira, João, Rodrigues, Ricardo+1 more|arXiv (Cornell University)|May 4, 2017

Topic Modeling31 references52 citations

TL;DR

This paper proposes a sequence-to-sequence neural model with attention and pointer-softmax mechanisms to generate factoid questions from text passages and predefined answers. It combines maximum likelihood training with policy gradient optimization using rewards for fluency (perplexity), answerability (QA model performance), and a hybrid of both, demonstrating that policy gradient fine-tuning significantly improves question quality, especially in answerability, over baseline models.

ABSTRACT

We propose a recurrent neural model that generates natural-language questions from documents, conditioned on answers. We show how to train the model using a combination of supervised and reinforcement learning. After teacher forcing for standard maximum likelihood training, we fine-tune the model using policy gradient techniques to maximize several rewards that measure question quality. Most notably, one of these rewards is the performance of a question-answering system. We motivate question generation as a means to improve the performance of question answering systems. Our model is trained and evaluated on the recent question-answering dataset SQuAD.

Motivation & Objective

• To develop an end-to-end, text-to-text neural model for generating natural-language questions conditioned on documents and answers.
• To improve question quality beyond maximum likelihood training by incorporating reinforcement learning with task-specific rewards.
• To evaluate the impact of different reward combinations—fluency (PPL), answerability (QA), and hybrid (RPPL + QA)—on generated question quality.
• To investigate whether training with a question-answering model's performance as a reward improves the model's ability to generate questions that are both fluent and answerable.
• To explore the trade-offs between fluency, specificity, and abstraction in neural question generation.

Proposed method

• Uses an encoder-decoder architecture with separate encoders for the document and answer, employing attention mechanisms to align relevant context.
• Employs a pointer-softmax mechanism to copy words from the document or answer, improving generation of named entities and key terms.
• Trains the model using teacher forcing with maximum likelihood estimation on (document, answer, question) triples from the SQuAD dataset.
• Fine-tunes the model using policy gradient (REINFORCE-style) optimization to maximize auxiliary rewards: language model perplexity (PPL), QA system accuracy (QA), and a weighted combination (RPPL + QA).
• Applies entropy regularization and dropout to prevent overfitting during policy gradient training.
• Evaluates generated questions using automatic metrics (BLEU, F1, PPL) and human judgment, with qualitative analysis of question specificity and fluency.

Experimental results

Research questions

• RQ1 Can a neural sequence-to-sequence model generate high-quality, factoid questions conditioned on a document and a predefined answer?
• RQ2 How does combining maximum likelihood training with policy gradient optimization improve question generation quality compared to standard supervised training?
• RQ3 To what extent do different reward functions—fluency (PPL), answerability (QA), or a combination—impact the specificity, fluency, and answerability of generated questions?
• RQ4 Does training with a downstream QA model's performance as a reward lead to questions that are more likely to be correctly answered by existing QA systems?
• RQ5 What are the trade-offs between fluency, specificity, and abstraction in neural question generation, and how can they be balanced?

Key findings

• Policy gradient training with the QA reward (RQA) increased the question-answering accuracy of the generated questions by 8.9% over the improved Seq2Seq baseline, demonstrating significant gains in answerability.
• The RQA training scheme achieved a QA score of 73.7% on a test set, slightly exceeding the 73.2% accuracy of the model on ground-truth questions, indicating the generator may have learned to exploit the QA model’s answering patterns.
• The hybrid training scheme (RPPL + QA) achieved the best balance between fluency and answerability, improving QA performance by a large margin over the maximum-likelihood baseline while also regaining some of the PPL performance.
• Questions generated under the RQA setting were less fluent, as indicated by higher perplexity (e.g., 405 vs. 114 in baseline), suggesting a trade-off between answerability and fluency.
• The model learned to generate questions with interrogative words (e.g., 'what', 'when') placed at the end, a pattern that increases similarity to training data and improves QA matching, though it may reduce naturalness.
• Despite high BLEU scores, the model struggled with abstraction, often generating questions with high word overlap to the input, indicating limited ability to rephrase or reason beyond surface-level reordering.