[Paper Review] Question Answering and Question Generation as Dual Tasks

Duyu Tang, Nan Duan+3 more|arXiv (Cornell University)|Jun 7, 2017

Topic ModelingComputer Science27 references170 citations

TL;DR

The paper jointly trains QA and QG as dual tasks by exploiting probabilistic correlations between them, improving both tasks across MARCO, SQUAD, and WikiQA, with QA competitive to strong baselines and QG gains in BLEU-4.

ABSTRACT

We study the problem of joint question answering (QA) and question generation (QG) in this paper. Our intuition is that QA and QG have intrinsic connections and these two tasks could improve each other. On one side, the QA model judges whether the generated question of a QG model is relevant to the answer. On the other side, the QG model provides the probability of generating a question given the answer, which is a useful evidence that in turn facilitates QA. In this paper we regard QA and QG as dual tasks. We propose a training framework that trains the models of QA and QG simultaneously, and explicitly leverages their probabilistic correlation to guide the training process of both models. We implement a QG model based on sequence-to-sequence learning, and a QA model based on recurrent neural network. As all the components of the QA and QG models are differentiable, all the parameters involved in these two models could be conventionally learned with back propagation. We conduct experiments on three datasets. Empirical results show that our training framework improves both QA and QG tasks. The improved QA model performs comparably with strong baseline approaches on all three datasets.

Motivation & Objective

• Motivate the study of QA and QG as inherently connected, dual tasks with potential mutual benefits.
• Propose a joint training framework that regularizes both models via their probabilistic relationship.
• Demonstrate end-to-end differentiable training of QA and QG models from scratch.
• Evaluate the framework on three datasets to assess improvements in QA and QG performance.

Proposed method

• Implement a QA model based on a recurrent neural network with bidirectional processing and a concatenated representation for question-answer pairs.
• Implement a sequence-to-sequence QG model with attention to generate questions from answer sentences.
• Formulate a duality-based regularization term that enforces P(a)P(q|a;θqg) ≈ P(q)P(a|q;θqa) and minimize its squared error.
• Train QA and QG models jointly, using loss components for QA, QG, and the dual regularization, updated via backpropagation.
• Use smoothed bigram language models as P_a(a) and P_q(q); sample answer candidates to approximate P(a|q;θqa) during training.

Experimental results

Research questions

• RQ1 Does dual training improve QA performance over training QA and QG separately?
• RQ2 Does dual training improve QG performance as measured by BLEU-4?
• RQ3 How does the dual framework perform across diverse datasets (MARCO, SQUAD, WikiQA) for QA and QG?
• RQ4 What are the qualitative effects of dual training on generated questions and QA scoring?

Key findings

• Dual QA improves QA performance over Basic QA on SQUAD and is comparable to strong baselines on MARCO.
• Dual QG shows BLEU-4 gains when trained with the dual framework compared to Basic QG on MARCO, SQUAD, and WikiQA.
• On WikiQA, Dual QA achieves results comparable to strong baseline methods.
• The joint training framework yields improvements in both QA and QG over individually trained models.
• Generated questions from Dual QG tend to share more informative overlap with correct questions, aiding QA.
• BLEU-4 scores for QG remain relatively low overall, but show consistent improvement with dual training.