[Paper Review] Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answering

Yang Deng, Wai Lam+5 more|arXiv (Cornell University)|Nov 22, 2019

Topic Modeling45 references4 citations

TL;DR

This paper proposes a joint learning framework, ASAS, that simultaneously performs answer selection and abstractive answer summarization in community question answering (CQA). By leveraging question-answer correlation to guide summary generation and using summaries to reduce noise in long answers, the model achieves state-of-the-art performance on both tasks and demonstrates strong transferability to low-resource CQA settings using a new large-scale dataset, WikiHowQA.

ABSTRACT

Community question answering (CQA) gains increasing popularity in both academy and industry recently. However, the redundancy and lengthiness issues of crowdsourced answers limit the performance of answer selection and lead to reading difficulties and misunderstandings for community users. To solve these problems, we tackle the tasks of answer selection and answer summary generation in CQA with a novel joint learning model. Specifically, we design a question-driven pointer-generator network, which exploits the correlation information between question-answer pairs to aid in attending the essential information when generating answer summaries. Meanwhile, we leverage the answer summaries to alleviate noise in original lengthy answers when ranking the relevancy degrees of question-answer pairs. In addition, we construct a new large-scale CQA corpus, WikiHowQA, which contains long answers for answer selection as well as reference summaries for answer summarization. The experimental results show that the joint learning method can effectively address the answer redundancy issue in CQA and achieves state-of-the-art results on both answer selection and text summarization tasks. Furthermore, the proposed model is shown to be of great transferring ability and applicability for resource-poor CQA tasks, which lack of reference answer summaries.

Motivation & Objective

• To address the redundancy and lengthiness of crowdsourced answers in CQA, which hinder accurate answer selection and user comprehension.
• To improve abstractive answer summarization by incorporating question-driven contextual information for more relevant and concise summaries.
• To reduce noise in long answers during answer selection by using generated summaries as a proxy for relevance.
• To enable effective transfer learning for resource-poor CQA tasks lacking reference summaries.
• To construct a new large-scale CQA dataset, WikiHowQA, supporting both answer selection and summarization tasks.

Proposed method

• Proposes a question-driven pointer-generator network that uses attention over question-answer pairs to guide abstractive summary generation.
• Designs a joint learning framework where answer summaries are used to refine the relevance scoring of question-answer pairs, reducing noise from lengthy answers.
• Employs a dual-stream mechanism: one for generating summaries using question-aware attention, and another for ranking answers based on summary relevance.
• Introduces a transfer learning strategy that fine-tunes a pre-trained model on WikiHowQA and adapts it to low-resource CQA domains without reference summaries.
• Uses a novel loss function that jointly optimizes for both answer selection accuracy and summary quality.
• Leverages the WikiHowQA dataset, which contains long answers and human-annotated reference summaries for training and evaluation.

Experimental results

Research questions

• RQ1 Can joint learning of answer selection and abstractive summarization improve performance on both tasks compared to independent models?
• RQ2 How effectively can question-aware summarization reduce noise in long answers during answer selection?
• RQ3 To what extent can a model pre-trained on a large-scale CQA dataset like WikiHowQA transfer to low-resource CQA domains without reference summaries?
• RQ4 Does using generated summaries as a proxy for relevance improve answer selection accuracy on long answers?
• RQ5 How does the joint model compare to extractive and abstractive summarization baselines in terms of summary quality and relevance?

Key findings

• The proposed ASAS model achieves state-of-the-art performance on answer selection, outperforming existing SOTA methods on the WikiHowQA dataset.
• On the WikiHowQA benchmark, ASAS achieves a 54.8% answer selection accuracy on the Travel domain, surpassing the best baseline (COALA at 52.9%).
• The model generates more precise and concise summaries than extractive (e.g., NeuralSum) and abstractive (e.g., PGN) baselines, as shown in qualitative examples.
• In low-resource settings, the ASAS model with fine-tuning (ASAS/Yes) achieves 56.5% accuracy on the Travel domain, outperforming the best baseline (COALA at 52.7%) by 3.8 percentage points.
• Even without fine-tuning, ASAS (Finetune/No) achieves competitive results across all domains, demonstrating strong zero-shot transferability.
• The joint learning framework significantly reduces the impact of answer redundancy, improving both selection accuracy and summary quality simultaneously.