[Paper Review] A Question-Focused Multi-Factor Attention Network for Question Answering

Souvik Kundu, Hwee Tou Ng|arXiv (Cornell University)|Jan 25, 2018

Topic Modeling4 citations

TL;DR

This paper proposes AMANDA, a question-focused multi-factor attention network that enhances document-based question answering by jointly modeling multi-sentence evidence aggregation and question type awareness. Using tensor-based multi-factor attention and max-attentional question encoding, AMANDA achieves state-of-the-art performance on NewsQA, TriviaQA, and SearchQA, significantly outperforming prior models by leveraging deeper reasoning and answer type inference.

ABSTRACT

Neural network models recently proposed for question answering (QA) primarily focus on capturing the passage-question relation. However, they have minimal capability to link relevant facts distributed across multiple sentences which is crucial in achieving deeper understanding, such as performing multi-sentence reasoning, co-reference resolution, etc. They also do not explicitly focus on the question and answer type which often plays a critical role in QA. In this paper, we propose a novel end-to-end question-focused multi-factor attention network for answer extraction. Multi-factor attentive encoding using tensor-based transformation aggregates meaningful facts even when they are located in multiple sentences. To implicitly infer the answer type, we also propose a max-attentional question aggregation mechanism to encode a question vector based on the important words in a question. During prediction, we incorporate sequence-level encoding of the first wh-word and its immediately following word as an additional source of question type information. Our proposed model achieves significant improvements over the best prior state-of-the-art results on three large-scale challenging QA datasets, namely NewsQA, TriviaQA, and SearchQA.

Motivation & Objective

• To address the limitation of existing QA models in synthesizing evidence across multiple sentences for complex reasoning tasks.
• To improve answer extraction by explicitly modeling question type and key question words for better answer type alignment.
• To enhance performance on open-world QA datasets requiring deeper comprehension beyond simple span matching.
• To develop an end-to-end framework that integrates passage-question interaction with answer type inference for improved generalization.

Proposed method

• Employing a tensor-based multi-factor attention mechanism to aggregate meaningful facts across multiple sentences, enabling multi-sentence reasoning and co-reference resolution.
• Using a max-attentional question aggregation mechanism to identify and encode the most informative words in the question for answer type inference.
• Incorporating sequence-level representations of the first wh-word and its following word as an explicit signal for question type during prediction.
• Applying bi-directional LSTMs to encode passage and question representations with contextual awareness.
• Combining word-level embeddings from pre-trained GloVe and character-level CNNs to improve out-of-vocabulary word handling.
• Using pointer networks to predict answer spans by regressing start and end positions in the passage.

Experimental results

Research questions

• RQ1 Can a multi-factor attention mechanism effectively synthesize evidence from multiple sentences to improve reasoning in open-domain QA?
• RQ2 How does focusing on key question words improve answer type alignment and prediction accuracy?
• RQ3 To what extent does incorporating explicit question type signals (e.g., wh-word sequences) enhance performance on long-form and complex answers?
• RQ4 Does the proposed model outperform existing state-of-the-art approaches on challenging, open-world QA benchmarks?

Key findings

• AMANDA achieves state-of-the-art performance on three large-scale QA datasets: NewsQA, TriviaQA, and SearchQA, outperforming all prior published models.
• The model shows significant improvements in F1 and EM scores, particularly on complex, multi-sentence reasoning tasks.
• Performance degrades on 'why' and 'other' question types and longer answers, indicating challenges in boundary prediction for longer spans.
• Error analysis reveals 42% of failures stem from answer ambiguity, 22% from context-question word mismatches, and 10% from complex inference needs.
• Ablation studies confirm the importance of both the multi-factor attention and max-attentional question encoding components for optimal performance.