[Paper Review] FlowQA: Grasping Flow in History for Conversational Machine Comprehension

Hsin-Yuan Huang, Eunsol Choi+1 more|arXiv (Cornell University)|Oct 6, 2018

Topic Modeling22 references82 citations

TL;DR

FlowQA introduces a Flow mechanism that carries intermediate representations from past questions to better handle conversational machine comprehension, achieving large F1 gains on CoQA and QuAC.

ABSTRACT

Conversational machine comprehension requires the understanding of the conversation history, such as previous question/answer pairs, the document context, and the current question. To enable traditional, single-turn models to encode the history comprehensively, we introduce Flow, a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure. Compared to approaches that concatenate previous questions/answers as input, Flow integrates the latent semantics of the conversation history more deeply. Our model, FlowQA, shows superior performance on two recently proposed conversational challenges (+7.2% F1 on CoQA and +4.0% on QuAC). The effectiveness of Flow also shows in other tasks. By reducing sequential instruction understanding to conversational machine comprehension, FlowQA outperforms the best models on all three domains in SCONE, with +1.8% to +4.4% improvement in accuracy.

Motivation & Objective

• Motivate conversational machine comprehension that requires understanding conversation history.
• Propose Flow to encode history via intermediate representations from previous reasoning steps.
• Integrate Flow with a base single-turn MC model using an alternating parallel architecture for efficiency.
• Demonstrate performance gains on CoQA, QuAC, and sequential instruction tasks from SCONE.

Proposed method

• Introduce Flow as a mechanism to transfer intermediate context representations across question turns.
• Develop Integration-Flow (IF) layers that alternate between context-driven processing and Flow-driven processing for parallelism.
• Use fully-aware attention and hierarchical question encoding (QHierRNN) to integrate history.
• Extend a single-turn MC model with FlowQA’s reasoning and answer-prediction components.
• Train and evaluate on CoQA and QuAC with standard F1 and HEQ metrics; demonstrate improvements over baselines.
• Showcase Flow’s applicability to sequential instruction understanding (SCONE) via reduction to conversational MC.

Experimental results

Research questions

• RQ1 How can we effectively incorporate historical reasoning signals into conversational machine comprehension beyond simple concatenation of previous QA pairs?
• RQ2 What is the impact of Flow on performance in conversational MC benchmarks (CoQA, QuAC) and on related sequential instruction tasks?
• RQ3 Does the alternating parallel IF architecture provide practical training speedups while preserving accuracy?
• RQ4 How critical is Flow relative to other components (e.g., QHierRNN) in FlowQA’s performance?

Key findings

• FlowQA achieves +7.2 percentage points F1 on CoQA and +4.0 percentage points F1 on QuAC over prior models.
• On CoQA, FlowQA substantially improves across domains, with FlowQA (2-Ans and All-Ans variants) showing strong gains over baselines.
• Flow is a critical component, with removing Flow causing notable drops in QuAC and CoQA performances (4+ points in some cases).
• The alternating IF architecture provides substantial training speedups (8.1x on CoQA, 4.2x on QuAC) compared to naive Flow implementations.
• Flow also yields improvements on SCONE’s sequential instruction domains, outperforming prior state-of-the-art models.
• In Table 1, FlowQA (1-Ans) reaches 75.0 overall F1 on CoQA, versus 67.8 for BiDAF++ (3-ctx) and lower baselines; Human performance is 88.8.