[Paper Review] Querent Intent in Multi-Sentence Questions

Laurie Burchell, Jie Chi+3 more|arXiv (Cornell University)|Oct 18, 2020

Topic Modeling19 references4 citations

TL;DR

This paper introduces a novel framework for identifying querent intent in multi-sentence questions (MSQs) by classifying them into five discourse relation types—Separable, Paraphrase, Disjunctive, Conditional, and Contrastive—based on speaker intent. Using a large-scale dataset of 162,745 MSQs from Stack Exchange, the authors develop a high-precision, rule-based classifier using surface-level contrastive features, achieving 82.9% precision but only 26.5% recall, highlighting the prevalence of implicit relations in real-world MSQs.

ABSTRACT

Multi-sentence questions (MSQs) are sequences of questions connected by relations which, unlike sequences of standalone questions, need to be answered as a unit. Following Rhetorical Structure Theory (RST), we recognise that different "question discourse relations" between the subparts of MSQs reflect different speaker intents, and consequently elicit different answering strategies. Correctly identifying these relations is therefore a crucial step in automatically answering MSQs. We identify five different types of MSQs in English, and define five novel relations to describe them. We extract over 162,000 MSQs from Stack Exchange to enable future research. Finally, we implement a high-precision baseline classifier based on surface features.

Motivation & Objective

• To address the gap in research on multi-sentence questions (MSQs), which are common in natural language but underexplored in question-answering systems.
• To define five distinct types of MSQs based on underlying discourse relations that reflect the querent’s intended answering strategy.
• To create a large, diverse, and publicly available dataset of 162,745 MSQs extracted from Stack Exchange to support future research.
• To develop a baseline classifier that identifies explicit discourse relations in MSQs using surface-level features.
• To demonstrate that current QA systems fail when treating MSQs as standalone questions, due to misaligned answering strategies.

Proposed method

• The authors define five MSQ types based on rhetorical structure theory (RST), each corresponding to a distinct querent intent and required answering strategy.
• They extract 162,745 two-question MSQs from 93 Stack Exchange sites using query-based data collection, filtering for question marks, length, and language.
• A rule-based classifier is built using binary contrastive features (e.g., presence/absence of discourse markers, punctuation, question structure) to distinguish the five MSQ types.
• Each MSQ type is assigned a unique feature mask specifying which features are required, disallowed, or ignored, ensuring mutual exclusivity.
• The classifier is evaluated on a hand-annotated test set of 271 MSQs with inter-annotator agreement, using precision and recall as metrics.
• The system focuses only on explicitly signaled relations, excluding implicit relations due to their complexity and ambiguity.

Experimental results

Research questions

• RQ1 How can querent intent in multi-sentence questions be systematically categorized based on discourse relations?
• RQ2 What are the most prevalent types of multi-sentence questions in real-world user queries?
• RQ3 Can surface-level linguistic features reliably classify MSQs into distinct intent-based categories?
• RQ4 How does the performance of a rule-based classifier compare to human annotation in identifying MSQ types?
• RQ5 To what extent are MSQs implicitly signaled, and how does this affect automatic classification?

Key findings

• The classifier achieved a precision of 82.9% on the annotated test set, indicating strong reliability in identifying the most explicitly signaled MSQ types.
• Recall was low at 26.5%, suggesting that most MSQs in real data are implicitly signaled and thus difficult to classify with surface features alone.
• Disjunctive and Conditional MSQs were most often explicitly signaled, as they commonly contain clear discourse markers like 'or' and 'if'.
• Separable MSQs, though the most frequent type, were poorly detected by the classifier, indicating they are predominantly implicit in structure.
• The annotation process revealed that many MSQs contain multiple intents or include phatic questions with no informational content, complicating classification.
• The dataset of 162,745 MSQs is publicly available, providing a valuable resource for future research on MSQ understanding and answering.