[Paper Review] AMPERSAND: Argument Mining for PERSuAsive oNline Discussions

Tuhin Chakrabarty, Christopher Hidey+3 more|arXiv (Cornell University)|Apr 30, 2020

Topic Modeling33 references4 citations

TL;DR

AMPERSAND proposes a novel argument mining framework for online persuasive discussions that integrates micro-level (argument components) and macro-level (dialogic relations) structures using fine-tuned BERT and Rhetorical Structure Theory (RST) discourse relations. It achieves significant improvements over state-of-the-art models in relation prediction by leveraging contextual and discourse-aware features, especially through windowed context clipping and RST-informed supervision, resulting in higher F-scores on intra- and inter-turn relations in CMV subreddit threads.

ABSTRACT

Argumentation is a type of discourse where speakers try to persuade their audience about the reasonableness of a claim by presenting supportive arguments. Most work in argument mining has focused on modeling arguments in monologues. We propose a computational model for argument mining in online persuasive discussion forums that brings together the micro-level (argument as product) and macro-level (argument as process) models of argumentation. Fundamentally, this approach relies on identifying relations between components of arguments in a discussion thread. Our approach for relation prediction uses contextual information in terms of fine-tuning a pre-trained language model and leveraging discourse relations based on Rhetorical Structure Theory. We additionally propose a candidate selection method to automatically predict what parts of one's argument will be targeted by other participants in the discussion. Our models obtain significant improvements compared to recent state-of-the-art approaches using pointer networks and a pre-trained language model.

Motivation & Objective

• To address the gap in argument mining for online dialogues by integrating micro-level argument components and macro-level dialogic relations.
• To improve relation prediction in persuasive discussion forums beyond monologue-focused models.
• To leverage distant supervision and transfer learning to enhance performance on low-resource argument mining tasks.
• To investigate the impact of discourse structure (via RST) and contextual windowing on relation prediction accuracy.
• To reduce the search space for relation prediction while improving precision without sacrificing recall.

Proposed method

• Fine-tune a pre-trained BERT model on a combination of manually annotated CMV subreddit data and two novel distantly-labeled datasets (IMHO+context and QR) to improve contextual understanding.
• Use Rhetorical Structure Theory (RST) to extract discourse relations (e.g., Evaluation, Antithesis) that signal argumentative relations, enhancing relation prediction.
• Apply windowed context clipping to limit the search space for relation prediction, improving precision by focusing on relevant neighboring components.
• Train a joint model to predict both argument components (claim, premise) and relations (support/attack) using a pipeline approach with BERT and RST features.
• Leverage discourse cues from RST parsers as additional features to complement BERT embeddings, improving generalization on unseen data.
• Use TF-IDF similarity to retrieve and analyze failure cases from external data, validating the model’s ability to learn discriminative discourse patterns.

Experimental results

Research questions

• RQ1 Can fine-tuning BERT on distantly-labeled data from similar but non-identical domains improve argument relation prediction in low-resource settings?
• RQ2 How do Rhetorical Structure Theory (RST) discourse relations contribute to distinguishing between support and attack relations in online discussions?
• RQ3 To what extent does context windowing improve precision in relation prediction without significantly reducing recall?
• RQ4 Can discourse-aware features from RST parsers complement BERT embeddings to outperform models relying solely on contextual representations?
• RQ5 How effective is the proposed method in capturing both intra-turn and inter-turn argumentative relations in real-world online debate threads?

Key findings

• The proposed model achieves a 39.3 F-score on intra-turn relation prediction with a window of +1, significantly outperforming prior state-of-the-art models.
• Using a window of +2 improves precision to 32.5% and F-score to 39.3%, demonstrating that context clipping enhances precision in high-recall scenarios.
• Incorporating RST discourse relations leads to a measurable improvement in relation prediction, especially for attack relations like Antithesis.
• The model trained on IMHO+context data correctly identifies argumentative relations in unseen examples, showing robustness to domain shift.
• The combination of BERT fine-tuning and RST features results in a 35.6 F-score for intra-turn relations with a window of +3, outperforming models without discourse supervision.
• The method reduces false positives by narrowing the context window, with the highest F-score (44.1) achieved at a +1 window, indicating optimal balance between recall and precision.