[Paper Review] Learning the Difference that Makes a Difference with Counterfactually-Augmented Data

Divyansh Kaushik, Eduard Hovy+1 more|arXiv (Cornell University)|Sep 26, 2019

Anomaly Detection Techniques and ApplicationsComputer Science33 references232 citations

TL;DR

The authors create counterfactually edited sentiment and NLI data via crowd workers to disentangle spurious from genuine signals, showing that models trained on a mix of original and counterfactually revised data become more robust to spurious cues.

ABSTRACT

Despite alarm over the reliance of machine learning systems on so-called spurious patterns, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are due to confounding (e.g., a common cause), but not direct or indirect causal effects. In this paper, we focus on natural language processing, introducing methods and resources for training models less sensitive to spurious patterns. Given documents and their initial labels, we task humans with revising each document so that it (i) accords with a counterfactual target label; (ii) retains internal coherence; and (iii) avoids unnecessary changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are less sensitive to this signal. Both datasets are publicly available.

Motivation & Objective

• Motivate a causality-inspired view of spurious associations in NLP and distinguish spurious from non-spurious signals.
• Introduce a human-in-the-loop data collection process that edits documents to fit counterfactual labels while preserving coherence.
• Extend IMDb sentiment and SNLI datasets with counterfactually revised examples to study model robustness.
• Evaluate how models trained on original, revised, and combined data perform under distribution shift and domain transfer.

Proposed method

• Crowd workers edit texts to make counterfactual labels apply while preserving coherence and minimizing unnecessary changes.
• Collect counterfactually revised sentiment (IMDb) and NLI (SNLI) data extending existing resources.
• Train and evaluate multiple models (SVM, NB, Bi-LSTM, ELMo-LSTM, BERT) on original, revised, and combined datasets.
• Compare in-domain and out-of-domain performance, and analyze changes in reliance on spurious cues.
• Inspect edit patterns and feature importance to identify which cues become non-predictive after augmentation.

Experimental results

Research questions

• RQ1 Can counterfactually edited data separate spurious signals from genuine predictive signals in NLP tasks?
• RQ2 Do models trained on a combination of original and counterfactually revised data generalize better to revised or out-of-domain inputs?
• RQ3 How do different model families (linear, Bi-LSTM, ELMo, BERT) respond to counterfactually augmented data in sentiment analysis and NLI?
• RQ4 What are the qualitative edit patterns humans use to alter labels, and how do these affect learned features?

Key findings

• Models trained on revised data perform well on revised data but poorly on original data, and vice versa.
• Combined training data yield strong performance on both original and revised data, often within ~3 points of the corresponding single-domain models.
• Spurious cues (e.g., genre mentions in sentiment) lose predictive power when revised data are included in training.
• BERT shows greater resilience to performance drops from counterfactual revisions than some other models, though gains from combining data are still evident.
• SNLI-based experiments show that hypothesis-only or premise-only signals are fragile; fine-tuning on RP and RH variants requires looking at both premise and hypothesis for robust performance.
• Out-of-domain tests (Amazon, Twitter, Yelp) show counterfactually augmented IMDb-trained models generally perform better than models trained on original data alone.