[Paper Review] Adversarial Filters of Dataset Biases

Ronan Le Bras, Swabha Swayamdipta+5 more|arXiv (Cornell University)|Feb 10, 2020

Adversarial Robustness in Machine LearningComputer Science125 citations

TL;DR

This paper analyzes AFLite, a model-based, iterative filtering method that removes dataset biases to reduce spurious correlations, improving out-of-distribution generalization while substantially lowering in-distribution performance on major benchmarks.

ABSTRACT

Large neural models have demonstrated human-level performance on language and vision benchmarks, while their performance degrades considerably on adversarial or out-of-distribution samples. This raises the question of whether these models have learned to solve a dataset rather than the underlying task by overfitting to spurious dataset biases. We investigate one recently proposed approach, AFLite, which adversarially filters such dataset biases, as a means to mitigate the prevalent overestimation of machine performance. We provide a theoretical understanding for AFLite, by situating it in the generalized framework for optimum bias reduction. We present extensive supporting evidence that AFLite is broadly applicable for reduction of measurable dataset biases, and that models trained on the filtered datasets yield better generalization to out-of-distribution tasks. Finally, filtering results in a large drop in model performance (e.g., from 92% to 62% for SNLI), while human performance still remains high. Our work thus shows that such filtered datasets can pose new research challenges for robust generalization by serving as upgraded benchmarks.

Motivation & Objective

• Motivate the problem of dataset biases causing overestimation of model performance.
• Provide a theoretical framework and practical approximation for optimum bias reduction.
• Empirically validate AFLite across NLP and vision tasks.
• Demonstrate how filtering biases affects in-distribution versus out-of-distribution performance.

Proposed method

• Define a formal representation-bias objective and AFOpt as the ideal but intractable bias-reduction target.
• Introduce AFLite as a scalable approximation that iteratively removes highly predictable instances using a predictability score p(i).
• Compute p(i) via out-of-sample predictions from models trained on random partitions of the data.
• Use a greedy slicing procedure that removes top-k highest-p(i) instances per iteration until size n or early-stop τ is reached.
• Apply pre-computed feature representations Φ(X) and a model family M to estimate predictability.
• Demonstrate the approach on NLP and vision benchmarks including SNLI, MultiNLI, QNLI, and ImageNet.

Experimental results

Research questions

• RQ1 Can AFLite reliably remove dataset biases beyond explicitly known artifacts?
• RQ2 Do models trained on AFLite-filtered data generalize better to out-of-distribution tasks?
• RQ3 How does AFLite affect in-distribution benchmark performance across NLP and vision datasets?
• RQ4 Is AFLite robust across different feature representations and model families?

Key findings

• AFLite reduces detectable dataset biases and makes benchmarks harder for models while leaving human performance relatively high.
• Models trained on AFLite-filtered data show improved generalization on out-of-distribution tasks such as HANS, NLI Diagnostics, Stress tests, and Adversarial NLI.
• On SNLI, AFLite-filtered data dramatically lowers in-domain model accuracy (e.g., from high-90s to around 60s for strong models), while human performance remains high.
• In NLP, AFLite lowers in-domain accuracy across RoBERTa, BERT, and ESIM+GloVe baselines, indicating the removal of biased yet easy examples.
• In vision, training on AFLite-filtered ImageNet data yields up to about a 2% absolute gain on adversarial Out-of-Distribution sets, despite large in-domain drops on standard validation.