[Paper Review] Learning from others' mistakes: Avoiding dataset biases without modeling them

Victor Sanh, Thomas Wolf+2 more|arXiv (Cornell University)|Dec 2, 2020

Topic Modeling48 references51 citations

TL;DR

The paper trains a robust main model by leveraging a weak, biased teacher (small model) in a product-of-experts framework, allowing the main model to learn from the teacher’s errors without explicitly modeling biases, improving out-of-distribution robustness across NLP tasks.

ABSTRACT

State-of-the-art natural language processing (NLP) models often learn to model dataset biases and surface form correlations instead of features that target the intended underlying task. Previous work has demonstrated effective methods to circumvent these issues when knowledge of the bias is available. We consider cases where the bias issues may not be explicitly identified, and show a method for training models that learn to ignore these problematic correlations. Our approach relies on the observation that models with limited capacity primarily learn to exploit biases in the dataset. We can leverage the errors of such limited capacity models to train a more robust model in a product of experts, thus bypassing the need to hand-craft a biased model. We show the effectiveness of this method to retain improvements in out-of-distribution settings even if no particular bias is targeted by the biased model.

Motivation & Objective

• Show that weak learners naturally capture dataset biases without explicit bias design.
• Demonstrate that a product-of-experts (PoE) training regime can reduce reliance on biases and improve OOD robustness.
• Explore trade-offs between weak learner capacity, in-distribution accuracy, and out-of-distribution generalization.
• Validate the approach on NLI (MNLI/HANS) and QA (SQuAD) benchmarks and synthetic bias scenarios.

Proposed method

• Train a weak learner f_W with standard cross-entropy on the data.
• Freeze f_W and train a main model f_M using a product-of-experts (PoE) combination: e = w + m, where softmax(e) ∝ softmax(w) ⊙ softmax(m).
• Optimize a multi-loss objective that combines CE and PoE CE to balance learning from weak learner mistakes and standard prediction.
• Vary the capacity of the weak learner to study the effect on in-distribution vs out-of-distribution performance.
• Evaluate robustness on adversarial/debiased datasets (e.g., MNLI-HANS, Adversarial SQuAD) and analyze bias-reduction correlations.
• Discuss the role of weak-learner certainty and correctness in shaping main-model gradients.

Experimental results

Research questions

• RQ1 Can a weak, biased model reveal dataset biases without explicit bias specification?
• RQ2 Does training a main model via PoE with a frozen weak learner improve generalization to out-of-distribution data without severely harming in-distribution performance?
• RQ3 How does weak-learner capacity affect the bias mitigation and robustness trade-off across tasks?
• RQ4 Is it possible to reduce reliance on known bias signals (or unknown biases) without hand-crafting biased models?

Key findings

• Weak learners tend to rely on shallow heuristics and high-certainty biased errors that mirror known dataset biases.
• PoE training reduces model reliance on the weak learner’s biases and improves out-of-distribution performance, with some trade-off in in-distribution accuracy.
• On MNLI/HANS, PoE-based main models achieve notable gains in handling heuristic-non-entailed cases while maintaining competitive MNLI performance.
• On Adversarial SQuAD benchmarks, PoE (with or without CE) improves adversarial robustness, sometimes more when using multi-loss optimization.
• Varying weak-learner capacity shows that smaller weak learners favor more balanced OD/ID performance, while larger weak learners can boost OD robustness at the expense of ID accuracy.