[Paper Review] Evaluating Models' Local Decision Boundaries via Contrast Sets

Matt Gardner, Yoav Artzi+24 more|arXiv (Cornell University)|Apr 6, 2020

Topic Modeling71 references44 citations

TL;DR

The paper introduces contrast sets as expert-crafted local perturbations around test instances to probe models' local decision boundaries, revealing robustness gaps across 10 NLP datasets with notable performance drops on contrast sets.

ABSTRACT

Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities. We propose a new annotation paradigm for NLP that helps to close systematic gaps in the test data. In particular, after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets. Contrast sets provide a local view of a model's decision boundary, which can be used to more accurately evaluate a model's true linguistic capabilities. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets (e.g., DROP reading comprehension, UD parsing, IMDb sentiment analysis). Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets---up to 25\% in some cases. We release our contrast sets as new evaluation benchmarks and encourage future dataset construction efforts to follow similar annotation processes.

Motivation & Objective

• Identify systematic gaps in NLP test data that allow naively trained models to achieve high accuracy.
• Propose an annotation paradigm to create contrast sets around test pivots.
• Assess how well models' local decision boundaries align with the true linguistic phenomena across diverse tasks.

Proposed method

• Define local decision boundary around a pivot x using a distance notion in input space.
• Construct contrast sets C(x) by expert perturbations that typically change the gold label.
• Evaluate models by contrast consistency: whether predictions match labels y' for all (x', y') in C(x).
• Sample pivots from i.i.d. test data and avoid i.i.d. contrast sets at training time to prevent artificial boundaries.
• Apply the process to 10 NLP datasets spanning tasks like reading comprehension, parsing, and sentiment with dataset-specific perturbations.
• Release the resulting contrast sets as benchmarks for future dataset construction and evaluation.

Experimental results

Research questions

• RQ1 How do model predictions change under small, linguistically meaningful perturbations around test instances?
• RQ2 Do contrast sets reveal local decision boundary misalignments not visible in standard i.i.d. test accuracy?
• RQ3 Are there task-specific perturbations that systematically reduce model performance across datasets?

Key findings

• Contrast sets reduce performance relative to original test sets across all 10 datasets, with drops up to 25% in some cases.
• Consistency on contrast sets often requires models to correctly handle local variations that align with the intended dataset phenomena.
• Different tasks and models show varying degrees of fragility, indicating prevalent gaps between dataset design and true linguistic capabilities.
• The authors provide a unified yet dataset-tailored approach to constructing contrast sets, demonstrating general applicability across NLP tasks.