[Paper Review] Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens

Nitish Joshi, Xiang Pan+1 more|arXiv (Cornell University)|Oct 25, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces a causal framework using probability of necessity (PN) and sufficiency (PS) to distinguish between two types of spurious features in NLP: irrelevant features (low PN, low PS) and necessary features (high PN, low PS). It shows that existing debiasing methods fail on necessary spurious features because they are inherently required for prediction, leading to performance drops when removed, and calls for tailored evaluation and training approaches for such features.

ABSTRACT

The term `spurious correlations' has been used in NLP to informally denote any undesirable feature-label correlations. However, a correlation can be undesirable because (i) the feature is irrelevant to the label (e.g. punctuation in a review), or (ii) the feature's effect on the label depends on the context (e.g. negation words in a review), which is ubiquitous in language tasks. In case (i), we want the model to be invariant to the feature, which is neither necessary nor sufficient for prediction. But in case (ii), even an ideal model (e.g. humans) must rely on the feature, since it is necessary (but not sufficient) for prediction. Therefore, a more fine-grained treatment of spurious features is needed to specify the desired model behavior. We formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates the causal relations between a feature and a label. We then show that this distinction helps explain results of existing debiasing methods on different spurious features, and demystifies surprising results such as the encoding of spurious features in model representations after debiasing.

Motivation & Objective

• To address the lack of fine-grained distinction between different types of spurious features in NLP, particularly those that are necessary for prediction.
• To formalize the distinction between irrelevant spurious features (low PN, low PS) and necessary spurious features (high PN, low PS) using causal concepts.
• To demonstrate that current debiasing methods—especially data balancing and representation debiasing—fail on necessary spurious features due to their inherent dependency on the label.
• To challenge the assumption that all spurious features can be treated uniformly in model robustness evaluation and training.
• To advocate for new training and evaluation paradigms tailored to necessary spurious features, especially in light of LLMs' potential to handle rare feature compositions.

Proposed method

• Formalize spurious features using causal models, defining the probability of necessity (PN) and probability of sufficiency (PS) as counterfactual measures.
• Use PN and PS to categorize spurious features: low PN/PS for irrelevant features (e.g., director names), high PN/low PS for necessary features (e.g., negation words).
• Propose that necessary spurious features cannot be safely removed or decoupled from labels via data balancing or representation debiasing without harming performance.
• Argue that evaluating robustness to necessary features requires context-aware perturbations, as they interact with other features to influence labels.
• Use structural causal models (SCMs) to represent input features, labels, and confounders, with counterfactual interventions to estimate PN and PS.
• Highlight that enforcing independence (e.g., via data balancing) or invariance (via representation debiasing) fails for high-PN features because they are causally necessary in the data-generating process.

Experimental results

Research questions

• RQ1 Are all spurious features in NLP equally problematic, or do they differ in their causal relationship to the label?
• RQ2 How can we formally distinguish between spurious features that are irrelevant (low PN, low PS) and those that are necessary (high PN, low PS) for prediction?
• RQ3 Why do standard debiasing methods like data balancing and representation debiasing fail to improve robustness on necessary spurious features?
• RQ4 How does the interaction between necessary spurious features and contextual features complicate the evaluation of model robustness?
• RQ5 What training and evaluation strategies are suitable for necessary spurious features, given their causal necessity?

Key findings

• Necessary spurious features, such as negation words, have high probability of necessity (PN) but low probability of sufficiency (PS), meaning they are crucial for prediction but not sufficient alone.
• Data balancing methods fail to achieve invariant performance on necessary spurious features because the feature-label dependence is causal, not correlational.
• Representation debiasing methods significantly hurt model performance when applied to necessary spurious features, as they remove features that are essential for correct prediction.
• The distinction between irrelevant and necessary spurious features explains why some debiasing methods appear to succeed in removing spurious features from representations while still maintaining performance.
• Existing notions like counterfactual invariance and uniform input features (UIF) do not apply to high-PN features, as they are causally dependent on the label in the true data-generating process.
• Large language models may offer a path forward for handling rare or unseen compositions of necessary spurious features due to their strong in-context learning and few-shot generalization capabilities.