[Paper Review] Rethinking Stability for Attribution-based Explanations

Chirag Agarwal, Nari Johnson+5 more|arXiv (Cornell University)|Mar 14, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper introduces Relative Stability metrics to evaluate attribution-based explanation methods by measuring how explanation changes relative to input, representation, or output perturbations of a model. It demonstrates that SmoothGrad produces the most stable explanations across seven methods on three real-world datasets, outperforming others by 12.7% on average in stability.

ABSTRACT

As attribution-based explanation methods are increasingly used to establish model trustworthiness in high-stakes situations, it is critical to ensure that these explanations are stable, e.g., robust to infinitesimal perturbations to an input. However, previous works have shown that state-of-the-art explanation methods generate unstable explanations. Here, we introduce metrics to quantify the stability of an explanation and show that several popular explanation methods are unstable. In particular, we propose new Relative Stability metrics that measure the change in output explanation with respect to change in input, model representation, or output of the underlying predictor. Finally, our experimental evaluation with three real-world datasets demonstrates interesting insights for seven explanation methods and different stability metrics.

Motivation & Objective

• To address the instability of attribution-based explanations in high-stakes ML applications, where small input changes yield large explanation shifts.
• To overcome limitations of prior stability metrics that only consider input perturbations and ignore model internals.
• To develop and validate new stability metrics—Relative Input, Representation, and Output Stability—that incorporate model behavior.
• To empirically compare the stability of seven state-of-the-art explanation methods using multiple datasets and predictive models.
• To establish theoretical bounds linking explanation instability to model Lipschitz constants and representation stability.

Proposed method

• Proposes Relative Stability metrics: Relative Input Stability (RIS), Relative Representation Stability (RRS), and Relative Output Stability (ROS), each measuring explanation change relative to input, representation, or output perturbations.
• Derives theoretical upper bounds for RIS using the product of the model’s input-to-hidden Lipschitz constant and RRS, establishing a formal link between model properties and explanation stability.
• Empirically evaluates stability by generating 50 infinitesimal perturbations per test instance and computing explanation changes across seven explanation methods.
• Uses logistic regression and artificial neural networks as predictive models, with explanations generated via VanillaGrad, Integrated Gradients, SmoothGrad, Input×Gradients, LIME, SHAP, and a random baseline.
• Applies L2 norm differences between original and perturbed explanations normalized by perturbation size to quantify instability.
• Validates theoretical bounds by comparing empirical RIS values with theoretical upper bounds derived from model architecture and representation stability.

Experimental results

Research questions

• RQ1 How do existing stability metrics fail to capture model-level behavior, particularly in relation to internal representations?
• RQ2 To what extent do attribution-based explanation methods remain stable under infinitesimal input, representation, or output perturbations?
• RQ3 Can theoretical bounds be derived that link explanation instability to model-specific properties such as Lipschitz constants?
• RQ4 Which explanation method exhibits the most robust performance across multiple stability variants and datasets?
• RQ5 How do stability metrics differ in sensitivity to input, representation, or output changes, and what insights do they reveal about explanation reliability?

Key findings

• Relative Input Stability (RIS) scores are consistently lower than RRS and ROS, indicating that input perturbations have a stronger normalizing effect on explanation differences.
• Theoretical upper bounds for RIS are empirically validated and found to be tight, with mean theoretical bounds 233% higher than empirical values across methods.
• SmoothGrad achieves the highest stability across all datasets and stability variants, outperforming other methods by an average of 12.7% in median instability.
• Representation and output stability metrics (RRS and ROS) reveal that model internals significantly influence explanation stability, highlighting the need to consider model behavior beyond input perturbations.
• No explanation method is completely stable, but SmoothGrad shows the most consistent robustness across datasets (Adult, Compas, German Credit) and predictive models (LR and ANN).
• Theoretical analysis confirms that explanation instability is bounded by the product of the model’s input-to-hidden Lipschitz constant and representation stability, providing a formal foundation for stability evaluation.