[Paper Review] New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

Arushi Gupta, Nikunj Saunshi+3 more|arXiv (Cornell University)|Nov 5, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper introduces a new intrinsic evaluation framework for saliency methods that enforces both completeness and soundness, addressing flaws in existing mask-based evaluations. It proposes a novel saliency method and demonstrates that total variation (TV) regularization enhances soundness by reducing spurious mask artifacts, leading to more reliable explanations in linear and deep networks.

ABSTRACT

Saliency methods compute heat maps that highlight portions of an input that were most {\em important} for the label assigned to it by a deep net. Evaluations of saliency methods convert this heat map into a new {\em masked input} by retaining the $k$ highest-ranked pixels of the original input and replacing the rest with extquotedblleft uninformative extquotedblright\ pixels, and checking if the net's output is mostly unchanged. This is usually seen as an {\em explanation} of the output, but the current paper highlights reasons why this inference of causality may be suspect. Inspired by logic concepts of {\em completeness \& soundness}, it observes that the above type of evaluation focuses on completeness of the explanation, but ignores soundness. New evaluation metrics are introduced to capture both notions, while staying in an {\em intrinsic} framework -- i.e., using the dataset and the net, but no separately trained nets, human evaluations, etc. A simple saliency method is described that matches or outperforms prior methods in the evaluations. Experiments also suggest new intrinsic justifications, based on soundness, for popular heuristic tricks such as TV regularization and upsampling.

Motivation & Objective

• To address the lack of soundness in standard intrinsic evaluations of saliency methods, which only verify completeness.
• To formalize soundness as a necessary condition for valid explanation, ensuring that incorrect labels cannot be justified by saliency masks.
• To develop a new intrinsic evaluation framework that uses only the original model and data, avoiding human annotations or auxiliary networks.
• To provide theoretical and empirical justification for common heuristic practices like TV regularization in saliency methods.
• To propose a new saliency method that achieves state-of-the-art performance under the new evaluation criteria.

Proposed method

• Introduces a formal framework based on logical completeness and soundness to evaluate saliency methods, requiring that explanations only justify correct predictions.
• Defines a masked input via a binary mask M, where only the top-k saliency-scored pixels are retained and others replaced with uninformative (gray) values.
• Proposes a new intrinsic evaluation metric based on AUC over both correct and incorrect labels to assess completeness and soundness simultaneously.
• Introduces a saliency method that selects salient pixels based on the product of input feature values and model weights, optimized under size and TV constraints.
• Applies total variation (TV) regularization to enforce spatial coherence in saliency maps, reducing artifacts that lead to incorrect label justification.
• Uses Chernoff bounds in a linear model setting to theoretically prove that TV-regularized saliency sets are unlikely to falsely justify incorrect labels.

Experimental results

Research questions

• RQ1 Can standard intrinsic evaluations be trusted to validate saliency methods, given that they only verify completeness and ignore soundness?
• RQ2 To what extent do common heuristics like TV regularization improve the soundness of saliency explanations?
• RQ3 Can a saliency method be designed to simultaneously satisfy completeness and soundness in intrinsic evaluations?
• RQ4 Is there a theoretical justification for why TV regularization reduces spurious mask artifacts in saliency maps?
• RQ5 How do existing saliency methods perform under the new completeness and soundness criteria compared to the proposed method?

Key findings

• The proposed saliency method matches or outperforms prior state-of-the-art methods on the new completeness and soundness evaluation metrics.
• Experiments show that TV regularization significantly reduces the AUC for incorrect labels in masked inputs, improving soundness without sacrificing completeness.
• In a linear model setting, TV-regularized saliency sets with size L = Ω(1/γ²) can reliably justify correct predictions with high probability, while failing to justify incorrect ones.
• Theoretical analysis proves that under bounded weights and inputs, the probability of falsely identifying an incorrect label using a TV-constrained saliency set decays exponentially with L.
• The study reveals that existing saliency methods can produce high completeness scores but fail soundness, as they can justify incorrect labels via artifact-prone masks.
• The results suggest that TV regularization is not just a heuristic but a necessary component for sound saliency explanations, especially in high-dimensional inputs.