[Paper Review] Rethinking Positive Aggregation and Propagation of Gradients in Gradient-based Saliency Methods

Ashkan Khakzar, Soroosh Baselizadeh+1 more|arXiv (Cornell University)|Dec 1, 2020

Explainable Artificial Intelligence (XAI)21 references4 citations

TL;DR

This paper demonstrates that positive aggregation and propagation of gradients in saliency methods—commonly used in GradCAM++, FullGrad, LRP, and guided backpropagation—render these methods insensitive to model predictions and parameter randomization. By replacing ReLU-based positive filtering with full gradient handling, the authors show that saliency maps become class-sensitive and model-sensitive, revealing that prior methods primarily recover visual saliency rather than explain model behavior.

ABSTRACT

Saliency methods interpret the prediction of a neural network by showing the importance of input elements for that prediction. A popular family of saliency methods utilize gradient information. In this work, we empirically show that two approaches for handling the gradient information, namely positive aggregation, and positive propagation, break these methods. Though these methods reflect visually salient information in the input, they do not explain the model prediction anymore as the generated saliency maps are insensitive to the predicted output and are insensitive to model parameter randomization. Specifically for methods that aggregate the gradients of a chosen layer such as GradCAM++ and FullGrad, exclusively aggregating positive gradients is detrimental. We further support this by proposing several variants of aggregation methods with positive handling of gradient information. For methods that backpropagate gradient information such as LRP, RectGrad, and Guided Backpropagation, we show the destructive effect of exclusively propagating positive gradient information.

Motivation & Objective

• To investigate why popular gradient-based saliency methods fail to explain model predictions despite producing visually plausible maps.
• To evaluate the impact of positive aggregation and propagation on class-sensitivity and sensitivity to model parameter randomization.
• To propose and test alternative aggregation and propagation methods that preserve gradient sign information.
• To empirically validate that positive-only gradient handling leads to saliency maps that reflect input image features rather than model behavior.
• To provide evidence that widely used saliency methods like GradCAM++, LRP, and Guided Backpropagation are not reliable for model interpretation.

Proposed method

• Proposed variants of saliency methods that replace ReLU-based positive aggregation with absolute value or sign-preserving aggregation to retain gradient sign information.
• Introduced a modified RectGrad method (RectGrad_Mod) that uses absolute values in the propagation rule, allowing both positive and negative gradients to flow backward.
• Used LRP-0 (γ=0) as a counterpart to LRP-γ=∞ to remove bias toward positive contributions in relevance propagation.
• Applied the pointing game and restricted pointing game to evaluate class-sensitivity across different output predictions.
• Conducted sanity checks on model parameter randomization to test sensitivity to model changes.
• Evaluated methods using the class-sensitivity metric from Rebuffi et al. (2020), measuring correlation between saliency maps for different predicted classes.

Experimental results

Research questions

• RQ1 Does positive aggregation of gradients in methods like GradCAM++ and FullGrad compromise their ability to explain model predictions?
• RQ2 To what extent do saliency maps from positive gradient propagation methods remain unchanged after model parameter randomization?
• RQ3 How does the inclusion of negative gradient information affect class-sensitivity in saliency maps?
• RQ4 Are methods like LRP, RectGrad, and Guided Backpropagation truly sensitive to the predicted output class, or do they merely recover visual saliency?
• RQ5 Can alternative aggregation and propagation rules that preserve gradient signs improve the reliability of saliency explanations?

Key findings

• Positive aggregation in GradCAM++ and FullGrad variants leads to saliency maps that are insensitive to the predicted output class, as shown by low class-sensitivity metrics and high pointing game accuracy across all classes.
• Saliency maps from positive propagation methods like Guided Backpropagation, RectGrad, and LRP-γ=∞ remain visually similar even after model parameter randomization, indicating insensitivity to model changes.
• The counterpart methods—such as Gradients, RectGrad_Mod, and LRP-0—show significantly improved sensitivity to model parameter randomization, confirming that positive-only propagation is the root cause of insensitivity.
• Class-sensitivity metrics reveal that positive propagation methods (e.g., Guided Backpropagation, LRP-γ=∞) have Spearman correlation values near 1.0, indicating near-identical saliency maps across different predictions.
• The restricted pointing game shows that positive propagation methods point to the same object class regardless of the predicted class, while counterpart methods show reduced accuracy in this setting, indicating true class-sensitivity.
• The proposed modifications—such as using absolute values in RectGrad and setting γ=0 in LRP—restore sensitivity to both model parameters and predicted classes, demonstrating that gradient sign preservation is essential for reliable attribution.