[논문 리뷰] Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks

Stefan Blücher, Johanna Vielhaben외 1명|arXiv (Cornell University)|2024. 01. 12.

Explainable Artificial Intelligence (XAI)인용 수 5

한 줄 요약

논문은 XAI에서 차폐(out occlusion) 전략을 평가하기 위한 Reference-out-of-Model-Scope (R-OMS) 점수와 MIF와 LIF를 결합한 대칭 관련 이익 (SRG)을 도입하여 40개 설정에서 일관된 픽셀 플리핑 벤치마크를 도출한다.

ABSTRACT

Feature removal is a central building block for eXplainable AI (XAI), both for occlusion-based explanations (Shapley values) as well as their evaluation (pixel flipping, PF). However, occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. This ambiguity limits the usefulness of occlusion-based approaches. For example, PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF). This study proposes two complementary perspectives to resolve this disagreement problem. Firstly, we address the common criticism of occlusion-based XAI, that artificial samples lead to unreliable model evaluations. We propose to measure the reliability by the R(eference)-Out-of-Model-Scope (OMS) score. The R-OMS score enables a systematic comparison of occlusion strategies and resolves the disagreement problem by grouping consistent PF rankings. Secondly, we show that the insightfulness of MIF and LIF is conversely dependent on the R-OMS score. To leverage this, we combine the MIF and LIF measures into the symmetric relevance gain (SRG) measure. This breaks the inherent connection to the underlying occlusion strategy and leads to consistent rankings. This resolves the disagreement problem, which we verify for a set of 40 different occlusion strategies.

연구 동기 및 목표

• Address the disagreement problem in occlusion-based XAI explanations and their evaluation (pixel flipping).
• Develop a quantitative, model-aware reliability score for occluded samples (R-OMS).
• Analyze how occlusion design choices (imputer, superpixel, model) influence PF benchmarks.
• Propose SRG to combine MIF and LIF into a strategy-agnostic ranking metric.
• Validate the approach across multiple occlusion strategies and model types.

제안 방법

• Define the occlusion evaluation framework using imputers of varying complexity (mean, train-set, histogram, cv2, diffusion).
• Introduce the Reference-out-of-model-scope (R-OMS) score based on the original class prediction for the reference sample.
• Assess occluded samples with different superpixel schemes (rectangular, SLIC, SAM) and across model architectures (ResNet50, timm-ResNet50, ViT).
• Demonstrate that occlusion strategy choice and PF benchmarks are linked, and that R-OMS explains ranking stability.
• Define symmetric relevance gain (SRG) as SRG[φ] = LRG[φ] + MRG[φ] to decouple rankings from occlusion strategy.
• Provide empirical results over 40 PF setups and 100 ImageNet samples to compare XAI methods.

실험 결과

연구 질문

• RQ1 How do occlusion strategy design choices (imputer, superpixel shape/number, model type) affect pixel-flipping (PF) benchmarks and method rankings?
• RQ2 Can the R-OMS score reliably characterize the reliability of occluded samples across XAI methods and occlusion strategies?
• RQ3 Does the SRG measure yield consistent XAI method rankings across diverse occlusion strategies?
• RQ4 What are the key factors driving consistency or disagreement in MIF/LIF-based PF benchmarks?
• RQ5 Do SRG-based rankings remain stable when aggregating over multiple PF setups?

주요 결과

• Diffusion imputers consistently yield higher R-OMS scores, while train-set imputers produce lower R-OMS and can mislead model predictions.
• R-OMS correlates with PF outcomes, and high R-OMS yields more consistent MIF/LIF rankings; NR-OMS is less informative.
• MIF rankings are most consistent for large R-OMS, while LIF show consistency at medium to low R-OMS; SRG combines both for stable rankings.
• SRG rankings remain largely independent of occlusion strategy, reducing the number of distinct rankings from 40 setups to about 7, improving interpretability.
• The number of superpixels is a strong predictor of PF ranking; semantic SAM superpixels increase R-OMS across imputers, reducing imputer sensitivity.
• SRG demonstrates higher quantitative stability (lower variance) than MRG/LRG across PF setups (example: Var(LRP) SRG = 0.0005 vs. 0.0110 and 0.0128 for MRG/LRG).