[Paper Review] How does this interaction affect me? Interpretable attribution for feature interactions

Michael Tsang, Sirisha Rambhatla+1 more|arXiv (Cornell University)|Jun 19, 2020

Explainable Artificial Intelligence (XAI)54 references34 citations

TL;DR

Archipelago presents ArchAttribute (interaction-aware attribution) and ArchDetect (interaction detection) to provide interpretable, axiomatic, and scalable explanations for feature interactions in black-box models. It demonstrates superior interpretability vs. prior methods on sentiment analysis and image classification, with visualizations across text, medical imaging, and ads/recommendations.

ABSTRACT

Machine learning transparency calls for interpretable explanations of how inputs relate to predictions. Feature attribution is a way to analyze the impact of features on predictions. Feature interactions are the contextual dependence between features that jointly impact predictions. There are a number of methods that extract feature interactions in prediction models; however, the methods that assign attributions to interactions are either uninterpretable, model-specific, or non-axiomatic. We propose an interaction attribution and detection framework called Archipelago which addresses these problems and is also scalable in real-world settings. Our experiments on standard annotation labels indicate our approach provides significantly more interpretable explanations than comparable methods, which is important for analyzing the impact of interactions on predictions. We also provide accompanying visualizations of our approach that give new insights into deep neural networks.

Motivation & Objective

• Motivate the need for interpretable explanations of how feature interactions drive predictions.
• Propose ArchAttribute for interaction-aware feature attribution that is model-agnostic and axiomatic.
• Introduce ArchDetect to efficiently identify disjoint interaction sets and higher-order interactions.
• Show empirical evidence that Archipelago yields more interpretable explanations than baselines on sentiment analysis and image classification.
• Provide visualizations illustrating interactions in NLP, medical imaging, and advertising/recommendation scenarios.

Proposed method

• Define statistical non-additive interaction to formalize when features jointly affect a model output.
• Introduce ArchAttribute to attribute prediction changes to sets of interacting features via f(x*_I + x'_-I) - f(x').
• Propose Set Attribution axioms and prove ArchAttribute satisfies completeness and other attribution axioms for feature sets.
• Develop ArchDetect to estimate pairwise interaction strength using a discrete mixed partial derivative interpretation and to merge pairwise interactions into disjoint higher-order interaction sets with O(p^2) efficiency.
• Provide a framework to visualize Archipelago explanations by combining ArchAttribute attributions with ArchDetect-identified interactions across text and image domains.

Experimental results

Research questions

• RQ1 How can we attribute predictions to feature interactions in a model-agnostic and axiomatic way?
• RQ2 Can we efficiently detect and merge interactions of arbitrary order into interpretable, disjoint sets?
• RQ3 Do interaction attributions produced by Archipelago align with ground-truth or annotation labels in sentiment analysis and image classification?
• RQ4 Do visualizations from Archipelago offer clearer insights into model behavior than existing methods?

Key findings

• ArchAttribute yields interpretable, model-agnostic attributions for feature interactions and satisfies an extended set of attribution axioms including a new Set Attribution axiom.
• ArchDetect efficiently detects pairwise interactions in O(p^2) and merges them into disjoint higher-order interaction sets, improving tractability.
• In sentiment analysis and image classification, Archipelago-based explanations correlate more strongly with ground-truth annotations than several baselines (e.g., ArchAttribute achieves high correlation scores on SST and strong segment AUC on ImageNet/COCO-based evaluation).
• Archipelago visualizations reveal both local and long-range interactions in text, and highlight semantically meaningful regions in images (e.g., “great vessels” in chest X-rays) linked to predictions.
• Compared against methods like IG, IH, STI, SI, SCD, and SOC, ArchDetect shows superior detection performance on synthetic tasks and ArchAttribute outperforms baselines on attribution tasks.