[Paper Review] Discretized Integrated Gradients for Explaining Language Models

Soumya Sanyal, Xiang Ren|arXiv (Cornell University)|Aug 31, 2021

Explainable Artificial Intelligence (XAI)26 references4 citations

TL;DR

This paper proposes Discretized Integrated Gradients (DIG), a method that improves attribution explainability in NLP by replacing linear interpolation paths in Integrated Gradients with non-linear, discrete paths in word embedding space. DIG uses two strategies—Greedy and MaxCount—to select anchor words close to real vocabulary, ensuring interpolated points are representative of actual data, resulting in more faithful gradients and superior performance on sentiment classification tasks across BERT, DistilBERT, and RoBERTa models.

ABSTRACT

As a prominent attribution-based explanation algorithm, Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. It measures feature importance by averaging the model's output gradient interpolated along a straight-line path in the input data space. However, such straight-line interpolated points are not representative of text data due to the inherent discreteness of the word embedding space. This questions the faithfulness of the gradients computed at the interpolated points and consequently, the quality of the generated explanations. Here we propose Discretized Integrated Gradients (DIG), which allows effective attribution along non-linear interpolation paths. We develop two interpolation strategies for the discrete word embedding space that generates interpolation points that lie close to actual words in the embedding space, yielding more faithful gradient computation. We demonstrate the effectiveness of DIG over IG through experimental and human evaluations on multiple sentiment classification datasets. We provide the source code of DIG to encourage reproducible research.

Motivation & Objective

• Address the limitation of Integrated Gradients (IG) in discrete text data due to non-representative, out-of-distribution interpolated points along straight-line paths.
• Improve the faithfulness of gradient-based attribution by ensuring interpolated points in the embedding space are close to actual words in the vocabulary.
• Develop two interpolation strategies—DIG-Greedy and DIG-MaxCount—that monotonically interpolate between input and baseline embeddings using real-word anchors.
• Demonstrate that DIG yields more plausible and accurate explanations than IG and other gradient-based baselines on multiple pre-trained language models and sentiment datasets.
• Enhance human trust in model predictions through human evaluation, showing DIG-generated explanations are perceived as more justifiable.

Proposed method

• Propose a non-linear interpolation path in the discrete word embedding space, replacing the straight-line path used in standard Integrated Gradients.
• Introduce DIG-Greedy, which selects the nearest real word to the monotonic projection of each input word as an anchor, minimizing distance to the monotonic path.
• Introduce DIG-MaxCount, which selects the word with the highest number of monotonic dimensions (i.e., dimensions aligned with the input-to-baseline direction) as the anchor, then modifies non-monotonic dimensions to ensure monotonicity.
• Construct interpolation paths using selected anchor words such that each point lies monotonically between the input and baseline embeddings, improving representativeness.
• Apply up-sampling to the path with factor f to increase point density and reduce integral approximation error, without significantly increasing computational cost.
• Use the standard Integrated Gradients formula but compute gradients only at discrete, data-representative points along the non-linear path, enhancing gradient fidelity.

Experimental results

Research questions

• RQ1 Can non-linear, discrete interpolation paths in word embedding space yield more faithful gradients than linear interpolation in text data?
• RQ2 How do DIG-Greedy and DIG-MaxCount compare to standard Integrated Gradients in terms of attribution quality on sentiment classification tasks?
• RQ3 To what extent do DIG-generated explanations improve human perception of model justification compared to IG and other baselines?
• RQ4 How do hyperparameters m (number of anchors) and f (up-sampling factor) affect the approximation error and performance of DIG?
• RQ5 Does up-sampling the path in DIG effectively reduce integral approximation error without increasing computational cost?

Key findings

• DIG outperforms Integrated Gradients and other gradient-based baselines on eight out of nine experimental settings across three pre-trained language models (BERT, DistilBERT, RoBERTa) and three sentiment datasets (SST2, IMDB, Rotten Tomatoes).
• The Delta % error for DIG decreases consistently with increasing up-sampling factor f, indicating improved integral approximation, while the WAE remains stable.
• For a fixed up-sampling factor f=0, increasing the number of anchors m leads to higher Delta % in DIG, indicating that longer paths increase approximation difficulty.
• Human evaluation shows that explanations generated by DIG are perceived as more plausible and trustworthy by end-users compared to those from IG and other baselines.
• DIG- MaxCount achieves superior performance on average across metrics, particularly in minimizing approximation error and improving faithfulness.
• The up-sampling strategy effectively reduces approximation error without significantly increasing WAE, validating its role as a scalable alternative to increasing m.