[Paper Review] An Extension of LIME with Improvement of Interpretability and Fidelity

Sheng Shi, Yangzhou Du+1 more|arXiv (Cornell University)|Apr 26, 2020

Explainable Artificial Intelligence (XAI)12 references4 citations

TL;DR

This paper proposes LEDSNA, an extension of LIME that improves local model interpretability and fidelity by incorporating feature dependency through clustering-based sampling and modeling nonlinear decision boundaries using Support Vector Regression (SVR). Experiments show LEDSNA achieves significantly better fidelity (higher R²) and lower approximation error than LIME in both image and text classification tasks.

ABSTRACT

While deep learning makes significant achievements in Artificial Intelligence (AI), the lack of transparency has limited its broad application in various vertical domains. Explainability is not only a gateway between AI and real world, but also a powerful feature to detect flaw of the models and bias of the data. Local Interpretable Model-agnostic Explanation (LIME) is a widely-accepted technique that explains the prediction of any classifier faithfully by learning an interpretable model locally around the predicted instance. As an extension of LIME, this paper proposes an high-interpretability and high-fidelity local explanation method, known as Local Explanation using feature Dependency Sampling and Nonlinear Approximation (LEDSNA). Given an instance being explained, LEDSNA enhances interpretability by feature sampling with intrinsic dependency. Besides, LEDSNA improves the local explanation fidelity by approximating nonlinear boundary of local decision. We evaluate our method with classification tasks in both image domain and text domain. Experiments show that LEDSNA's explanation of the back-box model achieves much better performance than original LIME in terms of interpretability and fidelity.

Motivation & Objective

• Address the limitation of LIME in handling feature correlations during perturbation sampling.
• Improve local explanation fidelity by modeling nonlinear decision boundaries instead of assuming local linearity.
• Enhance interpretability by preserving realistic feature dependencies in sampled instances.
• Develop a general-purpose, model-agnostic explanation method applicable to complex deep learning models in vision and NLP.
• Demonstrate consistent superiority over LIME across diverse classification tasks in terms of fidelity and interpretability metrics.

Proposed method

• Introduces a feature dependency-aware sampling strategy using superpixel clustering for images and word segmentation for text to generate realistic perturbed samples.
• Applies feature clustering to group correlated features and sample only relevant subsets, reducing noise and improving local model quality.
• Uses Support Vector Regression (SVR) with a kernel function to approximate the local nonlinear decision boundary of the black-box model.
• Optimizes the explanation model via a regularized loss function: $ J(\theta) = \text{argmin} \{ L(f(x), g_\theta(x)) + \lambda \Omega(\theta) \} $, balancing fidelity and model simplicity.
• Employs Gaussian kernel in SVR to map data into a higher-dimensional space, enabling effective nonlinear approximation.
• Constructs local explanations by fitting an interpretable model (SVR) on the dependency-aware, perturbed samples around the instance of interest.

Experimental results

Research questions

• RQ1 Can incorporating feature dependencies during sampling improve the quality of local explanations compared to uniform sampling in LIME?
• RQ2 Does modeling nonlinear decision boundaries lead to higher fidelity in local explanations than assuming local linearity?
• RQ3 How does the proposed method perform in comparison to LIME in terms of interpretability and fidelity across different data modalities?
• RQ4 To what extent does the use of clustering-based sampling reduce approximation error in local explanation models?
• RQ5 Can the method maintain high fidelity and interpretability in both image and text classification tasks?

Key findings

• LEDSNA achieves significantly lower local approximation error than LIME, with an average error reduction of one magnitude in text classification.
• In image classification, LEDSNA's R² score is consistently higher than LIME’s, indicating superior fidelity in approximating the black-box model’s predictions.
• For 95% of the 1000 test cases in text classification, LEDSNA’s approximation error was smaller than LIME’s, and for 98.4% of cases, its R² was higher.
• In sentiment analysis on Chinese text, LEDSNA achieved an R² of 0.9533 and error of 0.0001 for a sample with high confidence, outperforming LIME’s R² of 0.5779 and error of 0.0335.
• Visual explanations from LEDSNA provide more useful and accurate feature attribution than LIME, particularly in capturing semantic dependencies in text.
• The method demonstrates robust performance across diverse instances, including high-confidence and low-confidence predictions, confirming its reliability and generalization in real-world scenarios.