[Paper Review] Feature Importance and Explainability in Quantum Machine Learning

Luke Power, Krishnendu Guha|arXiv (Cornell University)|May 14, 2024

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

This paper proposes a comprehensive comparison of feature importance and explainability methods—LOO, permutation importance, ALE, and SHAP—between classical machine learning models (SVM, Random Forest) and quantum machine learning models (VQC, QSVC) on the Iris dataset using Qiskit. It demonstrates that while QML models exhibit distinct feature dependency patterns—notably high sensitivity to petal width in VQC—explainability techniques are transferable to QML, though quantum-specific methods may be needed for future scalability and reliability.

ABSTRACT

Many Machine Learning (ML) models are referred to as black box models, providing no real insights into why a prediction is made. Feature importance and explainability are important for increasing transparency and trust in ML models, particularly in settings such as healthcare and finance. With quantum computing's unique capabilities, such as leveraging quantum mechanical phenomena like superposition, which can be combined with ML techniques to create the field of Quantum Machine Learning (QML), and such techniques may be applied to QML models. This article explores feature importance and explainability insights in QML compared to Classical ML models. Utilizing the widely recognized Iris dataset, classical ML algorithms such as SVM and Random Forests, are compared against hybrid quantum counterparts, implemented via IBM's Qiskit platform: the Variational Quantum Classifier (VQC) and Quantum Support Vector Classifier (QSVC). This article aims to provide a comparison of the insights generated in ML by employing permutation and leave one out feature importance methods, alongside ALE (Accumulated Local Effects) and SHAP (SHapley Additive exPlanations) explainers.

Motivation & Objective

• To investigate how feature importance and explainability techniques apply to quantum machine learning (QML) models compared to classical ML models.
• To evaluate whether QML models exhibit different feature dependencies than classical models, particularly in terms of feature relevance and model robustness.
• To assess the feasibility and effectiveness of applying classical explainability methods—such as SHAP, ALE, and permutation importance—to QML models.
• To identify limitations in current explainability tools when applied to quantum models and highlight the need for quantum-native explainability techniques.
• To lay a foundation for explainable quantum machine learning (XQAI) by demonstrating that existing methods can be adapted to QML, despite hardware and scale constraints.

Proposed method

• Implemented classical ML models (SVM, Random Forest) and hybrid quantum models (VQC, QSVC) on the Iris dataset using IBM's Qiskit platform.
• Applied leave-one-out (LOO) feature importance by systematically removing each feature and measuring model accuracy changes.
• Used permutation importance to assess feature impact by shuffling feature values and observing performance degradation.
• Employed Accumulated Local Effects (ALE) to visualize how feature changes influence model predictions, with limitations due to VQC implementation constraints.
• Applied SHAP (SHapley Additive exPlanations) to interpret individual predictions and quantify feature contributions in both classical and quantum models.
• Compared results across models to identify differences in feature prioritization, sensitivity, and decision boundaries.

Experimental results

Research questions

• RQ1 How do different feature importance methods (LOO, permutation, ALE) compare in their ability to identify key features in QML models versus classical ML models?
• RQ2 What differences in feature dependency patterns emerge between classical models (SVM, Random Forest) and quantum models (VQC, QSVC) on the same dataset?
• RQ3 To what extent can classical explainability techniques like SHAP and ALE be effectively applied to quantum machine learning models?
• RQ4 What insights do explainability methods reveal about the decision-making processes of QML models, particularly in distinguishing between classes like Versicolor and Virginica?
• RQ5 What are the limitations of applying classical explainability tools to QML, and what implications does this have for developing quantum-native explainability methods?

Key findings

• The VQC model showed high sensitivity to the removal of petal width, suffering significant accuracy drops, indicating strong dependency on this feature, unlike classical models.
• Classical models (SVM and Random Forest) exhibited increased accuracy when petal features were removed, suggesting redundancy or overfitting, while QML models showed more consistent feature reliance.
• Permutation importance confirmed the dominance of petal features in classical models and revealed subtle differences in sepal feature impact in QSVC compared to classical counterparts.
• ALE analysis showed that QSVC followed classical model trends but displayed higher sensitivity to feature changes, with no flat lines in ALE plots, indicating strong feature influence across all feature values.
• SHAP analysis revealed that QSVC correctly classified a challenging test instance (point 4) as Virginica, while other models failed, highlighting its superior local explainability and feature interaction modeling.
• The VQC model could not be analyzed using ALE due to implementation constraints, underscoring the technical limitations of applying certain explainability methods to specific QML architectures.