[Paper Review] Interpretable Mixture of Experts

Aya Abdelsalam Ismail, Sercan Ö. Arık+4 more|arXiv (Cornell University)|Jun 5, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper introduces Interpretable Mixture of Experts (IME), a novel framework that combines interpretable experts—such as linear models or decision trees—with a learnable assignment module to produce high-accuracy, inherently interpretable predictions for tabular and time-series data. IME achieves state-of-the-art accuracy comparable to deep neural networks while providing faithful, human-understandable explanations that outperform post-hoc methods in user trust and interpretability.

ABSTRACT

The need for reliable model explanations is prominent for many machine learning applications, particularly for tabular and time-series data as their use cases often involve high-stakes decision making. Towards this goal, we introduce a novel interpretable modeling framework, Interpretable Mixture of Experts (IME), that yields high accuracy, comparable to `black-box' Deep Neural Networks (DNNs) in many cases, along with useful interpretability capabilities. IME consists of an assignment module and a mixture of experts, with each sample being assigned to a single expert for prediction. We introduce multiple options for IME based on the assignment and experts being interpretable. When the experts are chosen to be interpretable such as linear models, IME yields an inherently-interpretable architecture where the explanations produced by IME are the exact descriptions of how the prediction is computed. In addition to constituting a standalone inherently-interpretable architecture, IME has the premise of being integrated with existing DNNs to offer interpretability to a subset of samples while maintaining the accuracy of the DNNs. Through extensive experiments on 15 tabular and time-series datasets, IME is demonstrated to be more accurate than single interpretable models and perform comparably with existing state-of-the-art DNNs in accuracy. On most datasets, IME even outperforms DNNs, while providing faithful explanations. Lastly, IME's explanations are compared to commonly-used post-hoc explanations methods through a user study -- participants are able to better predict the model behavior when given IME explanations, while finding IME's explanations more faithful and trustworthy.

Motivation & Objective

• To address the critical need for reliable, faithful, and interpretable model explanations in high-stakes applications involving tabular and time-series data.
• To develop a framework that maintains high predictive accuracy comparable to black-box deep neural networks while providing inherently interpretable predictions.
• To enable both full and partial interpretability by allowing interpretable experts and assignment modules, or integration with DNNs for hybrid interpretability.
• To evaluate whether IME's explanations are more trustworthy and easier to interpret than post-hoc explanation methods like saliency maps.
• To demonstrate generalization across diverse real-world datasets, including both tabular and time-series data types.

Proposed method

• IME uses a mixture of experts architecture where each input sample is assigned to a single expert via a differentiable assignment module, ensuring sparse expert selection.
• The framework supports multiple configurations: S-IME (single-level assignment) and H-IME (hierarchical assignment), with options for interpretable or DNN-based assignment and experts.
• Interpretable experts include linear models, decision trees, or other inherently interpretable models, ensuring that predictions are exactly described by their feature contributions.
• The assignment module can be trained end-to-end using a composite loss function that includes utility, smoothness, and expert assignment regularization to improve generalization and stability.
• IME integrates past prediction errors as input features to the assignment module, enhancing its ability to route samples to the most appropriate expert.
• The framework supports both full interpretability (when both assignment and experts are interpretable) and partial interpretability (when only experts are interpretable), enabling trade-offs between accuracy and interpretability.

Experimental results

Research questions

• RQ1 Can a mixture of interpretable experts achieve predictive accuracy comparable to state-of-the-art deep neural networks on tabular and time-series data?
• RQ2 Does IME produce explanations that are more faithful, trustworthy, and easier to interpret than post-hoc explanation methods like saliency maps?
• RQ3 Can IME generalize across diverse data types, including both tabular and multivariate time-series data, without sacrificing performance?
• RQ4 How does the inclusion of past prediction errors in the assignment module affect model performance and routing accuracy?
• RQ5 What is the optimal number of experts and model capacity for balancing accuracy and interpretability in IME?

Key findings

• IME outperforms single interpretable models like linear regression and decision trees on 15 real-world tabular and time-series datasets, achieving higher accuracy in most cases.
• IME achieves state-of-the-art performance on several benchmarks, even outperforming black-box DNNs such as LSTMs, particularly when using linear models with longer input sequences.
• User studies show that participants better predicted model behavior and found IME's explanations more trustworthy and faithful compared to saliency-based explanations.
• Ablation studies confirm that all components of the loss function—utility, smoothness, and expert assignment regularization—are essential for optimal performance.
• The optimal number of experts is 40 for linear model experts and 20 for soft decision trees, with performance degrading when experts are underutilized due to overcapacity.
• IME maintains high accuracy even when the assignment module is a DNN, while still providing interpretable predictions through the expert components, enabling hybrid interpretability.