[Paper Review] Explainable Empirical Risk Minimization

Zhang, L., Karakasidis, G.+3 more|arXiv (Cornell University)|Sep 3, 2020

Explainable Artificial Intelligence (XAI)19 references4 citations

TL;DR

This paper introduces Explainable Empirical Risk Minimization (EERM), a novel framework that learns machine learning hypotheses by balancing predictive risk and subjective explainability through user-specific feedback. By using conditional entropy of predictions given user signals as a regularizer, EERM tailors model interpretability to individual users' cognitive frameworks, achieving improved explainability without sacrificing accuracy, as demonstrated in linear regression and decision tree tasks on weather and hate-speech detection datasets.

ABSTRACT

The successful application of machine learning (ML) methods becomes increasingly dependent on their interpretability or explainability. Designing explainable ML systems is instrumental to ensuring transparency of automated decision-making that targets humans. The explainability of ML methods is also an essential ingredient for trustworthy artificial intelligence. A key challenge in ensuring explainability is its dependence on the specific human user ("explainee"). The users of machine learning methods might have vastly different background knowledge about machine learning principles. One user might have a university degree in machine learning or related fields, while another user might have never received formal training in high-school mathematics. This paper applies information-theoretic concepts to develop a novel measure for the subjective explainability of the predictions delivered by a ML method. We construct this measure via the conditional entropy of predictions, given a user feedback. The user feedback might be obtained from user surveys or biophysical measurements. Our main contribution is the explainable empirical risk minimization (EERM) principle of learning a hypothesis that optimally balances between the subjective explainability and risk. The EERM principle is flexible and can be combined with arbitrary machine learning models. We present several practical implementations of EERM for linear models and decision trees. Numerical experiments demonstrate the application of EERM to detecting the use of inappropriate language on social media.

Motivation & Objective

• To address the challenge of varying user expertise in machine learning explainability by introducing a personalized, user-dependent measure of explainability.
• To develop a flexible learning principle that optimally trades off empirical risk and subjective explainability for individual users.
• To enable the integration of user feedback—such as survey responses or biophysical signals—into the training process to improve model interpretability.
• To demonstrate that incorporating user-specific intuition via feedback signals enhances model trustworthiness and performance in real-world applications.
• To provide a generalizable framework applicable to diverse models, including linear models and decision trees, without requiring model-specific modifications.

Proposed method

• The method uses conditional entropy H(h|u) of model predictions h given a user feedback signal u as a regularizer to quantify subjective explainability.
• The user feedback signal u is collected via surveys, biophysical measurements, or other user-specific indicators and is used to define user-defined similarity between data points.
• EERM formulates a regularized risk minimization problem where the objective function includes both empirical risk and a penalty term based on H(h|u), constrained by an upper bound η.
• For linear models, the method optimizes the weight vector w under the constraint that H(h|u) ≤ η, using Gaussian approximation of the user signal distribution.
• For decision trees, EERM learns a separate tree for each user signal value u, enforcing consistency in predictions across similar user feedbacks by constraining tree depth or structure.
• The framework is implemented via algorithms that integrate user feedback during training, with no need for ongoing user input post-training, enabling scalable deployment.

Experimental results

Research questions

• RQ1 How can machine learning models be made explainable in a way that accounts for the varying background knowledge of different users?
• RQ2 Can a user-specific measure of explainability be constructed using information-theoretic principles to reflect user intuition?
• RQ3 To what extent does incorporating user feedback signals into the learning process improve model interpretability without degrading predictive performance?
• RQ4 How can the trade-off between model risk and subjective explainability be systematically optimized for different types of models?
• RQ5 Can EERM be effectively applied to real-world tasks such as weather prediction and hate-speech detection with measurable improvements in explainability and accuracy?

Key findings

• In the weather prediction task, increasing the upper bound η of conditional entropy led to a monotonic decrease in empirical risk, indicating improved model performance as explainability constraints were relaxed.
• The optimal weight vector w in linear regression was found to shift systematically with changes in η, reflecting the model's adaptation to user-specific explainability requirements.
• For hate-speech detection, the EERM-learned decision tree with η = 2 bits achieved a test accuracy of 0.929, demonstrating strong performance while maintaining user-specific explainability.
• The resulting decision tree explicitly used the user signal u (based on frequent words in offensive tweets) as the root node, showing that the model learned to align predictions with user intuition.
• The method successfully balanced predictive accuracy and subjective explainability, with conditional entropy serving as an effective regularizer that improved model trustworthiness for diverse users.
• Numerical experiments confirmed that EERM can be flexibly applied to both linear models and decision trees, with consistent improvements in explainability and performance across datasets.