[Paper Review] Explanation-Based Tuning of Opaque Machine Learners with Application to Paper Recommendation

Benjamin Charles Germain Lee, Kyle Lo+2 more|arXiv (Cornell University)|Mar 9, 2020

Explainable Artificial Intelligence (XAI)28 references4 citations

TL;DR

This paper introduces LIMEADE, a framework that enables users to tune opaque machine learning models—like neural recommenders—based on local surrogate explanations (e.g., LIME). Applied to Semantic Sanity, a scientific paper recommender, LIMEADE significantly improved user control, trust, and satisfaction in a user study, bridging the gap between explanation and interactive model tuning for black-box models.

ABSTRACT

Research in human-centered AI has shown the benefits of machine-learning systems that can explain their predictions. Methods that allow users to tune a model in response to the explanations are similarly useful. While both capabilities are well-developed for transparent learning models (e.g., linear models and GA2Ms), and recent techniques (e.g., LIME and SHAP) can generate explanations for opaque models, no method currently exists for tuning of opaque models in response to explanations. This paper introduces LIMEADE, a general framework for tuning an arbitrary machine learning model based on an explanation of the model's prediction. We apply our framework to Semantic Sanity, a neural recommender system for scientific papers, and report on a detailed user study, showing that our framework leads to significantly higher perceived user control, trust, and satisfaction.

Motivation & Objective

• To address the lack of interactive tuning capabilities for opaque (black-box) machine learning models despite advances in explanation methods.
• To develop a general-purpose framework that allows users to adjust model behavior based on local explanations of predictions.
• To evaluate whether explanation-based tuning improves user perception of control, trust, and satisfaction in real-world recommendation systems.
• To apply and validate the framework in a practical setting—scientific paper recommendation—using a user study.

Proposed method

• LIMEADE integrates local surrogate explanations (e.g., LIME) to interpret predictions from an opaque model, such as a neural network.
• It exposes salient features from the explanation to users, allowing them to modify feature weights or values directly.
• User-specified changes are mapped back to the original model’s input space, and the model’s prediction is re-evaluated to reflect the tuned behavior.
• The framework supports iterative tuning, enabling users to refine predictions through successive explanation-guided adjustments.
• The method is general and applicable to any model-agnostic explanation technique, including LIME and SHAP.

Experimental results

Research questions

• RQ1 Can users effectively tune an opaque machine learning model using explanation-based feedback?
• RQ2 Does explanation-based tuning improve users’ perceived control over model predictions?
• RQ3 Does interactive tuning based on explanations enhance user trust in the model’s recommendations?
• RQ4 How does user satisfaction with the tuning process compare to non-tunable or non-explanatory systems?

Key findings

• Users reported significantly higher perceived control when using LIMEADE compared to non-tunable baselines.
• Trust in the model’s recommendations increased substantially after users engaged in explanation-based tuning.
• User satisfaction with the recommendation system was significantly higher when tuning was supported via explanations.
• The framework successfully enabled users to make intuitive, behaviorally meaningful adjustments to a complex neural recommender system.