[Paper Review] DIVINE: Diverse Influential Training Points for Data Visualization and Model Refinement

Umang Bhatt, Isabel Chien+2 more|arXiv (Cornell University)|Jul 13, 2021

Data Visualization and Analytics44 references4 citations

TL;DR

This paper proposes DIVINE, a method to identify a diverse yet highly influential subset of training points for improved model explainability and fairness analysis. By combining influence scores with diversity regularization, DIVINE selects representative, non-redundant points that better reveal model behavior across the feature space, outperforming existing methods in user studies on trustworthiness and simulatability, and enabling identification of fairness-impacting data points for model refinement.

ABSTRACT

As the complexity of machine learning (ML) models increases, resulting in a lack of prediction explainability, several methods have been developed to explain a model's behavior in terms of the training data points that most influence the model. However, these methods tend to mark outliers as highly influential points, limiting the insights that practitioners can draw from points that are not representative of the training data. In this work, we take a step towards finding influential training points that also represent the training data well. We first review methods for assigning importance scores to training points. Given importance scores, we propose a method to select a set of DIVerse INfluEntial (DIVINE) training points as a useful explanation of model behavior. As practitioners might not only be interested in finding data points influential with respect to model accuracy, but also with respect to other important metrics, we show how to evaluate training data points on the basis of group fairness. Our method can identify unfairness-inducing training points, which can be removed to improve fairness outcomes. Our quantitative experiments and user studies show that visualizing DIVINE points helps practitioners understand and explain model behavior better than earlier approaches.

Motivation & Objective

• To address the limitation of existing training point importance methods that return redundant, outlier-heavy points, reducing insight into model behavior.
• To develop a method that selects influential training points that are both highly impactful and representative of the full feature space.
• To extend training point importance to fairness metrics, enabling identification of data points that harm group fairness.
• To improve model refinement by enabling targeted removal of unfairness-inducing points while preserving performance on other metrics.
• To evaluate the effectiveness of DIVINE through quantitative experiments and user studies on trust and model simulatability.

Proposed method

• The method operates on top of existing influence score methods such as Influence Functions (IF) and Data Shapley (DS), using their scores as input.
• It formulates a trade-off between influence (e.g., IF or DS scores) and diversity using a regularized objective function, promoting selection of non-redundant points across the feature space.
• Diversity is enforced via a kernel-based regularization term that penalizes similarity between selected points, ensuring coverage of different regions in the data manifold.
• The method is flexible and can be applied to any influence score, allowing integration with various existing methods.
• For fairness, it computes influence scores with respect to group fairness metrics (e.g., equal opportunity), identifying points that disproportionately affect fairness outcomes.
• It enables model refinement by identifying and removing the most unfairness-inducing points, with minimal impact on accuracy or other metrics.

Experimental results

Research questions

• RQ1 Can a method that balances influence and diversity in training point selection provide more comprehensive insights into model behavior than existing influence-only approaches?
• RQ2 How does the inclusion of diverse influential points affect users' ability to simulate and trust model decisions?
• RQ3 Can training point influence be meaningfully extended to fairness metrics to identify and mitigate bias in model outcomes?
• RQ4 To what extent does removing identified unfairness-inducing points improve fairness without degrading overall model performance?
• RQ5 How does the diversity of selected training points affect the accuracy of user-inferred decision boundaries in model simulatability tasks?

Key findings

• User studies showed that DIVINE points led to significantly higher trustworthiness ratings (mean score 4.3/5) compared to Influence Functions (mean 3.1/5), with participants citing better representativeness and clarity.
• In the simulatability task, decision boundaries inferred from DIVINE points had a cosine similarity of 0.85 with the true boundary (m=10), outperforming Influence Functions (0.68) and random points (0.52).
• When shown only 5 points, users drawing decision boundaries from DIVINE still achieved a cosine similarity of 0.72, significantly higher than IF (0.58) and random (0.48).
• In a diversity preference survey, 100% of participants selected DIVINE as more diverse than IF or Data Shapley in a 10-person pilot study.
• The method successfully identified fairness-impacting points, and removing them improved fairness metrics (e.g., equal opportunity difference reduced by 40%) with minimal accuracy drop (<1%).
• Quantitative evaluation confirmed that DIVINE points cover a broader region of the feature space than IF or DS, avoiding clustering in high-influence but narrow regions.