[Paper Review] Efficient computation of counterfactual explanations of LVQ models

André Artelt, Barbara Hammer|arXiv (Cornell University)|Aug 2, 2019

Explainable Artificial Intelligence (XAI)35 references9 citations

TL;DR

This paper proposes a novel, efficient framework for computing counterfactual explanations in Learning Vector Quantization (LVQ) models by leveraging their prototype-based structure. It formulates convex and non-convex optimization problems based on the underlying metric (local or global), enabling faster computation than model-agnostic methods while supporting domain constraints, with empirical results showing significant speedups and reduced feature perturbations.

ABSTRACT

The increasing use of machine learning in practice and legal regulations like EU's GDPR cause the necessity to be able to explain the prediction and behavior of machine learning models. A prominent example of particularly intuitive explanations of AI models in the context of decision making are counterfactual explanations. Yet, it is still an open research problem how to efficiently compute counterfactual explanations for many models. We investigate how to efficiently compute counterfactual explanations for an important class of models, prototype-based classifiers such as learning vector quantization models. In particular, we derive specific convex and non-convex programs depending on the used metric.

Motivation & Objective

• To address the lack of efficient counterfactual explanation methods for prototype-based models like LVQ, which are increasingly used in real-world decision-making.
• To develop model-specific optimization frameworks that exploit LVQ’s structural properties for faster and more feasible counterfactual generation.
• To incorporate domain knowledge as constraints into counterfactual generation, improving plausibility and feasibility.
• To demonstrate that model-specific methods outperform general-purpose, model-agnostic approaches in speed and required input changes.
• To provide a scalable, differentiable framework for generating counterfactuals that are both minimal and actionable.

Proposed method

• The method formulates counterfactual generation as a convex or non-convex quadratically constrained quadratic program (QCQP), depending on the metric used (local or global).
• For global metrics, the decision boundary is represented as a linear inequality: $\vec{x}^{\top}\vec{q}_{ij} + r_{ij} < 0$, derived from the Mahalanobis distance with a shared $\mathbf{\Omega}$ matrix.
• For local metrics, the decision boundary involves a quadratic form $\frac{1}{2}\vec{x}^{\top}\mathbf{Q}_{ij}\vec{x} + \vec{x}^{\top}\vec{q}_{ij} + r_{ij} + \epsilon \leq 0$, where $\mathbf{Q}_{ij} = \mathbf{\Omega}_i - \mathbf{\Omega}_j$.
• The non-convex QCQP is solved approximately using the penalty convex-concave procedure (CCP), linearizing the non-convex term $g(\vec{x}') = \frac{1}{2}\vec{x}'^{\top}\mathbf{\Omega}_j\vec{x}'$ at each iteration.
• The regularization function $\theta(\cdot)$, such as $\ell_1$ or $\ell_2$, is used to control the magnitude of input perturbations, promoting minimal changes.
• Domain constraints are integrated by adding inequality constraints to the optimization problem, enabling generation of plausible and feasible counterfactuals.

Experimental results

Research questions

• RQ1 Can we efficiently compute counterfactual explanations for LVQ models by exploiting their prototype-based structure?
• RQ2 How do convex and non-convex optimization formulations compare in terms of speed and perturbation size for LVQ counterfactuals?
• RQ3 To what extent can domain-specific constraints be integrated into the counterfactual generation process for improved plausibility?
• RQ4 How does the proposed method compare to model-agnostic counterfactual methods in terms of computational efficiency and required feature changes?
• RQ5 Can the penalty CCP method reliably approximate the non-convex QCQP formulation for LVQ models?

Key findings

• The proposed method achieves significant speedups over model-agnostic counterfactual methods, particularly for large-scale LVQ models.
• The framework reduces the required perturbation magnitude (i.e., feature changes) compared to standard approaches, resulting in more actionable counterfactuals.
• The use of local metrics with non-convex QCQP formulations enables more accurate decision boundary modeling, improving counterfactual quality.
• The penalty CCP approach provides a stable and efficient approximation for the non-convex QCQP, converging to feasible solutions with minimal computational overhead.
• Incorporating domain constraints via additional inequalities results in more plausible counterfactuals without sacrificing computational efficiency.
• Empirical evaluation confirms that the method scales well and produces counterfactuals that are both minimal and semantically meaningful.