[Paper Review] Learning to Receive Help: Intervention-Aware Concept Embedding Models

Mateo Espinosa Zarlenga, Katherine M. Collins+4 more|arXiv (Cornell University)|Sep 29, 2023

Explainable Artificial Intelligence (XAI)Computer Science3 citations

TL;DR

This paper introduces Intervention-aware Concept Embedding Models (IntCEMs), a novel concept-interpretable neural network architecture that learns an end-to-end intervention policy to improve receptiveness to test-time concept interventions. By training with a regularized objective that simulates intervention trajectories, IntCEMs significantly outperform state-of-the-art models in accuracy gains when expert feedback is applied, while maintaining competitive performance without interventions.

ABSTRACT

Concept Bottleneck Models (CBMs) tackle the opacity of neural architectures by constructing and explaining their predictions using a set of high-level concepts. A special property of these models is that they permit concept interventions, wherein users can correct mispredicted concepts and thus improve the model's performance. Recent work, however, has shown that intervention efficacy can be highly dependent on the order in which concepts are intervened on and on the model's architecture and training hyperparameters. We argue that this is rooted in a CBM's lack of train-time incentives for the model to be appropriately receptive to concept interventions. To address this, we propose Intervention-aware Concept Embedding models (IntCEMs), a novel CBM-based architecture and training paradigm that improves a model's receptiveness to test-time interventions. Our model learns a concept intervention policy in an end-to-end fashion from where it can sample meaningful intervention trajectories at train-time. This conditions IntCEMs to effectively select and receive concept interventions when deployed at test-time. Our experiments show that IntCEMs significantly outperform state-of-the-art concept-interpretable models when provided with test-time concept interventions, demonstrating the effectiveness of our approach.

Motivation & Objective

• To address the lack of train-time incentives for Concept Bottleneck Models (CBMs) to be receptive to test-time concept interventions.
• To improve model performance under expert feedback by explicitly training for receptiveness to interventions.
• To develop a differentiable, end-to-end framework that learns an optimal intervention policy during training.
• To maintain strong performance without interventions while significantly improving accuracy when interventions are applied.

Proposed method

• IntCEMs are based on Concept Embedding Models (CEMs), which represent concepts as high-dimensional embeddings, enabling differentiable manipulation.
• The model learns an intervention policy end-to-end using a policy gradient approach, selecting which concepts to intervene on to maximize expected performance.
• A novel regularizer in the loss function penalizes the model for poor performance after simulated intervention sequences drawn from the learned policy.
• Interventions are modeled as differentiable operations, allowing backpropagation through intervention trajectories during training.
• The architecture jointly optimizes for prediction accuracy and receptiveness to expert feedback via a dual-objective training regime.

Experimental results

Research questions

• RQ1 Can a model be trained to be more receptive to test-time concept interventions through explicit training incentives?
• RQ2 How does learning an intervention policy during training affect performance gains when expert feedback is applied?
• RQ3 Does the proposed method outperform existing concept-interpretable models under intervention scenarios?
• RQ4 Can the model maintain strong performance without interventions while being highly receptive when feedback is provided?

Key findings

• IntCEMs achieve state-of-the-art performance under test-time interventions, with a 4.5% average accuracy gain on MNIST-Add-Incomp compared to baselines.
• On CUB-Incomp, IntCEMs achieve 91.47% accuracy with 100% intervention rate, outperforming the next best method by 3.1 percentage points.
• In the 50% intervention setting on CelebA, IntCEMs reach 62.01% accuracy with the learned policy, a 10.5% improvement over random intervention.
• Even without interventions, IntCEMs maintain competitive performance, with 99.51% accuracy on MNIST-Add, matching the best baseline.
• The model's learned intervention policy consistently outperforms random and heuristic policies (e.g., CooP) across all datasets and intervention rates.