[Paper Review] Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees

Johanna Vielhaben, Stefan Blücher+1 more|arXiv (Cornell University)|Jan 27, 2023

Rough Sets and Fuzzy Logic12 citations

TL;DR

The paper introduces Multi-dimensional Concept Discovery (MCD), a framework that discovers multi-dimensional concepts in hidden model representations using sparse subspace clustering, assigns complete, globally interpretable decompositions of model reasoning, and provides concept activation maps and relevance heatmaps for faithful XAI explanations.

ABSTRACT

The completeness axiom renders the explanation of a post-hoc XAI method only locally faithful to the model, i.e. for a single decision. For the trustworthy application of XAI, in particular for high-stake decisions, a more global model understanding is required. Recently, concept-based methods have been proposed, which are however not guaranteed to be bound to the actual model reasoning. To circumvent this problem, we propose Multi-dimensional Concept Discovery (MCD) as an extension of previous approaches that fulfills a completeness relation on the level of concepts. Our method starts from general linear subspaces as concepts and does neither require reinforcing concept interpretability nor re-training of model parts. We propose sparse subspace clustering to discover improved concepts and fully leverage the potential of multi-dimensional subspaces. MCD offers two complementary analysis tools for concepts in input space: (1) concept activation maps, that show where a concept is expressed within a sample, allowing for concept characterization through prototypical samples, and (2) concept relevance heatmaps, that decompose the model decision into concept contributions. Both tools together enable a detailed understanding of the model reasoning, which is guaranteed to relate to the model via a completeness relation. This paves the way towards more trustworthy concept-based XAI. We empirically demonstrate the superiority of MCD against more constrained concept definitions.

Motivation & Objective

• Motivate global model understanding for high-stakes decisions beyond local attributions.
• Propose a general, faithful concept definition that uses multi-dimensional linear subspaces (D4) and a completeness relation.
• Develop a practical discovery pipeline using sparse subspace clustering and PCA to identify concept subspaces.
• Provide tools (activation maps and relevance heatmaps) to analyze concepts in input space while preserving completeness.
• Demonstrate superiority of MCD over constrained concept definitions on ImageNet models.

Proposed method

• Define a concept as a d^l-dimensional linear subspace C^l in the F-dimensional hidden feature space.
• Use Sparse Subspace Clustering (SSC) to cluster feature vectors into subspaces without relying on proximity or preset dimensions.
• Construct concept bases for each cluster via PCA, determining intrinsic subspace dimension d^l (heuristic by Fukunaga–Olsen).
• Decompose activations of new samples into concept contributions via a complete basis including an orthogonal complement C^⊥.
• Compute concept activation maps to localize concept expression in input space; derive concept relevance heatmaps by projecting pooled features onto concept bases.
• Define a global concept relevance score η to quantify how much the concepts explain the final prediction (completeness).

Experimental results

Research questions

• RQ1 Can multi-dimensional subspace concepts faithfully decompose a model's reasoning with a completeness guarantee?
• RQ2 Do multi-dimensional concepts enable more concise global explanations than one-dimensional/orthogonal concepts?
• RQ3 How can we visualize and quantify concept influence at both local (per-sample) and global levels?
• RQ4 What is the impact of different concept discovery variants (MCD-SSC, MCD-kmeans, ICE/MCD-PCA, MCD-SSC-orth) on completeness and faithfulness?
• RQ5 Are concept activation maps and relevance heatmaps applicable across architectures, including vision transformers, and compatible with linear final layers?

Key findings

• MCD achieves a completeness decomposition where the final prediction can be fully reconstructed from concept relevances.
• Multi-dimensional concepts typically provide more concise explanations (fewer concepts needed to reach a given completeness) than constrained definitions (D1-D3).
• Concept activation maps localize where a concept is expressed in input space and can be paired with prototypes for interpretation.
• Concept relevance heatmaps decompose class logits into concept contributions, preserving a global completeness relation.
• The framework is demonstrated on ImageNet models (ResNet-50, ResNet-50v2, Swin-T) and supports various concept discovery variants.
• Orthogonal subspaces (MCD-SSC-orth) were found to be less faithful to the model than arbitrarily rotated subspaces.