[Paper Review] Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR

Sandra Wachter, Brent Mittelstadt+1 more|arXiv (Cornell University)|Nov 1, 2017

Privacy-Preserving Technologies in DataComputer Science162 citations

TL;DR

The paper argues that data controllers can provide useful counterfactual explanations to affected individuals under the GDPR without exposing the full internal workings of algorithmic decision systems, focusing on actionable guidance for contesting or achieving desired outcomes.

ABSTRACT

There has been much discussion of the right to explanation in the EU General Data Protection Regulation, and its existence, merits, and disadvantages. Implementing a right to explanation that opens the black box of algorithmic decision-making faces major legal and technical barriers. Explaining the functionality of complex algorithmic decision-making systems and their rationale in specific cases is a technically challenging problem. Some explanations may offer little meaningful information to data subjects, raising questions around their value. Explanations of automated decisions need not hinge on the general public understanding how algorithmic systems function. Even though such interpretability is of great importance and should be pursued, explanations can, in principle, be offered without opening the black box. Looking at explanations as a means to help a data subject act rather than merely understand, one could gauge the scope and content of explanations according to the specific goal or action they are intended to support. From the perspective of individuals affected by automated decision-making, we propose three aims for explanations: (1) to inform and help the individual understand why a particular decision was reached, (2) to provide grounds to contest the decision if the outcome is undesired, and (3) to understand what would need to change in order to receive a desired result in the future, based on the current decision-making model. We assess how each of these goals finds support in the GDPR. We suggest data controllers should offer a particular type of explanation, unconditional counterfactual explanations, to support these three aims. These counterfactual explanations describe the smallest change to the world that can be made to obtain a desirable outcome, or to arrive at the closest possible world, without needing to explain the internal logic of the system.

Motivation & Objective

• Motivate the need for explainability under the GDPR and identify barriers to opening the black box of algorithmic decisions.
• Propose unconditional counterfactual explanations as a practical form of explanation.
• Define three aims for explanations from an individual’s perspective: understanding, contesting, and changing outcomes.

Proposed method

• Analyze how explanations can aid data subjects without revealing internal model details.
• Argue for unconditional counterfactual explanations describing minimal world changes to achieve a desired outcome.
• Map explanation goals to GDPR provisions and assess legal support for each goal.

Experimental results

Research questions

• RQ1 Can counterfactual explanations inform data subjects about why a decision was made without disclosing internal model logic?
• RQ2 Do counterfactual explanations support the rights to contest and to obtain a more desirable outcome under the GDPR?
• RQ3 What type of explanations best align with the GDPR to empower individuals while preserving proprietary systems?

Key findings

• Counterfactual explanations can inform individuals without exposing the black box.
• Unconditional counterfactual explanations support understanding, contestability, and recourse to obtain desired outcomes.
• Such explanations align with GDPR aims by focusing on minimal changes needed rather than internal mechanisms.