[Paper Review] Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems

Shalmali Joshi, Oluwasanmi Koyejo+3 more|arXiv (Cornell University)|Jul 22, 2019

Scientific Computing and Data Management18 references95 citations

TL;DR

Introduces REVISE, a gradient-based method that uses data manifolds (via generative models) to provide realistic, minimal recourse for individuals facing undesirable outcomes in both classification and causal decision-making systems, including handling immutable attributes and diagnosing model bias.

ABSTRACT

Machine learning based decision making systems are increasingly affecting humans. An individual can suffer an undesirable outcome under such decision making systems (e.g. denied credit) irrespective of whether the decision is fair or accurate. Individual recourse pertains to the problem of providing an actionable set of changes a person can undertake in order to improve their outcome. We propose a recourse algorithm that models the underlying data distribution or manifold. We then provide a mechanism to generate the smallest set of changes that will improve an individual's outcome. This mechanism can be easily used to provide recourse for any differentiable machine learning based decision making system. Further, the resulting algorithm is shown to be applicable to both supervised classification and causal decision making systems. Our work attempts to fill gaps in existing fairness literature that have primarily focused on discovering and/or algorithmically enforcing fairness constraints on decision making systems. This work also provides an alternative approach to generating counterfactual explanations.

Motivation & Objective

• Motivate the need for actionable, realistic recourse for individuals affected by ML decisions.
• Propose a framework that models the data distribution to generate minimal, achievable changes along the data manifold.
• Extend recourse generation from pure classification to causal decision-making models.
• Handle immutable variables by conditioning on them and adapting the framework to conditional causal models.
• Provide diagnostic capabilities to reveal biases and confounding in trained decision-making systems.

Proposed method

• Model the data distribution with generative models (e.g., VAEs, HI-VAE, BiGAN-style encoders) to obtain a latent manifold of realistic samples.
• Define recourse as finding a nearby point on the data manifold that flips the decision from undesired to desired, optimizing a regularized objective combining prediction loss and manifold closeness (Equation 1 / Equation 2).
• Use gradient-based updates in the latent space to minimize the objective while crossing the decision boundary, then map back to the input space to obtain recourse attributes.
• Introduce a tunable regularization weight lambda to balance closeness to the original instance and achieving the desired outcome ( Equation 2 ).
• Extend the approach to causal decision-making by replacing the loss with causal-surrogate objectives (Equation 3) and by traversing latent space that corresponds to hidden confounders; discuss do-interventions and recourse under different treatments.
• Address immutable variables by conditioning on them and proposing conditional causal models (e.g., conditional CEVAE) to ensure suggested changes respect immutability.

Experimental results

Research questions

• RQ1 Can recourse be generated along the data manifold to ensure realism and feasibility for individuals?
• RQ2 Can the REVISE framework provide actionable recourse for both standard classification and causal decision-making systems?
• RQ3 How do immutable attributes affect the feasibility and quality of recourse, and how can models accommodate them?
• RQ4 How does the presence of confounding or biased data influence the quality and nature of generated recourse?
• RQ5 Can recourse also serve as a diagnostic tool to reveal biases in black-box decision systems?

Key findings

• The proposed REVISE algorithm can produce recourse that traverses the data manifold to cross the decision boundary while staying realistic to the observed data distribution.
• Recourse is demonstrated for both supervised classification and causal models, highlighting the framework’s generality beyond linear classifiers.
• Handling immutable variables via conditional causal models allows recourse without changing immutable attributes, enabling fairer recommendations.
• Experiments on real datasets show qualitative and diagnostic evidence that data bias and hidden confounding influence recourse quality and sparsity.
• Recourse can reveal bias in black-box models by showing how recourse paths interact with biased attributes (e.g., gender or hair color in gender-recognition examples).
• The framework provides a diagnostic tool to compare classifiers learned under biased vs. unbiased data and to study the impact of confounding on recourse.