[Paper Review] The Best Decisions Are Not the Best Advice: Making Adherence-Aware Recommendations

Julien Grand-Clément, Jean Pauphilet|arXiv (Cornell University)|Sep 5, 2022

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper proposes an adherence-aware optimization framework for expert-in-the-loop decision systems, where human decision-makers partially deviate from algorithmic recommendations. By modeling the human's switching behavior between a baseline policy and algorithmic suggestions, the framework computes recommendation policies that are robust to such deviations and provably outperform both the baseline and standard algorithmic policies, even under arbitrary adherence levels.

ABSTRACT

Many high-stake decisions follow an expert-in-loop structure in that a human operator receives recommendations from an algorithm but is the ultimate decision maker. Hence, the algorithm's recommendation may differ from the actual decision implemented in practice. However, most algorithmic recommendations are obtained by solving an optimization problem that assumes recommendations will be perfectly implemented. We propose an adherence-aware optimization framework to capture the dichotomy between the recommended and the implemented policy and analyze the impact of partial adherence on the optimal recommendation. We show that overlooking the partial adherence phenomenon, as is currently being done by most recommendation engines, can lead to arbitrarily severe performance deterioration, compared with both the current human baseline performance and what is expected by the recommendation algorithm. Our framework also provides useful tools to analyze the structure and to compute optimal recommendation policies that are naturally immune against such human deviations, and are guaranteed to improve upon the baseline policy.

Motivation & Objective

• To address the gap in algorithmic recommendation systems that assume perfect adherence, while real-world experts often deviate from recommendations.
• To model and quantify the impact of partial adherence on system performance in high-stake sequential decisions.
• To design a recommendation policy that is inherently robust to human deviations and guarantees improved performance over both the baseline and standard algorithmic policies.
• To provide a tractable, structured, and flexible optimization framework that can be efficiently computed and adapted to various adherence patterns.

Proposed method

• Formulates a novel adherence-aware Markov Decision Process (AdaMDP) that models the decision maker's switching behavior between a baseline policy and algorithmic recommendations.
• Introduces a surrogate MDP formulation where rewards and transition probabilities depend on the baseline policy and adherence level θ.
• Uses a rectangularity assumption on the set of feasible baseline policies to ensure existence and computability of optimal recommendations.
• Reduces the adherence-aware optimization problem to a classical robust MDP problem, enabling efficient computation via known algorithms.
• Analyzes structural properties such as stationarity, determinism, and monotonicity of the optimal policy with respect to adherence level.
• Extends the framework to handle state-dependent, action-dependent, uncertain, and partially known baseline policies.

Experimental results

Research questions

• RQ1 How does partial adherence by human decision-makers affect the performance of algorithmic recommendations in expert-in-the-loop systems?
• RQ2 Can we design recommendation policies that are provably robust to human deviations and outperform both the baseline and standard algorithmic policies?
• RQ3 What structural and computational properties does an adherence-aware recommendation policy possess under different adherence patterns?
• RQ4 Under what conditions can the human decision-maker safely ignore the issue of partial adherence in certain states?
• RQ5 How can the framework be extended to handle time-varying or uncertain adherence levels?

Key findings

• The optimal recommendation policy under partial adherence is always stationary and deterministic, ensuring practical implementability.
• The framework guarantees that the optimal recommendation policy will outperform both the baseline policy and the standard algorithmic policy, regardless of the adherence level.
• Even rare human deviations from recommendations can lead to arbitrarily poor performance when ignored in algorithm design, highlighting the risk of non-adherence.
• The optimal return is monotonic with respect to the adherence level, meaning higher adherence always leads to better or equal performance.
• The optimization problem can be reformulated as a robust MDP under mild assumptions, enabling efficient computation when the baseline policy set is convex and compact.
• The framework identifies states where partial adherence has no impact on recommendations, enabling policy simplification in those contexts.