[Paper Review] BIASeD: Bringing Irrationality into Automated System Design

Aditya Gulati, Miguel Ángel Lozano+2 more|arXiv (Cornell University)|Oct 1, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This theoretical paper proposes integrating human cognitive biases into AI system design to enhance human-AI collaboration, introducing a five-stage taxonomy of biases across perception, interpretation, and decision-making. It advocates for AI systems that model, understand, and adapt to irrational human behaviors—moving beyond bias mitigation toward constructive, personalized collaboration.

ABSTRACT

Human perception, memory and decision-making are impacted by tens of cognitive biases and heuristics that influence our actions and decisions. Despite the pervasiveness of such biases, they are generally not leveraged by today's Artificial Intelligence (AI) systems that model human behavior and interact with humans. In this theoretical paper, we claim that the future of human-machine collaboration will entail the development of AI systems that model, understand and possibly replicate human cognitive biases. We propose the need for a research agenda on the interplay between human cognitive biases and Artificial Intelligence. We categorize existing cognitive biases from the perspective of AI systems, identify three broad areas of interest and outline research directions for the design of AI systems that have a better understanding of our own biases.

Motivation & Objective

• To address the lack of integration of human cognitive biases in current AI systems despite their pervasive influence on human decision-making.
• To propose a new taxonomy of cognitive biases categorized by stages in the human decision-making cycle: presentation, interpretation, value attribution, recall, and decision.
• To identify 20 key cognitive biases suitable for AI system design and outline three research directions for bias-aware AI.
• To shift from manipulating biases for commercial gain to leveraging them constructively in AI for improved trust, interpretability, and collaboration.
• To establish a research agenda for modeling, understanding, and mitigating cognitive biases in AI-human interaction

Proposed method

• Proposes a five-stage taxonomy of cognitive biases based on the human decision-making cycle: presentation, interpretation, value attribution, recall, and decision.
• Classifies 20 representative cognitive biases across these five stages, such as anchoring (presentation), confirmation bias (interpretation), and availability bias (recall).
• Introduces three research directions: (1) human-AI interaction with bias-aware design, (2) embedding bias mechanisms into AI algorithms for robustness, and (3) computational modeling of biases for personalized mitigation.
• Draws on existing frameworks like Bayesian modeling and generative models to inform computational modeling of biases.
• Proposes using AI to observe and model human behavior to detect biases automatically and suggest personalized interventions.
• Leverages insights from persuasive computing and behavioral science to design adaptive, user-specific bias mitigation strategies.

Experimental results

Research questions

• RQ1 How can AI systems be designed to recognize and adapt to human cognitive biases in real-time human-AI interactions?
• RQ2 To what extent do human-AI interactions exhibit the same cognitive biases as human-to-human interactions?
• RQ3 Can mechanisms underlying human cognitive biases be intentionally embedded into AI algorithms to improve robustness and efficiency?
• RQ4 What unifying computational framework can model cognitive biases across diverse tasks and individuals for personalized mitigation?
• RQ5 How can AI systems support users in reducing bias without relying solely on awareness, using personalized, persuasive strategies?

Key findings

• Cognitive biases are systematically present across all stages of human perception, interpretation, and decision-making, and are not merely errors but fundamental to human cognition.
• Current AI systems largely ignore human cognitive biases despite their critical role in shaping human behavior and decision-making.
• A five-stage taxonomy of cognitive biases—presentation, interpretation, value attribution, recall, and decision—provides a structured framework for AI system design.
• The paper identifies 20 key cognitive biases (e.g., anchoring, confirmation bias, availability bias) as prime candidates for integration into AI systems.
• Three research directions are proposed: (1) bias-aware human-AI interaction, (2) leveraging bias mechanisms in AI algorithms, and (3) computational modeling for personalized mitigation.
• Existing approaches like Bayesian modeling and generative models offer promising foundations for modeling cognitive biases computationally, but no unified, task-independent framework yet exists.