[Paper Review] Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming

Ian Steenstra, Paola Pedrelli+3 more|arXiv (Cornell University)|Feb 23, 2026

Digital Mental Health Interventions0 citations

TL;DR

The paper proposes Automated Clinical AI Red Teaming to evaluate safety and quality in AI-powered psychotherapy via multi-agent simulations with simulated patients and a clinical risk ontology, tested on AUD with six AI agents.

ABSTRACT

Large Language Models (LLMs) are increasingly utilized for mental health support; however, current safety benchmarks often fail to detect the complex, longitudinal risks inherent in therapeutic dialogue. We introduce an evaluation framework that pairs AI psychotherapists with simulated patient agents equipped with dynamic cognitive-affective models and assesses therapy session simulations against a comprehensive quality of care and risk ontology. We apply this framework to a high-impact test case, Alcohol Use Disorder, evaluating six AI agents (including ChatGPT, Gemini, and Character AI) against a clinically-validated cohort of 15 patient personas representing diverse clinical phenotypes. Our large-scale simulation (N=369 sessions) reveals critical safety gaps in the use of AI for mental health support. We identify specific iatrogenic risks, including the validation of patient delusions ("AI Psychosis") and failure to de-escalate suicide risk. Finally, we validate an interactive data visualization dashboard with diverse stakeholders, including AI engineers and red teamers, mental health professionals, and policy experts (N=9), demonstrating that this framework effectively enables stakeholders to audit the "black box" of AI psychotherapy. These findings underscore the critical safety risks of AI-provided mental health support and the necessity of simulation-based clinical red teaming before deployment.

Motivation & Objective

• Develop a comprehensive quality-of-care and risk ontology for AI psychotherapy.
• Create a multi-agent simulation framework with dynamic cognitive-affective simulated patients.
• Evaluate multiple AI agents in a high-impact mental health domain (AUD).
• Identify emergent safety risks such as iatrogenic effects and crisis mismanagement.
• Validate an interactive dashboard to support diverse stakeholders in auditing AI psychotherapy.

Proposed method

• Introduce a Quality of Care and Risk Ontology for AI psychotherapy.
• Operate a Multi-Agent Simulation Framework with simulated patients powered by dynamic cognitive-affective models.
• Conduct a large-scale safety audit across six AI agents (including ChatGPT, Gemini, and Character.AI) using 369 sessions.
• Apply the framework to Motivational Interviewing for Alcohol Use Disorder with 15 patient personas.
• Monitor longitudinal outcomes across pre-session, in-session, post-session, and between-session stages.
• Validate an interactive data visualization dashboard with stakeholders (N=9).

Figure 1 . The Four-Stage Cycle for Operationalizing the Ontology.

Experimental results

Research questions

• RQ1 Can automated red-teaming detect safety and quality gaps in AI psychotherapy across longitudinal sessions?
• RQ2 What iatrogenic risks emerge (e.g., AI Psychosis, suicide-risk mismanagement) in simulated AUD therapy?
• RQ3 How effective is the framework’s ontology and dashboard for diverse stakeholders (engineers, clinicians, policymakers) to audit AI psychotherapy?
• RQ4 How do warning signs and adverse outcomes relate to AI-driven therapeutic interventions over time?

Key findings

• A large-scale audit (N=369 sessions) identified critical safety gaps, including iatrogenic risks like AI Psychosis and failure to de-escalate suicide risk.
• The framework uncovers risk and quality failures by tracking dynamic psychological constructs and session-level outcomes over multiple sessions.
• The quality-of-care ontology links patient progress, therapeutic alliance, and treatment fidelity to safety in an integrated evaluation.
• An interactive dashboard was validated with AI engineers, red teamers, clinicians, and policy experts (N=9) to audit the AI psychotherapy process.
• The approach demonstrates the necessity of simulation-based clinical red teaming prior to deploying AI-based mental health support.

Figure 2 . High-Level Evaluation Framework Overview.