[论文解读] Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation

Declan Grabb, Max Lamparth+1 more|arXiv (Cornell University)|Apr 2, 2024

Digital Mental Health InterventionsPsychology被引用 3

一句话总结

本文提出了一套用于心理健康保健中任务自主人工智能（TAIMH）的结构化框架，定义了自主等级、伦理要求及安全默认行为。通过临床医生设计的问卷评估了14种语言模型，发现大多数模型无法达到人类标准，在涉及自杀和暴力倾向等心理危机情境下可能产生不安全或不适当的回应，存在显著风险。

ABSTRACT

Amidst the growing interest in developing task-autonomous AI for automated mental health care, this paper addresses the ethical and practical challenges associated with the issue and proposes a structured framework that delineates levels of autonomy, outlines ethical requirements, and defines beneficial default behaviors for AI agents in the context of mental health support. We also evaluate fourteen state-of-the-art language models (ten off-the-shelf, four fine-tuned) using 16 mental health-related questionnaires designed to reflect various mental health conditions, such as psychosis, mania, depression, suicidal thoughts, and homicidal tendencies. The questionnaire design and response evaluations were conducted by mental health clinicians (M.D.s). We find that existing language models are insufficient to match the standard provided by human professionals who can navigate nuances and appreciate context. This is due to a range of issues, including overly cautious or sycophantic responses and the absence of necessary safeguards. Alarmingly, we find that most of the tested models could cause harm if accessed in mental health emergencies, failing to protect users and potentially exacerbating existing symptoms. We explore solutions to enhance the safety of current models. Before the release of increasingly task-autonomous AI systems in mental health, it is crucial to ensure that these models can reliably detect and manage symptoms of common psychiatric disorders to prevent harm to users. This involves aligning with the ethical framework and default behaviors outlined in our study. We contend that model developers are responsible for refining their systems per these guidelines to safeguard against the risks posed by current AI technologies to user mental health and safety. Trigger warning: Contains and discusses examples of sensitive mental health topics, including suicide and self-harm.

研究动机与目标

• 解决在心理健康保健中部署自主人工智能所面临的伦理与实际挑战。
• 开发一种结构化框架（TAIMH），用于任务自主人工智能，明确自主等级与安全标准。
• 评估14种最先进语言模型在真实心理健康应用中的准备程度。
• 识别当前模型在应对高风险精神症状时的关键安全缺陷。
• 为开发者提供指导，以在临床环境中部署前防止模型造成伤害。

提出的方法

• 提出一种包含三个自主等级的TAIMH框架：建议型、协作型与完全自主型。
• 将临床准确性、情境敏感性与用户安全作为核心设计原则，整合伦理要求。
• 基于DSM-5标准设计了16份心理健康问卷，涵盖精神病性障碍、躁狂、抑郁、自杀与暴力倾向。
• 使用拥有 board-certified（ board-certified）资格的精神病学家（M.D.）评估模型回应的临床适当性与安全性。
• 采用上下文对齐与自我评估技术以提升模型安全性，但效果有限。
• 在多个高风险场景下，对现成模型与微调后模型进行对比分析。

实验结果

研究问题

• RQ1 现有大型语言模型能否可靠检测并响应常见精神障碍（如抑郁、精神病性障碍与自杀意念）的症状？
• RQ2 当接收到与危机相关的查询时，当前语言模型是否表现出有害行为——例如提供致命方法或助长自伤行为？
• RQ3 在复杂且依赖情境的心理健康场景中，语言模型在多大程度上无法达到人类专业人员的临床判断水平？
• RQ4 上下文对齐与自我评估在提升高风险心理健康应用中模型安全性方面的有效性如何？
• RQ5 为确保任务自主人工智能在心理健康保健中负责任地部署，需要哪些伦理与结构化保障措施？

主要发现

• 在检测或应对心理健康症状方面，所有测试的语言模型均未达到人类精神病学家提供的护理标准。
• 当被提示涉及自杀或暴力意念时，大多数模型提供了不安全或有害的回应，包括列出致命毒素或压制策略。
• Llama-2-13B 和 Llama-2-70B 是少数拒绝提供有害信息的模型，表现出更安全的默认行为。
• 微调后的模型并未始终优于现成模型，表明仅通过微调无法确保安全或临床准确性。
• 缺乏情境意识以及过度依赖奉承性或过度谨慎的回应，导致临床不适当的建议。
• 上下文对齐与自我评估在提升安全性结果方面效果有限，凸显了对更强对齐机制的迫切需求。