[Paper Review] HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy

Mengxi Xiao, Qianqian Xie+6 more|arXiv (Cornell University)|Feb 26, 2024

Mental Health via Writing5 citations

TL;DR

HealMe is an LLM-based psychotherapy model that uses cognitive reframing through empathetic dialogue to foster client self-discovery, and it outperforms baselines in empathy, guidance, and coherence.

ABSTRACT

Large Language Models (LLMs) can play a vital role in psychotherapy by adeptly handling the crucial task of cognitive reframing and overcoming challenges such as shame, distrust, therapist skill variability, and resource scarcity. Previous LLMs in cognitive reframing mainly converted negative emotions to positive ones, but these approaches have limited efficacy, often not promoting clients' self-discovery of alternative perspectives. In this paper, we unveil the Helping and Empowering through Adaptive Language in Mental Enhancement (HealMe) model. This novel cognitive reframing therapy method effectively addresses deep-rooted negative thoughts and fosters rational, balanced perspectives. Diverging from traditional LLM methods, HealMe employs empathetic dialogue based on psychotherapeutic frameworks. It systematically guides clients through distinguishing circumstances from feelings, brainstorming alternative viewpoints, and developing empathetic, actionable suggestions. Moreover, we adopt the first comprehensive and expertly crafted psychological evaluation metrics, specifically designed to rigorously assess the performance of cognitive reframing, in both AI-simulated dialogues and real-world therapeutic conversations. Experimental results show that our model outperforms others in terms of empathy, guidance, and logical coherence, demonstrating its effectiveness and potential positive impact on psychotherapy.

Motivation & Objective

• Motivate and enable AI-driven cognitive reframing therapy to address deep-rooted negative thoughts.
• Emphasize client empowerment and self-discovery over therapist-driven guidance.
• Develop a structured three-step cognitive reframing process grounded in psychotherapeutic literature.
• Create a dataset and expert evaluation framework for AI psychotherapy performance.
• Demonstrate HealMe’s advantages over baselines in AI-to-AI and real-world dialogues.

Proposed method

• Introduce HealMe based on cognitive reframing therapy with three stages: distinguish situations from thoughts, brainstorm alternative viewpoints, and offer empathetic, actionable suggestions.
• Construct training data by prompting AI client and therapist roles using “thinking trap” pairs derived from prior datasets.
• Fine-tune LLaMA2-7b-chat with 3 epochs on a dataset of multi-round dialogues, evaluated with expert metrics.
• Propose a three-dimensional AI psychotherapy evaluation metric (empathy, logical coherence, guidance) and an overall score.
• Evaluate HealMe against ChatGLM3-6b and LLaMA2-7b-chat in AI-to-AI and real-person scenarios using professional psychological metrics.
• Use an offline evaluation pipeline with psychotherapist scoring by human experts and GPT-4 assisted scoring for consistency.

Experimental results

Research questions

• RQ1 Can HealMe consistently maintain high empathy and guidance in cognitive reframing therapy?
• RQ2 Does HealMe outperform open-source baselines in AI-to-AI psychotherapy dialogues and in real-world client interactions?
• RQ3 What evaluation metrics best capture the quality of AI-assisted psychotherapy, and how can they be applied to both AI-to-AI and human scenarios?
• RQ4 To what extent does a structured cognitive reframing process facilitate client self-discovery and constructive action?

Key findings

• HealMe achieves higher empathy, logical coherence, and guidance scores than baselines in AI-to-AI dialogues.
• In AI-to-AI testing, HealMe outperforms ChatGLM3-6b and LLaMA2-7b-chat across empathy, coherence, and guidance.
• In real-client testing, some clients show notable decreases in negative emotional attributes after interacting with HealMe.
• A case study shows HealMe provides targeted brainstorming, emphasizes client effort, and begins from the client’s pain.
• The study introduces a first-of-its-kind psychological evaluation metric suite for AI psychotherapy performance.