[Paper Review] Initial Risk Probing and Feasibility Testing of Glow: a Generative AI-Powered Dialectical Behavior Therapy Skills Coach for Substance Use Recovery and HIV Prevention

Liying Wang, Madison Lee+7 more|arXiv (Cornell University)|Feb 8, 2026

Digital Mental Health Interventions0 citations

TL;DR

The paper evaluates Glow, a GenAI-powered DBT skills coach for HIV risk reduction and substance use recovery, using user-driven adversarial testing to assess safety across 37 risk probes. It identifies vulnerabilities and misinformation and discusses mitigation needs before clinical trials.

ABSTRACT

Background: HIV and substance use represent interacting epidemics with shared psychological drivers - impulsivity and maladaptive coping. Dialectical behavior therapy (DBT) targets these mechanisms but faces scalability challenges. Generative artificial intelligence (GenAI) offers potential for delivering personalized DBT coaching at scale, yet rapid development has outpaced safety infrastructure. Methods: We developed Glow, a GenAI-powered DBT skills coach delivering chain and solution analysis for individuals at risk for HIV and substance use. In partnership with a Los Angeles community health organization, we conducted usability testing with clinical staff (n=6) and individuals with lived experience (n=28). Using the Helpful, Honest, and Harmless (HHH) framework, we employed user-driven adversarial testing wherein participants identified target behaviors and generated contextually realistic risk probes. We evaluated safety performance across 37 risk probe interactions. Results: Glow appropriately handled 73% of risk probes, but performance varied by agent. The solution analysis agent demonstrated 90% appropriate handling versus 44% for the chain analysis agent. Safety failures clustered around encouraging substance use and normalizing harmful behaviors. The chain analysis agent fell into an "empathy trap," providing validation that reinforced maladaptive beliefs. Additionally, 27 instances of DBT skill misinformation were identified. Conclusions: This study provides the first systematic safety evaluation of GenAI-delivered DBT coaching for HIV and substance use risk reduction. Findings reveal vulnerabilities requiring mitigation before clinical trials. The HHH framework and user-driven adversarial testing offer replicable methods for evaluating GenAI mental health interventions.

Motivation & Objective

• Motivate scalable, personalized DBT coaching for HIV prevention and substance use recovery using Generative AI.
• Address safety and reliability concerns in GenAI-delivered mental health interventions.
• Provide a framework for systematic safety testing in collaboration with community partners.
• Identify concrete safety vulnerabilities and misinformation risks to inform mitigation before trials.

Proposed method

• Develop Glow, a GenAI-powered DBT skills coach delivering chain and solution analysis.
• Partner with a Los Angeles community health organization for usability testing with clinicians and people with lived experience.
• Apply Helpful, Honest, and Harmless (HHH) framework and user-driven adversarial testing to elicit contextually realistic risk probes.
• Evaluate safety performance across 37 risk probe interactions.
• Compare performance between agents: solution analysis vs chain analysis.
• Document safety failures and misinformation to guide mitigations.

Experimental results

Research questions

• RQ1 Can Glow safely handle contextually realistic risk probes in a DBT-based coaching framework for HIV prevention and substance use recovery?
• RQ2 What safety vulnerabilities and misinformation risks emerge in GenAI-delivered DBT coaching, and how do different agents perform on risk probes?
• RQ3 What methodological framework best supports systematic safety evaluation of GenAI mental health interventions?
• RQ4 What mitigation steps are needed before proceeding to clinical trials?
• RQ5 How does stakeholder collaboration influence usability and safety outcomes?

Key findings

• Glow appropriately handled 73% of risk probes overall, with varying performance by agent.
• The solution analysis agent achieved 90% appropriate handling compared with 44% for the chain analysis agent.
• Safety failures clustered around encouraging substance use and normalizing harmful behaviors.
• The chain analysis agent exhibited an 'empathy trap' by providing validation that reinforced maladaptive beliefs.
• 27 instances of DBT skill misinformation were identified during testing.