[论文解读] VITA: A Multi-modal LLM-based System for Longitudinal, Autonomous, and Adaptive Robotic Mental Well-being Coaching

Micol Spitale, Minja Axelsson+1 more|arXiv (Cornell University)|Dec 15, 2023

Digital Mental Health Interventions被引用 6

一句话总结

VITA 是一种多模态、基于大语言模型（LLM）的机器人教练系统，能够自主适应用户面部表情和语音模式，提供个性化的心理健康支持干预。在为期4周的真实世界研究中，共有17名参与者参与，VITA 显著提升了心理健康水平，减少了互动中断，并被用户认为比预设脚本模型更具响应性和同理心。

ABSTRACT

Recently, several works have explored if and how robotic coaches can promote and maintain mental well-being in different settings. However, findings from these studies revealed that these robotic coaches are not ready to be used and deployed in real-world settings due to several limitations that span from technological challenges to coaching success. To overcome these challenges, this paper presents VITA, a novel multi-modal LLM-based system that allows robotic coaches to autonomously adapt to the coachee's multi-modal behaviours (facial valence and speech duration) and deliver coaching exercises in order to promote mental well-being in adults. We identified five objectives that correspond to the challenges in the recent literature, and we show how the VITA system addresses these via experimental validations that include one in-lab pilot study (N=4) that enabled us to test different robotic coach configurations (pre-scripted, generic, and adaptive models) and inform its design for using it in the real world, and one real-world study (N=17) conducted in a workplace over 4 weeks. Our results show that: (i) coachees perceived the VITA adaptive and generic configurations more positively than the pre-scripted one, and they felt understood and heard by the adaptive robotic coach, (ii) the VITA adaptive robotic coach kept learning successfully by personalising to each coachee over time and did not detect any interaction ruptures during the coaching, (iii) coachees had significant mental well-being improvements via the VITA-based robotic coach practice. The code for the VITA system is openly available via: https://github.com/Cambridge-AFAR/VITA-system.

研究动机与目标

• 解决现有机器人教练互动性与响应性不足的问题，避免用户产生负面感知。
• 通过随时间适应个体学员的行为模式，实现长期个性化支持。
• 检测并缓解互动中断，以维持教练关系与会话流畅性。
• 通过集成先进的大语言模型（LLMs），提升对话质量，使其更接近人类教练的行为。
• 通过长期机器人教练干预，实现学员心理健康水平的持续且显著改善。

提出的方法

• 通过实时行为分析，整合多模态输入——包括从视频中提取的面部情绪值（valence）和从音频中提取的语音时长。
• 采用强化学习框架，根据学员反馈和行为动态调整教练策略。
• 嵌入大语言模型（LLM）以生成自然、上下文恰当且富有同理心的言语回应。
• 使用安全过滤层（如 OpenAI Moderation API）对回应内容进行过滤，确保互动的伦理质量。
• 采用随时间递增的奖励函数，反映学员参与度的提升和对系统适应性的感知。
• 部署实时、自主的教练循环，根据自适应模型选择并提供积极心理学练习。

Figure 1 . The components of the VITA system for adaptive robotic mental well-being coaching.

实验结果

研究问题

• RQ1 与预设脚本模型相比，自主式多模态机器人教练是否能显著提升学员对互动性和响应性的感知？
• RQ2 在为期4周的时间内，机器人教练在多大程度上能够适应个体学员的行为模式（如面部表情、语音时长）？
• RQ3 在真实世界机器人教练会话中，能否有效检测并最小化互动中断？
• RQ4 基于大语言模型的对话能力在多大程度上影响了学员被理解与被倾听的感受？
• RQ5 长期使用VITA进行机器人教练是否能带来显著且可测量的心理健康水平提升？

主要发现

• 学员对自适应配置的VITA（包括通用型配置）的评价显著优于预设脚本模型，报告称自己感到被理解与被倾听。
• 自适应VITA系统在长期使用中成功实现个性化，各次会话中强化学习的奖励信号显著提升。
• 互动中断被减少至平均每场3次（每次约3秒），相比以往工作（15–20次，每次约10秒）有显著改善。
• 学员报告心理健康水平显著提升，四週教练干预后，RPWS量表得分具有统计学上的显著增长。
• 访谈显示，学员认为LLM驱动的回应自然且内容上富有同理心，但部分人指出回应在情感深度上仍显不足。
• 系统在为期4周的真实世界部署中表现出稳健性能，期间未检测到系统故障或重大崩溃。

Figure 2 . Architecture of the HARMONI modules integrated in the robotic coach system.