[论文解读] Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

Anqi Li, Ruihan Wang+6 more|arXiv (Cornell University)|Feb 25, 2026

Mental Health via Writing被引用 0

一句话总结

简要结论：本文提出一个四维框架来评估文本 기반 咨询中对客户抵抗的辅导者回应，创建专家标注数据集，并训练一个微调的 Llama-3.1 模型，优于基线且能够生成解释；一个概念验证研究表明 AI 生成的反馈提升了咨询师的回应质量。

ABSTRACT

Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.

研究动机与目标

• 建立一个理论驱动的多维框架，用于评估文本化咨询中对客户抵抗的辅导者回应。
• 创建一个带有解释的专家标注数据集，用于抵抗-回应干预。
• 训练一个大语言模型，产生细粒度评估和可操作、可解释的解释。
• 证明 AI 生成的反馈在抵抗情境中提升辅导者表现的实际效用。

提出的方法

• 提出一个四维框架：对 autonom y、立场对齐、情感共鸣、会话导向，每个维度具备三种表达水平（无、弱、强）。
• 从 ClientBehavior 与 ObserverWAI 对话中构建专家标注数据集，包含抵抗检测与辅导者回应，以及解释。
• 对 Llama-3.1-8B-Instruct 进行任务的全参数微调，采用5-fold 交叉验证并通过过采样解决类别不平衡问题。
• 在基线（包括 GPT-4o 和 Claude-3.5-Sonnet）上使用 macro-F1 和准确率评估分类性能；通过自动评估指标（BLEU/Rouge）和人工评定评估解释质量。
• 进行一个包含43名辅导员的概念验证研究，以线性混合效应模型测试 AI 生成反馈的有效性。

Figure 1: Overview of our framework for evaluating counselor responses to client resistance. The framework comprises four core communication mechanisms: Respect for Autonomy , Stance Alignment , Emotional Resonance , and Conversational Orientation . For each mechanism, responses are further categori

实验结果

研究问题

• RQ1 多维框架是否能够在抵抗情境中可靠地区分四种沟通机制下辅导者回应表达的水平？
• RQ2 通过带有解释的任务特定微调是否能在分类与解释质量上超越基线大语言模型？
• RQ3 AI 生成的解释和反馈在实时辅导员培训与技能发展中是否可用且有益？

主要发现

Model Name	Respect for Autonomy F1	Respect for Autonomy Acc.	Stance Alignment F1	Stance Alignment Acc.	Emotional Resonance F1	Emotional Resonance Acc.	Conversational Orientation F1	Conversational Orientation Acc.
Our Model	80.92 ± 1.55	87.06 ± 0.85	77.56 ± 1.78	84.06 ± 1.37	77.34 ± 3.67	78.68 ± 3.32	77.87 ± 0.54	77.64 ± 0.56
Explanations	73.24 ± 1.92	83.38 ± 0.57	70.17 ± 1.30	80.74 ± 0.42	73.23 ± 2.67	75.52 ± 3.21	73.21 ± 1.48	74.15 ± 1.73

• 我们的模型在四种机制上的 macro-F1 达到 77.34–81.00%，准确率为 77.64–87.06%，相比 GPT-4o 与 Claude-3.5-Sonnet 提升了 20 点以上 F1；
• 在训练中融入解释至少比仅标签训练获得约 4 个 F1 点的增益。
• 自动化解释的 BLEU-1 = 0.60，人工评估的框架一致性、证据锚定与清晰/具体性质量分数为 2.8–2.9/3.0。
• 解释显示出强烈的词汇对齐（BLEU-1 0.60）和接近天花板的人类评分，表明高质量、可操作的反馈。
• 在受控试验中，接收 AI 生成反馈的辅导员相比对照组显著提升了抵抗-回应的质量（四维中的相位交互效应）。
• 标注可靠性显著（Cohen’s κ 0.74–0.77），覆盖四种机制，具有高质量的解释性推理。

Figure 2: Interaction effects between experimental groups and phases across four dimensions. Solid green lines represent the control group, while dashed orange lines represent the experimental group. Points denote the mean values, and error bars indicate 95% confidence intervals. The results reveal