[论文解读] Automated speech- and text-based classification of neuropsychiatric conditions in a multidiagnostic setting

Lasse Hansen, Roberta Rocca+9 more|arXiv (Cornell University)|Jan 13, 2023

Voice and Speech Disorders被引用 6

一句话总结

本研究在多诊断环境下，采用传统机器学习与先进的Transformer模型，评估了基于语音和文本的自动化神经精神疾病分类方法，涵盖重度抑郁障碍、精神分裂症和自闭症谱系障碍。与二元分类相比，多类别分类中的性能显著下降，表明二元任务中识别出的标志物可能反映的是临床与非临床人群的一般性差异，而非特定疾病特征，且语音与文本模型的结合显示出互补的诊断价值。

ABSTRACT

Speech patterns have been identified as potential diagnostic markers for neuropsychiatric conditions. However, most studies only compare a single clinical group to healthy controls, whereas clinical practice often requires differentiating between multiple potential diagnoses (multiclass settings). To address this, we assembled a dataset of repeated recordings from 420 participants (67 with major depressive disorder, 106 with schizophrenia and 46 with autism, as well as matched controls), and tested the performance of a range of conventional machine learning models and advanced Transformer models on both binary and multiclass classification, based on voice and text features. While binary models performed comparably to previous research (F1 scores between 0.54-0.75 for autism spectrum disorder, ASD; 0.67-0.92 for major depressive disorder, MDD; and 0.71-0.83 for schizophrenia); when differentiating between multiple diagnostic groups performance decreased markedly (F1 scores between 0.35-0.44 for ASD, 0.57-0.75 for MDD, 0.15-0.66 for schizophrenia, and 0.38-0.52 macro F1). Combining voice and text-based models yielded increased performance, suggesting that they capture complementary diagnostic information. Our results indicate that models trained on binary classification may learn to rely on markers of generic differences between clinical and non-clinical populations, or markers of clinical features that overlap across conditions, rather than identifying markers specific to individual conditions. We provide recommendations for future research in the field, suggesting increased focus on developing larger transdiagnostic datasets that include more fine-grained clinical features, and that can support the development of models that better capture the complexity of neuropsychiatric conditions and naturalistic diagnostic assessment.

研究动机与目标

• 评估语音和文本基机器学习模型在真实临床环境中区分多种神经精神疾病诊断的表现。
• 探究在二元分类（如某一疾病与对照组）上训练的模型是否能推广到反映实际诊断挑战的多类别设置中。
• 检查语音和文本特征在精神疾病多类别分类中是否提供互补信息。
• 评估最先进的Transformer模型相较于传统机器学习方法在该多诊断背景下的有效性。
• 为未来自动化精神病学评估研究提供可操作建议，强调应采用更大、更具异质性的数据集和更精细的临床预测目标。

提出的方法

• 收集了420名参与者（包括67名重度抑郁障碍患者、106名精神分裂症患者、46名自闭症谱系障碍患者及匹配对照组）的重复语音和文本录音，构建了一个新型数据集。
• 使用标准工具（如OpenSMILE、Covarep）提取语音特征，包括音高、能量、频谱系数和语调特征；对语音转录以获取语言学特征，如情感倾向、词频及语义嵌入。
• 在语音和文本模态上，分别训练传统机器学习模型（如SVM、随机森林）和深度学习模型，包括微调的BERT基及其它Transformer架构。
• 通过早期融合或晚期融合策略结合语音和文本模型，以评估跨模态的性能增益。
• 在二元分类（某一疾病 vs. 对照组）和多类别分类（三种疾病 + 对照组）任务中评估模型性能，报告各类设置下的F1分数和宏F1分数。
• 开展消融研究以评估各模态和模型类型对性能的贡献，并探讨预训练与微调对性能的影响。

Figure 1 : Processing and training pipelines for feature-based text, voice, and ensemble models (see section 2 and Glossary for details on each processing step). For transformer-based text models, we use the same pre- and postprocessing step, while feature extraction and predictive modeling are perf

实验结果

研究问题

• RQ1 与二元分类设置相比，语音和文本基机器学习模型在同时分类多种神经精神疾病时表现如何？
• RQ2 在精神疾病的多类别分类中，语音和文本特征在多大程度上提供互补的诊断信息？
• RQ3 在多诊断背景下，最先进的Transformer模型是否优于传统机器学习模型？
• RQ4 当诊断共享重叠的临床特征时，当前模型在捕捉特定疾病标志物方面存在哪些局限性？
• RQ5 未来研究如何提升自动化精神病学分类系统的临床相关性和鲁棒性？

主要发现

• 二元分类模型表现与以往研究相当，F1分数范围为：自闭症谱系障碍0.54–0.75，重度抑郁障碍0.67–0.92，精神分裂症0.71–0.83。
• 在多类别分类中，性能显著下降，F1分数范围为：自闭症谱系障碍0.35–0.44，重度抑郁障碍0.57–0.75，精神分裂症0.15–0.66，宏F1为0.38–0.52。
• 结合语音和文本特征可提升分类性能，表明这些模态捕获了互补的诊断信息。
• Transformer模型并未始终优于传统机器学习模型，表明架构复杂性本身并不能保证在此背景下获得更优性能。
• 多类别设置中性能的下降表明，二元任务中识别出的标志物可能反映的是临床与非临床人群之间的一般性差异，而非特定疾病特征。
• 本研究建议将研究重点从宽泛的疾病类别诊断，转向预测更精细的临床特征（如情绪状态、认知负荷或症状严重程度），以提升临床实用性。

Figure 2 : Binary and multiclass performance per group and classification setup. The x-axis indicates the three different diagnostic groups, as well as overall performance in the multiclass setting. The y-axis indicates performance measured in terms of F1-scores for group vs matched controls (left p