[论文解读] Mental Illness Classification on Social Media Texts using Deep Learning and Transfer Learning

Iqra Ameer, Muhammad Arif+3 more|arXiv (Cornell University)|Jul 3, 2022

Mental Health via Writing被引用 28

一句话总结

本文比较传统机器学习、深度学习和迁移学习模型在Reddit帖子上的多类精神疾病检测，发现 RoBERTa 作为最佳迁移学习方法，准确率为 0.83。Bi-LSTM 在 DL 方法中也表现出色。

ABSTRACT

Given the current social distance restrictions across the world, most individuals now use social media as their major medium of communication. Millions of people suffering from mental diseases have been isolated due to this, and they are unable to get help in person. They have become more reliant on online venues to express themselves and seek advice on dealing with their mental disorders. According to the World health organization (WHO), approximately 450 million people are affected. Mental illnesses, such as depression, anxiety, etc., are immensely common and have affected an individuals' physical health. Recently Artificial Intelligence (AI) methods have been presented to help mental health providers, including psychiatrists and psychologists, in decision making based on patients' authentic information (e.g., medical records, behavioral data, social media utilization, etc.). AI innovations have demonstrated predominant execution in numerous real-world applications broadening from computer vision to healthcare. This study analyzes unstructured user data on the Reddit platform and classifies five common mental illnesses: depression, anxiety, bipolar disorder, ADHD, and PTSD. We trained traditional machine learning, deep learning, and transfer learning multi-class models to detect mental disorders of individuals. This effort will benefit the public health system by automating the detection process and informing appropriate authorities about people who require emergency assistance.

研究动机与目标

• 由于 COVID-19 时期在线交流增加和社交距离，推动使用社交媒体数据进行自动化精神健康筛查。
• 研究从 Reddit 帖子中检测五种精神疾病和一个 none 类的可行性。
• 在标准化的 Reddit 数据集上比较传统机器学习、深度学习和迁移学习方法。
• 评估类间性能并分析从短文本中检测抑郁和焦虑所面临的挑战。

提出的方法

• 应用四种传统 ML 分类器，使用 TF-IDF 词 n-gram，最长3。
• 评估包括 GRU、Bi-GRU、CNN、LSTM 和 Bi-LSTM 的深度学习模型。
• 对数据集进行预训练模型 BERT、XLNet 和 RoBERTa 的微调。
• 将预训练 RoBERTa 基模型作为多类分类的主要迁移学习方法。
• 使用提供的 Reddit 数据集（16,930 条帖子，含 train/dev/test 拆分）进行评估。

Figure 1: Mental Illness Dataset Statistics

实验结果

研究问题

• RQ1 传统 ML 方法结合 n-gram 特征是否能在 Reddit 精神疾病分类任务上达到有竞争力的准确率？
• RQ2 深度学习模型在该任务上是否优于传统 ML？
• RQ3 哪些迁移学习模型（BERT、XLNet、RoBERTa）在 Reddit 的多类精神疾病检测中表现最好？
• RQ4 模型在区分 none 类方面表现如何，且哪些错误在疾病类别之间较常见？

主要发现

• RoBERTa 在测试模型中以 0.83 的整体最高准确率和 0.83 的 F1-score 获得最高表现。
• XLNet 与 RoBERTa 相近，准确率 0.79，F1-score 0.80。
• Bi-LSTM 在深度学习模型中表现最佳，准确率 0.78，F1-score 0.79。
• 传统 ML 使用词 1–3-grams 的准确率为 0.78，F1-score 0.67，显示出有竞争力的性能。
• ROBOTa 类别层级结果在 none 上表现极高（F1 0.98），在疾病类别上存在显著差异，ADHD 和 PTSD 的 F1 高于抑郁和焦虑。
• RoBERTa 能以高精度和召回率正确识别 none-class 帖子，表明对非疾病内容的假阳性较低。

Figure 2: RoBERTa confusion matrix