[Paper Review] Multi-Task Learning for Mental Health using Social Media Text

Adrian Benton, Margaret Mitchell+1 more|arXiv (Cornell University)|Dec 10, 2017

Mental Health via Writing32 references60 citations

TL;DR

This paper presents a neural multi-task learning framework that predicts suicide risk and multiple mental health conditions from Twitter text, using gender as an auxiliary task, and shows improvements over single-task baselines, especially for data-sparse conditions.

ABSTRACT

We introduce initial groundwork for estimating suicide risk and mental health in a deep learning framework. By modeling multiple conditions, the system learns to make predictions about suicide risk and mental health at a low false positive rate. Conditions are modeled as tasks in a multi-task learning (MTL) framework, with gender prediction as an additional auxiliary task. We demonstrate the effectiveness of multi-task learning by comparison to a well-tuned single-task baseline with the same number of parameters. Our best MTL model predicts potential suicide attempt, as well as the presence of atypical mental health, with AUC > 0.8. We also find additional large improvements using multi-task learning on mental health tasks with limited training data.

Motivation & Objective

• Demonstrate the utility of multi-task learning (MTL) for predicting suicide risk and mental health conditions from social media text.
• Explore how auxiliary tasks (including gender) and task selection affect predictive performance.
• Provide fair comparisons to single-task baselines with the same parameter count to isolate benefits of MTL.
• Show how MTL benefits tasks with limited training data through shared representations.
• Assess practical clinical relevance via metrics emphasizing low false positives (e.g., TPR@FPR=0.1).

Proposed method

• Use a neural multi-task architecture with a shared input-to-hidden representation and per-task hidden layers for multiple mental health prediction tasks.
• Represent input as frequencies of the 1- to 5-gram character features (top 5,000 n-grams) derived from entire Twitter histories.
• Train with mini-batch Adagrad, jointly optimizing all tasks, enabling asynchronous updates across disjoint data.
• Compare MTL to baseline logistic regression (LR) on character n-grams and to single-task learning (STL) neural models with equal parameter counts.
• Evaluate using ROC/AUC and TPR at FPR=0.1, plus precision-recall analyses, across a set of mental health conditions and gender.
• Experiment with various auxiliary task subsets to assess their impact on main task performance and analyze comorbidity benefits.

Experimental results

Research questions

• RQ1 Does multi-task learning improve prediction of suicide risk and mental health conditions over single-task models with the same parameter count?
• RQ2 What is the impact of including gender as an auxiliary task on prediction accuracy across conditions?
• RQ3 How does the choice of auxiliary tasks influence performance, especially for data-poor conditions like bipolar disorder and PTSD?
• RQ4 Can predicting multiple related conditions (comorbidity) act as regularization to improve rare condition prediction?
• RQ5 Are there optimal task subsets that maximize performance for selected main tasks (e.g., anxiety, depression, suicide)?

Key findings

• MTL improves several mental health predictions over STL and LR; for example, suicide prediction achieves AUC of 0.848 with MTL.
• MTL with gender as an auxiliary task improves performance on several conditions, including anxiety, panic, and PTSD.
• MTL yields notable gains for low-data conditions (bipolar disorder and PTSD) by leveraging comorbidity with data-rich conditions (depression, anxiety).
• PTSD AUC improves from 0.770 (LR) to 0.786 (MTL), while STL performs worse (0.667); bipolar improves from 0.752 (LR) / 0.552 (STL) to 0.723 (MTL).
• Using all auxiliary tasks yields the best main-task performance across neuroatypicality, suicide, depression, anxiety, eating disorders, panic, schizophrenia, bipolar, and PTSD; gender as auxiliary generally helps but not uniformly for gender prediction itself.