[Paper Review] Early Risk Detection of Pathological Gambling, Self-Harm and Depression Using BERT

Ana-Maria Bucur, Adrian Cosma+1 more|arXiv (Cornell University)|Jun 30, 2021

Mental Health via WritingPsychology33 references17 citations

TL;DR

This paper presents a BERT-based approach for early risk detection of pathological gambling, self-harm, and depression severity using social media text from Reddit. By leveraging fine-tuned BERT models with contrastive learning to align user posts with questionnaire answers, the BLUE team achieved state-of-the-art performance on Task 3 (depression severity), with a 30.36% AHR score, demonstrating the effectiveness of semantic similarity learning in weakly-supervised mental health screening.

ABSTRACT

Early risk detection of mental illnesses has a massive positive impact upon the well-being of people. The eRisk workshop has been at the forefront of enabling interdisciplinary research in developing computational methods to automatically estimate early risk factors for mental issues such as depression, self-harm, anorexia and pathological gambling. In this paper, we present the contributions of the BLUE team in the 2021 edition of the workshop, in which we tackle the problems of early detection of gambling addiction, self-harm and estimating depression severity from social media posts. We employ pre-trained BERT transformers and data crawled automatically from mental health subreddits and obtain reasonable results on all three tasks.

Motivation & Objective

• To develop computational methods for early detection of mental health risks using social media text.
• To address the challenge of limited labeled data in mental health NLP by leveraging transfer learning and weak supervision.
• To improve depression severity estimation by modeling semantic similarity between user posts and questionnaire answers.
• To enhance detection of pathological gambling and self-harm through data augmentation from mental health subreddits.
• To contribute to real-world early intervention by enabling scalable, automated risk screening.

Proposed method

• Fine-tuned BERT models to classify responses to the Beck Depression Inventory (BDI) across 21 questions with 4 answer options each.
• Employed contrastive learning by computing dot-product similarity between user post embeddings and answer text embeddings to improve semantic alignment.
• Constructed a multi-head architecture with joint prediction of answer choices and depression severity scores (minimal/mild/moderate/severe).
• Used AdamW optimizer with learning rates of 0.0002 (for classification) and 0.00002 (for similarity learning), trained for 2 epochs.
• Augmented training data by crawling Reddit subreddits related to gambling and self-harm to improve model generalization.
• At inference, selected the answer with maximum probability across all user posts per individual to handle noisy or sparse input.

Experimental results

Research questions

• RQ1 Can BERT-based models effectively detect early signs of pathological gambling from unstructured social media text?
• RQ2 How can semantic similarity between user posts and standardized questionnaire answers improve depression severity estimation?
• RQ3 Does contrastive learning improve performance in weakly-supervised mental health risk detection with noisy, sparse data?
• RQ4 Can data augmentation from mental health subreddits enhance model performance on self-harm and gambling detection tasks?
• RQ5 What is the relative performance of different BERT fine-tuning strategies—classification, joint prediction, or similarity learning—for depression screening?

Key findings

• The similarity learning approach (run 2) achieved the best overall performance on Task 3, with a 30.36% AHR score, outperforming pure classification and ensemble methods.
• The best-performing model (run 2) achieved a 75.42% ADODL score, indicating strong alignment with ground truth answers.
• Ensemble methods (run 3 and 4) improved results slightly but did not surpass the standalone similarity learning model.
• The model demonstrated robustness in low-resource settings, achieving 65.42% ACR and 75.42% ADODL on the self-harm detection task.
• The use of Reddit-based data augmentation improved model generalization, particularly for pathological gambling and self-harm detection.
• Contrastive learning significantly enhanced performance by leveraging semantic meaning of answer formulations, proving more effective than standard classification in noisy regimes.