[Paper Review] Discovering Mental Health Research Topics with Topic Modeling

Xin Gao, Cem Sazara|arXiv (Cornell University)|Aug 25, 2023

Mental Health via WritingPsychology3 citations

TL;DR

This study applies a fine-tuned BERTopic model to 96,676 mental health research papers (2010–2023) to identify evolving research topics and trends. Using BERT-based embeddings and topic modeling, it outperforms LDA-BERT and Top2Vec in coherence and diversity metrics, revealing key themes like suicide, substance use, and autism, while a GPT-3.5-turbo-driven analysis of ML methods shows a shift toward deep learning and NLP in mental health research.

ABSTRACT

Mental health significantly influences various aspects of our daily lives, and its importance has been increasingly recognized by the research community and the general public, particularly in the wake of the COVID-19 pandemic. This heightened interest is evident in the growing number of publications dedicated to mental health in the past decade. In this study, our goal is to identify general trends in the field and pinpoint high-impact research topics by analyzing a large dataset of mental health research papers. To accomplish this, we collected abstracts from various databases and trained a customized Sentence-BERT based embedding model leveraging the BERTopic framework. Our dataset comprises 96,676 research papers pertaining to mental health, enabling us to examine the relationships between different topics using their abstracts. To evaluate the effectiveness of the model, we compared it against two other state-of-the-art methods: Top2Vec model and LDA-BERT model. The model demonstrated superior performance in metrics that measure topic diversity and coherence. To enhance our analysis, we also generated word clouds to provide a comprehensive overview of the machine learning models applied in mental health research, shedding light on commonly utilized techniques and emerging trends. Furthermore, we provide a GitHub link* to the dataset used in this paper, ensuring its accessibility for further research endeavors.

Motivation & Objective

• To identify and analyze evolving research topics in mental health research over the past decade, particularly in response to the COVID-19 pandemic.
• To evaluate the performance of advanced topic modeling techniques—BERTopic, LDA-BERT, and Top2Vec—on mental health research paper abstracts.
• To uncover dominant and emerging machine learning techniques used in mental health research through LLM-powered extraction and visualization.
• To provide a publicly accessible dataset and model to support future research in computational mental health.

Proposed method

• Collected abstracts from arXiv, ACM, bioRxiv, medRxiv, and PubMed, spanning January 2010 to March 2023, resulting in a dataset of 96,676 mental health research papers.
• Trained a custom Sentence-BERT based embedding model within the BERTopic framework to generate contextual document embeddings and cluster them into semantically coherent topics.
• Evaluated topic quality using multiple metrics: NPMI (Normalized Pointwise Mutual Information), Cv (Coefficient of Topic Coherence), TD (Topic Diversity), and Inv. RBO (Inverted Rank-Biased Overlap).
• Compared BERTopic’s performance against two state-of-the-art baselines: LDA-BERT and Top2Vec, using the same evaluation metrics.
• Utilized a GPT-3.5-turbo API prompt to extract and categorize machine learning methods mentioned in the papers, enabling trend analysis over time.
• Generated a word cloud to visually represent the frequency and prominence of machine learning models in mental health research, highlighting shifts from traditional ML to deep learning.

Figure 1: 2D UMAP Visualization of BERTopic Clusters

Experimental results

Research questions

• RQ1 What are the dominant and emerging research topics in mental health research from 2010 to 2023, particularly during and after the COVID-19 pandemic?
• RQ2 How does BERTopic compare to LDA-BERT and Top2Vec in terms of topic coherence, diversity, and interpretability when applied to mental health research papers?
• RQ3 What trends in machine learning methodology are evident in mental health research, and how have they evolved over time?
• RQ4 Which clinical and psychosocial themes are most prominently represented in the topic modeling output, and what do they reveal about research priorities?

Key findings

• BERTopic outperformed both LDA-BERT and Top2Vec in topic diversity (TD score of 0.7208) and topic coherence (Cv score of 0.8068), indicating higher-quality, more interpretable topics.
• The BERTopic model achieved a normalized pointwise mutual information (NPMI) score of 0.2892, reflecting strong semantic coherence among words in each topic.
• Top2Vec achieved the highest Inv. RBO score (0.9669), indicating strong word order similarity and diversity, though BERTopic surpassed it in overall coherence and interpretability.
• Topic modeling revealed key themes such as suicide (with terms like 'suicidal', 'ideation', 'attempt'), substance use (tobacco, cannabis, opioids), and child-parent relationships in autism.
• A word cloud generated from GPT-3.5-turbo-extracted ML methods showed a clear shift from traditional machine learning before 2017 to a surge in neural networks, NLP, CV, and reinforcement learning after 2017.
• The study provides a publicly available dataset via GitHub (https://github.com/stella-gao/Mental-Health-Research-Paper-Dataset), enabling reproducibility and future research.

Figure 2: Word Scores of Top 8 Topics from BERTopic