[Paper Review] Computer-Assisted Text Analysis for Social Science: Topic Models and Beyond

Ryan Wesslen|arXiv (Cornell University)|Mar 29, 2018

Computational and Text Analysis Methods68 references16 citations

TL;DR

This paper reviews the evolution and application of topic models in social science, emphasizing structural topic modeling (STM) for causal inference and interpretability. It integrates computer-assisted text analysis with social science research, advancing methods for evaluation, visualization, and handling multi-modality in unstructured text data.

ABSTRACT

Topic models are a family of statistical-based algorithms to summarize, explore and index large collections of text documents. After a decade of research led by computer scientists, topic models have spread to social science as a new generation of data-driven social scientists have searched for tools to explore large collections of unstructured text. Recently, social scientists have contributed to topic model literature with developments in causal inference and tools for handling the problem of multi-modality. In this paper, I provide a literature review on the evolution of topic modeling including extensions for document covariates, methods for evaluation and interpretation, and advances in interactive visualizations along with each aspect's relevance and application for social science research.

Motivation & Objective

• To survey the development and adoption of topic models in social science research, particularly in response to the growing volume of unstructured text data.
• To highlight the limitations of traditional text analysis methods like human coding and dictionary-based approaches in handling large-scale text.
• To present structural topic modeling (STM) as an extension of LDA that enables causal inference and integration of document-level covariates.
• To examine recent advances in evaluation, interpretation, and interactive visualization for topic models in social science contexts.
• To identify future research opportunities in explainable AI (XAI) and interactive systems for text analysis in computational social science.

Proposed method

• Uses a literature review to trace the evolution of topic modeling from latent semantic indexing (LSI) and LDA to advanced models like STM.
• Applies the bag-of-words (BoW) assumption to convert text into document-term matrices (DTM), enabling statistical modeling of word co-occurrence patterns.
• Introduces structural topic modeling (STM) as an extension of LDA that incorporates document-level covariates to estimate causal effects on topic prevalence.
• Employs posterior predictive checks, model selection, and visualization tools (e.g., Shiny, Vega-lite) for model evaluation and interpretation.
• Integrates word embedding models like word2vec and GloVe as complementary approaches to count-based topic models for richer semantic representation.
• Proposes the development of explainable, interactive visualization systems to enhance interpretability and user insight in topic modeling workflows.

Experimental results

Research questions

• RQ1 How can topic models be adapted to support causal inference in social science research using text data?
• RQ2 What are the key methodological advances in topic modeling that improve interpretability and evaluation for social science applications?
• RQ3 How do structural topic models (STM) extend traditional LDA to incorporate document-level covariates and enable causal analysis?
• RQ4 What role do interactive visualizations and explainable AI (XAI) play in enhancing the transparency and usability of topic models?
• RQ5 How do word embedding models like word2vec and GloVe compare to traditional topic models in capturing semantic meaning in text?

Key findings

• Topic models, particularly LDA and STM, have become essential tools for analyzing large-scale unstructured text in social science, enabling the discovery of latent topics without prior labeling.
• The structural topic model (STM) successfully extends LDA by incorporating document covariates, allowing researchers to estimate causal effects of variables like political ideology or survey responses on topic prevalence.
• Interactive visualization tools such as stmViz and stmBrowser are available but limited in functionality, highlighting a need for more robust, explainable interfaces.
• Word embedding models like word2vec and GloVe provide deeper semantic understanding by modeling word context, outperforming simple count-based models in capturing meaning.
• The integration of topic modeling with explainable AI (XAI) offers a path toward transparent, interpretable, and interactive systems that combine human insight with machine learning outputs.
• The stm R package enables widespread use of STM with built-in functions for preprocessing, model selection, posterior predictive checks, and static visualizations, supporting reproducible research.