[Paper Review] Multiscale Analysis of Count Data through Topic Alignment

Julia Fukuyama, Kris Sankaran+1 more|PubMed|Sep 12, 2021

Bioinformatics and Genomic Networks19 references4 citations

TL;DR

This paper introduces topic alignment, a method to compare topic models across different values of K, enabling multiscale analysis of count data. By constructing weighted graphs of topic similarities using inner products or optimal transport, it provides diagnostics like coherence and refinement scores to identify stable, transient, or splitting topics, offering deeper insight than single-K models—validated on simulated and real microbiota data with the R package alto.

ABSTRACT

Topic modeling is a popular method used to describe biological count data. With topic models, the user must specify the number of topics $K$. Since there is no definitive way to choose $K$ and since a true value might not exist, we develop a method, which we call topic alignment, to study the relationships across models with different $K$. In addition, we present three diagnostics based on the alignment. These techniques can show how many topics are consistently present across different models, if a topic is only transiently present, or if a topic splits into more topics when $K$ increases. This strategy gives more insight into the process of generating the data than choosing a single value of $K$ would. We design a visual representation of these cross-model relationships, show the effectiveness of these tools for interpreting the topics on simulated and real data, and release an accompanying R package, alto.

Motivation & Objective

• Address the challenge of selecting the number of topics K in topic models, which lacks a definitive method and may suppress or distort biological patterns.
• Overcome the limitations of single-K topic modeling by enabling cross-model comparison to reveal relationships across scales.
• Develop diagnostics to assess topic stability, transience, and splitting behavior across different K values.
• Provide a practical, post-estimation framework compatible with existing topic modeling workflows, avoiding the need for new multiscale models.
• Detect model mis-specifications such as elevated heterogeneity or strain switching in microbiota data through alignment patterns.

Proposed method

• Construct a weighted graph where nodes represent topics from models fitted at different K values, and edges encode topic similarity.
• Estimate alignment weights using either the inner product between topic vectors or optimal transport distances between topic distributions.
• Define a coherence score to measure how consistently a topic persists across multiple K values.
• Introduce a refinement score to quantify how finely resolved topics emerge from coarser ones, indicating whether topics split or merge with increasing K.
• Use the alignment structure to guide interpretation of topics at large K by relating them to coarser, more interpretable topics.
• Implement the method in the R package alto, with support for visualization and reproducibility via vignettes and HPC scripts.

Figure 1: How to read a topic alignment. Construction of weights is discussed in Section 3.1 and paths are defined in Subsection 3.2.2 .

Experimental results

Research questions

• RQ1 How do topics from topic models with different K values relate to one another in terms of similarity and stability?
• RQ2 Can topic alignment detect biologically plausible deviations from the LDA generative model, such as strain switching or elevated heterogeneity?
• RQ3 To what extent do topics remain coherent or split as K increases, and how can this be quantified?
• RQ4 Can alignment diagnostics help identify spurious or transient topics that do not persist across scales?
• RQ5 How can alignment improve interpretability of high-K topic models by relating them to lower-K, more interpretable prototypes?

Key findings

• Topic alignment successfully identifies stable topics that persist across multiple K values, indicating robust underlying structure in the data.
• The coherence score effectively distinguishes long-lived topics from transient ones, with high coherence indicating consistent presence across scales.
• The refinement score reveals that some topics split into multiple sub-topics as K increases, reflecting finer-grained biological sub-communities.
• On a vaginal microbiota dataset, the method recovered known Community State Types (CSTs) while also identifying novel, finer-scale sub-community structures.
• In simulations, topic alignment detected model mis-specifications such as strain switching and elevated heterogeneity, with alignment patterns reflecting these deviations.
• The R package alto enables reproducible analysis and visualization of topic alignments, supporting interactive exploration of topic relationships across K.

Figure 2: Top panels (a-b) illustrate topic alignment for product (a) and transport (b) alignments. The bottom panels (a-c) illustrate the diagnostic scores characterizing the alignment. (a) Each vertical column corresponds to a topic. Each circle encodes weights $\gamma_{iv}$ for a single sample $i