[Paper Review] Vectorized Bayesian Inference for Latent Dirichlet-Tree Allocation

Zheng Wang, Nizar Bouguila|arXiv (Cornell University)|Feb 21, 2026

Bayesian Methods and Mixture Models0 citations

TL;DR

The paper generalizes LDA by replacing the Dirichlet prior with a Dirichlet-Tree prior (LDTA) and develops fully vectorized mean-field variational inference and expectation propagation, GPU-accelerated for scalable inference.

ABSTRACT

Latent Dirichlet Allocation (LDA) is a foundational model for discovering latent thematic structure in discrete data, but its Dirichlet prior cannot represent the rich correlations and hierarchical relationships often present among topics. We introduce the framework of Latent Dirichlet-Tree Allocation (LDTA), a generalization of LDA that replaces the Dirichlet prior with an arbitrary Dirichlet-Tree (DT) distribution. LDTA preserves LDA's generative structure but enables expressive, tree-structured priors over topic proportions. To perform inference, we develop universal mean-field variational inference and Expectation Propagation, providing tractable updates for all DT. We reveal the vectorized nature of the two inference methods through theoretical development, and perform fully vectorized, GPU-accelerated implementations. The resulting framework substantially expands the modeling capacity of LDA while maintaining scalability and computational efficiency.

Motivation & Objective

• Motivate modeling rich topic correlations and hierarchies beyond Dirichlet constraints.
• Generalize LDA by substituting Dirichlet priors with Dirichlet-Tree priors to capture structured topic relationships.
• Develop scalable, vectorized inference algorithms suitable for large corpora.
• Provide theoretical foundations for Dirichlet-Tree distributions and their conjugacy to multinomial likelihoods.
• Demonstrate practical applicability across text, image, and bioinformatics data.

Proposed method

• Formalize Dirichlet-Tree distributions, their exponential form, and conjugacy to Multinomial likelihoods.
• Introduce Latent Dirichlet-Tree Allocation (LDTA) as a Dirichlet-Tree–driven generalization of LDA.
• Derive a vectorized universal mean-field variational inference (MFVI) algorithm for LDTA.
• Derive a vectorized Expectation Propagation (EP) algorithm for LDTA.
• Introduce the Bayesian operator to simplify and unify posterior updates within the Dirichlet-Tree framework.
• Present vectorized, GPU-accelerated implementations to enable scalable inference.

Experimental results

Research questions

• RQ1 Can LDTA accurately model hierarchical and correlated topic structures through Dirichlet-Tree priors?
• RQ2 How can MFVI and EP be developed and vectorized for LDTA to maintain scalability?
• RQ3 What are the computational and statistical benefits of using Dirichlet-Tree priors over standard Dirichlet priors in topic models?
• RQ4 How do Dirichlet-Tree priors affect conjugacy, updates, and posterior approximations in LDTA?
• RQ5 Do LDTA methods perform well on diverse data domains (text, images, bioinformatics) compared to traditional LDA?

Key findings

• LDTA extends LDA by enabling expressive, tree-structured priors over topic proportions.
• The authors derive fully vectorized MFVI and EP algorithms with tractable updates for Dirichlet-Tree priors.
• The Dirichlet-Tree distribution is shown to be conjugate to multinomial likelihood, supporting scalable Bayesian updates.
• A Bayesian operator is introduced to streamline and unify posterior updates in LDTA.
• GPU-accelerated, vectorized implementations significantly improve scalability on large datasets.
• Experiments span document modeling, image classification, and RNA-sequencing demonstrating broad applicability.