[Paper Review] 3M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecular Structure Generation

Huaisheng Zhu, Teng Xiao+1 more|arXiv (Cornell University)|Mar 11, 2024

Biomedical Text Mining and Ontologies4 citations

TL;DR

3M-Diffusion proposes a novel latent multi-modal diffusion model for text-guided generation of molecular graphs, leveraging a graph autoencoder to map molecules to a continuous latent space aligned with text embeddings. By training a diffusion model in this aligned latent space, it generates diverse, novel, and semantically accurate molecular structures that match textual descriptions, outperforming SOTA methods on multiple benchmarks.

ABSTRACT

Generating molecular structures with desired properties is a critical task with broad applications in drug discovery and materials design. We propose 3M-Diffusion, a novel multi-modal molecular graph generation method, to generate diverse, ideally novel molecular structures with desired properties. 3M-Diffusion encodes molecular graphs into a graph latent space which it then aligns with the text space learned by encoder-based LLMs from textual descriptions. It then reconstructs the molecular structure and atomic attributes based on the given text descriptions using the molecule decoder. It then learns a probabilistic mapping from the text space to the latent molecular graph space using a diffusion model. The results of our extensive experiments on several datasets demonstrate that 3M-Diffusion can generate high-quality, novel and diverse molecular graphs that semantically match the textual description provided.

Motivation & Objective

• To address the limitation of existing methods in generating diverse and novel molecular graphs from textual descriptions.
• To overcome the degeneracy and structural ambiguity of SMILES encoding in molecular generation.
• To improve the diversity and quality of text-guided molecular generation beyond beam search or greedy decoding.
• To enable conditional generation of molecules with desired properties (e.g., solubility, drug-likeness) using prompt engineering.
• To establish a unified framework for multi-modal molecular generation by aligning text and molecular graph latent spaces.

Proposed method

• A graph autoencoder encodes molecular graphs into a continuous latent space, decoupling structural complexity from the generation process.
• Text and molecular graph encoders are pre-trained via contrastive learning on a large corpus of molecule-text pairs to align their latent representations.
• A latent diffusion model learns a probabilistic mapping from the text-conditional latent space to the molecular graph latent space.
• The diffusion process operates in the continuous latent space, enabling high-level semantic control while preserving structural fidelity.
• Generated molecules are decoded from the latent space and evaluated for validity, novelty, and semantic alignment with the input text.
• Conditional generation is achieved via prompt engineering, where specific textual prompts guide the model toward desired molecular properties.

Experimental results

Research questions

• RQ1 Can a latent multi-modal diffusion model generate diverse and novel molecular graphs that semantically match a given textual description?
• RQ2 How does aligning the latent spaces of text and molecular graphs improve generation quality and diversity?
• RQ3 Can the model outperform SOTA methods in both unconditional and text-guided molecular graph generation tasks?
• RQ4 To what extent can the model generate molecules with desired properties (e.g., high logP, drug-likeness) through prompt conditioning?
• RQ5 How does the model balance structural similarity to reference molecules with the introduction of novel components?

Key findings

• 3M-Diffusion outperforms SOTA methods, including MolT5-large, in generating diverse and novel molecular graphs while maintaining high semantic alignment with textual prompts.
• The model generates molecules with higher logP values compared to MolT5-large, indicating improved solubility properties through prompt-based conditioning.
• Qualitative analysis shows that 3M-Diffusion generates molecules with higher structural similarity to reference molecules (e.g., flavones) while introducing novel substructures.
• Ablation studies confirm that latent space alignment between text and molecular graphs is critical for performance, with significant drops in quality when alignment is removed.
• The model successfully generates molecules sharing common substructures with ground truth, demonstrating its ability to preserve key pharmacophoric features.
• The method achieves strong performance across four real-world benchmarks, confirming its robustness and generalization capability in text-guided molecular generation.