[Paper Review] Interactive Molecular Discovery with Natural Language

Zheni Zeng, Bangchen Yin+8 more|arXiv (Cornell University)|Jun 21, 2023

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

This paper introduces conversational molecular design, a novel interactive paradigm enabling researchers to describe and edit target molecules using natural language. The authors propose ChatMol, a knowledge-enhanced generative model that bridges natural and chemical language by integrating experimental properties and 3D molecular spatial knowledge, significantly improving performance in molecular generation and understanding tasks with reduced training costs compared to existing methods.

ABSTRACT

Natural language is expected to be a key medium for various human-machine interactions in the era of large language models. When it comes to the biochemistry field, a series of tasks around molecules (e.g., property prediction, molecule mining, etc.) are of great significance while having a high technical threshold. Bridging the molecule expressions in natural language and chemical language can not only hugely improve the interpretability and reduce the operation difficulty of these tasks, but also fuse the chemical knowledge scattered in complementary materials for a deeper comprehension of molecules. Based on these benefits, we propose the conversational molecular design, a novel task adopting natural language for describing and editing target molecules. To better accomplish this task, we design ChatMol, a knowledgeable and versatile generative pre-trained model, enhanced by injecting experimental property information, molecular spatial knowledge, and the associations between natural and chemical languages into it. Several typical solutions including large language models (e.g., ChatGPT) are evaluated, proving the challenge of conversational molecular design and the effectiveness of our knowledge enhancement method. Case observations and analysis are conducted to provide directions for further exploration of natural-language interaction in molecular discovery.

Motivation & Objective

• To address the high technical barrier and low interactivity in current molecular design systems, which rely on specialized chemical notations like SMILES.
• To enable flexible, multi-turn human-machine interaction for molecular design using natural language as the primary medium.
• To bridge the semantic gap between natural language descriptions and chemical language representations (e.g., SMILES) in molecular tasks.
• To enhance language models with domain-specific molecular knowledge—particularly experimental properties and 3D structural information—for improved accuracy and interpretability.
• To develop a scalable, knowledge-augmented generative model that supports both molecule understanding and generation in a conversational setting.

Proposed method

• The authors design a conversational molecular design framework where users interact with the model via multi-turn dialogues to describe, retrieve, or modify molecules using natural language.
• They introduce ChatMol, a fine-tuned generative pre-trained language model enhanced with joint pre-training on SMILES strings and biochemical literature to learn cross-lingual associations.
• To inject molecular knowledge, the model is trained on experimental property data from knowledge bases (e.g., physical and chemical properties in natural language) and spatial structure information from molecular toolkit calculations (e.g., bond types, ring systems, aromaticity).
• A rule-based filtering pipeline is used to construct the ChEBI-dia dataset from existing molecule-description parallel datasets, ensuring high-quality, diverse conversational examples.
• The model employs masked language modeling on both SMILES and natural language texts, followed by a cross-lingual alignment head to map natural language entities to their corresponding chemical expressions.
• Knowledge injection is implemented via dual supervision: (1) entity recognition and SMILES lookup from literature, and (2) integration of 3D structural features using force field-based conformation generation.

Figure 1: Conversational molecular design diagram.

Experimental results

Research questions

• RQ1 Can natural language serve as an effective and intuitive interface for interactive molecular design, replacing or supplementing traditional chemical notation?
• RQ2 How can a single language model effectively and synergistically understand and generate both natural language and chemical language in a conversational context?
• RQ3 To what extent does injecting domain-specific molecular knowledge—such as experimental properties and 3D structural features—improve model performance in molecular generation and understanding?
• RQ4 How does the proposed knowledge-enhanced model compare to standard LLMs and existing molecular models in terms of accuracy, efficiency, and generalization?
• RQ5 What are the key challenges in evaluating conversational molecular design systems, especially when ground truth descriptions are ambiguous or incomplete?

Key findings

• ChatMol significantly outperforms baseline models, including large language models like ChatGPT, in both molecule understanding and generation tasks, demonstrating the effectiveness of knowledge injection.
• The model achieves superior performance with substantially lower training costs compared to larger-scale models, indicating high parameter efficiency.
• The integration of experimental properties from knowledge bases and 3D spatial structure information improves the model’s ability to generate chemically valid and functionally plausible molecules.
• Case studies reveal that ChatMol can correctly interpret complex natural language instructions involving property constraints and structural modifications, such as 'make it more hydrophobic' or 'add a benzene ring'.
• The ChEBI-dia dataset, constructed using rule-based filters, provides a valuable benchmark for future research in conversational molecular design.
• Despite promising results, evaluation remains challenging due to ambiguity in natural language descriptions and the lack of one-to-one correspondence between descriptions and molecules.

Figure 2: Knowledgeable and versatile training process for ChatMol. The black arrows indicate the input to output direction of the task.