[Paper Review] Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective

Jiatong Li, Yunqing Liu+5 more|arXiv (Cornell University)|Jun 11, 2023

Machine Learning in Materials Science40 references22 citations

TL;DR

MolReGPT uses retrieval-based prompts and in-context learning with ChatGPT to translate between molecules and natural language for molecule discovery, beating fine-tuned MolT5-base on key tasks without any fine-tuning.

ABSTRACT

Molecule discovery plays a crucial role in various scientific fields, advancing the design of tailored materials and drugs. However, most of the existing methods heavily rely on domain experts, require excessive computational cost, or suffer from sub-optimal performance. On the other hand, Large Language Models (LLMs), like ChatGPT, have shown remarkable performance in various cross-modal tasks due to their powerful capabilities in natural language understanding, generalization, and in-context learning (ICL), which provides unprecedented opportunities to advance molecule discovery. Despite several previous works trying to apply LLMs in this task, the lack of domain-specific corpus and difficulties in training specialized LLMs still remain challenges. In this work, we propose a novel LLM-based framework (MolReGPT) for molecule-caption translation, where an In-Context Few-Shot Molecule Learning paradigm is introduced to empower molecule discovery with LLMs like ChatGPT to perform their in-context learning capability without domain-specific pre-training and fine-tuning. MolReGPT leverages the principle of molecular similarity to retrieve similar molecules and their text descriptions from a local database to enable LLMs to learn the task knowledge from context examples. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation. Experimental results show that compared to fine-tuned models, MolReGPT outperforms MolT5-base and is comparable to MolT5-large without additional training. To the best of our knowledge, MolReGPT is the first work to leverage LLMs via in-context learning in molecule-caption translation for advancing molecule discovery. Our work expands the scope of LLM applications, as well as providing a new paradigm for molecule discovery and design.

Motivation & Objective

• Motivate efficient molecule discovery by translating between molecules and natural language to aid design of materials and drugs.
• Reduce reliance on domain experts and costly fine-tuning by leveraging LLMs with retrieval-based prompts.
• Ground LLM outputs in domain-relevant examples retrieved from local databases.
• Demonstrate that an LLM (ChatGPT) can outperform fine-tuned baselines on Mol2Cap and Cap2Mol tasks.
• Provide a runnable framework and evidence that prompts can guide cross-modal molecule understanding without parameter updates.

Proposed method

• Propose MolReGPT, a four-stage framework: Molecule-Caption Retrieval, Prompt Management, In-Context Few-Shot Molecule Learning, and Generation Calibration.
• Use Morgan Fingerprints with Dice similarity for retrieving structurally similar molecules to ground captions.
• Use BM25-based caption retrieval to find semantically similar captions for Cap2Mol tasks.
• Construct system prompts with retrieved examples to guide ChatGPT through in-context learning without fine-tuning.
• Enforce a predefined JSON output format and apply generation calibration to correct format or re-query when needed.

Experimental results

Research questions

• RQ1 Can retrieval-augmented in-context learning enable ChatGPT to perform molecule captioning (Mol2Cap) and text-based molecule generation (Cap2Mol) without fine-tuning?
• RQ2 Do Morgan Fingerprints-based molecule retrieval and BM25-based caption retrieval improve LLM-grounded molecule-caption translation performance?
• RQ3 How does MolReGPT compare to fine-tuned baselines (e.g., MolT5-base) on Mol2Cap and Cap2Mol tasks?
• RQ4 What is the impact of the number of retrieved examples (n-shot) on translation quality?

Key findings

• MolReGPT achieves Mol2Cap Text2Mol score of 0.560 on 10-shot Prompts, surpassing MolT5-base without fine-tuning.
• MolReGPT achieves Cap2Mol Text2Mol score of 0.571 on 10-shot Prompts, and increases Text2Mol by about 3% compared to MolT5 (no fine-tuning).
• On Mol2Cap, GPT-3.5-turbo with 10-shot MolReGPT matches or exceeds fine-tuned baselines in several metrics (BLEU, ROUGE, METEOR).
• For Cap2Mol, MolReGPT with 10-shot prompts delivers competitive BLEU and EM scores and higher Morgan FTS and Text2Mol metrics than baselines.
• Overall, MolReGPT outperforms fine-tuned MolT5-base on the reported sub-tasks without any fine-tuning.