[Paper Review] ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback

Shengchao Liu, Jiongxiao Wang+5 more|arXiv (Cornell University)|May 29, 2023

Chemical Synthesis and AnalysisBiochemistry, Genetics and Molecular Biology18 citations

TL;DR

ChatDrug uses a three-module framework (PDDS, ReDF, and conversation) with Retrieval and Domain Feedback to perform text-guided editing for small molecules, peptides, and proteins, achieving best performance on 33 of 39 tasks.

ABSTRACT

Recent advancements in conversational large language models (LLMs), such as ChatGPT, have demonstrated remarkable promise in various domains, including drug discovery. However, existing works mainly focus on investigating the capabilities of conversational LLMs on chemical reaction and retrosynthesis. While drug editing, a critical task in the drug discovery pipeline, remains largely unexplored. To bridge this gap, we propose ChatDrug, a framework to facilitate the systematic investigation of drug editing using LLMs. ChatDrug jointly leverages a prompt module, a retrieval and domain feedback (ReDF) module, and a conversation module to streamline effective drug editing. We empirically show that ChatDrug reaches the best performance on 33 out of 39 drug editing tasks, encompassing small molecules, peptides, and proteins. We further demonstrate, through 10 case studies, that ChatDrug can successfully identify the key substructures (e.g., the molecule functional groups, peptide motifs, and protein structures) for manipulation, generating diverse and valid suggestions for drug editing. Promisingly, we also show that ChatDrug can offer insightful explanations from a domain-specific perspective, enhancing interpretability and enabling informed decision-making. This research sheds light on the potential of ChatGPT and conversational LLMs for drug editing. It paves the way for a more efficient and collaborative drug discovery pipeline, contributing to the advancement of pharmaceutical research and development.

Motivation & Objective

• Motivate AI-assisted drug editing as a multi-modal, conversational task beyond traditional structure-centric approaches.
• Develop a prompt-design and retrieval-augmented system to guide ChatGPT in drug editing.
• Demonstrate broad effectiveness across small molecules, peptides, and proteins with a domain-aware feedback loop.
• Showcase interpretability and case studies to highlight substructure and motif identification for drug editing.

Proposed method

• PDDS: design domain-specific prompts to steer ChatGPT toward high-level property editing for three drug types.
• ReDF: retrieve structurally similar candidates and inject domain feedback into prompts to guide generation.
• Conversation module: implement iterative rounds where failed edits trigger retrieval-based refinements and re-prompts.
• Evaluate using a multi-task benchmark with 39 editing tasks across small molecules, peptides, and proteins.
• Treat ChatDrug as a parameter-free, prompt-engineering-based system without learning.

Experimental results

Research questions

• RQ1 Can a conversational LLM be effectively guided to edit drugs across small molecules, peptides, and proteins using domain-aware prompts and retrieval-based feedback?
• RQ2 Does a retrieval and domain feedback loop improve the quality and diversity of edits compared to zero-shot and in-context learning baselines?
• RQ3 What are the effects of conversation rounds and feedback thresholds on editing performance?
• RQ4 Can ChatDrug provide interpretable, domain-relevant explanations and identify key substructures or motifs involved in edits?

Key findings

• ChatDrug achieves the best performance on 33 out of 39 drug editing tasks across small molecules, peptides, and proteins.
• Qualitative case studies show ChatDrug identifies key substructures, motifs, or protein regions responsible for desired edits.
• Across molecule editing, ChatDrug outperforms baselines on multiple single- and multi-objective tasks, with notable improvements over random or standard MoleculeSTM baselines.
• For peptides, ChatDrug edits enhance binding motifs in a way that aligns with experimental peptide-MHC binding trends.
• For proteins, ChatDrug edits yield increased secondary structure content (more helix or more strand) as evaluated by downstream predictors and folding tools.
• Ablation studies indicate that conversational refinement and domain feedback injections substantially outperform zero-shot and single-pass retrieval approaches, with performance improving as rounds increase up to a point (convergence around C=2).