[Paper Review] DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration

Sizhe Liu, Yizhou Lu+5 more|arXiv (Cornell University)|Nov 24, 2024

Scientific Computing and Data Management7 citations

TL;DR

DrugAgent uses a multi-agent LLM framework to automate ML programming in drug discovery, with domain-specific tooling and dynamic idea-space management; a case study achieves an F1 score of 0.92 for ADMET absorption prediction on PAMPA data.

ABSTRACT

Recent advancements in Large Language Models (LLMs) have opened new avenues for accelerating drug discovery processes. Despite their potential, several critical challenges remain unsolved, particularly in translating theoretical ideas into practical applications within the highly specialized field of pharmaceutical research, limiting practitioners from leveraging the latest AI development in drug discovery. To this end, we introduce DrugAgent, a multi-agent framework aimed at automating machine learning (ML) programming in drug discovery. DrugAgent incorporates domain expertise by identifying specific requirements and building domain-specific tools, while systematically exploring different ideas to find effective solutions. A preliminary case study demonstrates DrugAgent's potential to overcome key limitations LLMs face in drug discovery, moving toward AI-driven innovation. For example, DrugAgent is able to complete the ML programming pipeline end-to-end, from data acquisition to performance evaluation for the ADMET prediction task, and finally select the best model, where the random forest model achieves an F1 score of 0.92 when predicting absorption using the PAMPA dataset.

Motivation & Objective

• Bridge the gap between general-purpose LLM reasoning and domain-specific drug discovery needs.
• Automate ML programming tasks in drug discovery from data acquisition to model evaluation without human coding.
• Introduce domain-specific tools and an idea-space management strategy to improve exploration efficiency.
• Demonstrate end-to-end automation on ADMET prediction and compare against general-purpose frameworks.

Proposed method

• Introduce an automated LLM-based multi-agent system for drug discovery ML programming.
• Incorporate an LLM Instructor to identify domain knowledge needs and prepare tools.
• Use an LLM Planner to manage and refine an idea space via generation and pruning.
• Develop domain-specific tools with unit tests and a reusable toolbox for data acquisition, fingerprinting, and model evaluation.
• Demonstrate an end-to-end pipeline from data acquisition to model evaluation, selecting the best model.

Figure 1: Framework overview of DrugAgent. Given an AI-based drug discovery task described in natural language (i.e., user’s input, e.g., design an AI model to predict Absorption (one of the ADMET properties) using the PAMPA dataset (Siramshetty, Shah et al. 2021 ) , the LLM Planner first produces a

Experimental results

Research questions

• RQ1 How can domain-specific knowledge be explicitly identified and integrated into LLM-driven ML programming for drug discovery?
• RQ2 Can a multi-agent framework systematically explore and prune infeasible or suboptimal ideas to improve automation efficiency?
• RQ3 How does DrugAgent perform on standard AI-driven drug discovery tasks (ADMET, DTI, molecule optimization) compared to general-purpose baselines?

Key findings

• DrugAgent can automate end-to-end ML programming for ADMET prediction on PAMPA data.
• A random forest model achieved F1 = 0.92 and ROC-AUC = 0.817 on PAMPA absorption prediction.
• ChemBERTa achieved F1 = 0.916 and ROC-AUC = 0.776 on the same task.
• DrugAgent outperforms a general-purpose framework (ReAct) by effectively integrating domain knowledge and tool construction, reducing reliance on human intervention.
• The framework combines idea-space management with domain-tool construction to prune ineffective approaches (e.g., molecular graph construction).

Figure 2: Comparison of ReAct (a) and DrugAgent (b) on an ADMET prediction task using the PAMPA dataset. ReAct, a general-purpose framework, fails due to hallucinated API calls and an inability to self-debug, requiring human intervention to proceed. It focuses solely on fine-tuning a pretrained lang