[Paper Review] MDCrow: Automating Molecular Dynamics Workflows with Large Language Models

Quintina L. Campbell, Sam Cox+3 more|ArXiv.org|Feb 13, 2025

Machine Learning in Materials Science14 citations

TL;DR

MDCrow is an LLM-driven agent that autonomously automates molecular dynamics workflows using a toolset of 40+ MD-related utilities, evaluated across 25 tasks and multiple base models. It shows strong performance with gpt-4o and llama3-405b, and robustness across prompt styles for larger models.

ABSTRACT

Molecular dynamics (MD) simulations are essential for understanding biomolecular systems but remain challenging to automate. Recent advances in large language models (LLM) have demonstrated success in automating complex scientific tasks using LLM-based agents. In this paper, we introduce MDCrow, an agentic LLM assistant capable of automating MD workflows. MDCrow uses chain-of-thought over 40 expert-designed tools for handling and processing files, setting up simulations, analyzing the simulation outputs, and retrieving relevant information from literature and databases. We assess MDCrow's performance across 25 tasks of varying required subtasks and difficulty, and we evaluate the agent's robustness to both difficulty and prompt style. exttt{gpt-4o} is able to complete complex tasks with low variance, followed closely by exttt{llama3-405b}, a compelling open-source model. While prompt style does not influence the best models' performance, it has significant effects on smaller models.

Motivation & Objective

• Demonstrate a fully autonomous LLM-agent (MDCrow) that can complete MD workflows from setup to analysis.
• Assess MDCrow performance across tasks of varying difficulty and subtasks.
• Evaluate robustness of MDCrow to different prompt styles and base LLM choices.
• Compare MDCrow to baselines and analyze its ability to retrieve literature and manage long-running simulations.

Proposed method

• Construct MDCrow as an LLM agent using LangChain with a ReAct-style prompt and over 40 specialized MD tools.
• Categorize tools into Information Retrieval, PDB & Protein handling, Simulation, and Analysis.
• Use OpenMM and MDTraj-based workflows for simulation setup, execution, and analysis; generate Python scripts for adaptation.
• Implement a chat-resume feature that saves run context and files in a checkpoint directory to resume sessions.
• Evaluate performance across 25 prompts with varying subtasks and multiple base LLMs (gpt-3.5-turbo, gpt-4-turbo, gpt-4o, llama, Claude).
• Compare MDCrow against a ReAct baseline (Python REPL) and a single-query LLM.

Figure 1: A. MDCrow workflow. Starting with a user prompt and initialized with a set of MD tools, MDCrow follows a chain-of-thought process until it completes all tasks in the prompt. The final output includes a response, along with all resulting analyses and files. B . The tool distribution categor

Experimental results

Research questions

• RQ1 Can MDCrow autonomously complete MD workflows from data retrieval to analysis?
• RQ2 How does MDCrow performance vary with task complexity (number of subtasks) and different base LLMs?
• RQ3 How robust is MDCrow to prompt style variations and model type?
• RQ4 Does MDCrow outperform baseline approaches in completing MD tasks and handling file management and errors?
• RQ5 To what extent can MDCrow extrapolate to tasks outside its explicit toolset via its chatting feature?

Key findings

• MDCrow completes a majority of tasks, with best performance using gpt-4o, achieving higher accuracy and subtasks completed than baselines.
• Open-source llama3-405b provides a compelling alternative with strong performance and robustness.
• Performance generally declines with task complexity for weaker models, while gpt-4o and llama-405b show stable performance across subtasks.
• Prompt style significantly affects smaller models but has less impact on top-performing models.
• MDCrow with optimized settings reaches about 72% task accuracy in their evaluation, and the system is robust to variations in instruction style for strong models.

Figure 2: Example Chat Example of chat with MDCrow. The user first asks to download PDB files for two systems. Then, once MDCrow has completed this task, the user asks for analysis of the files. Next, the user asks for a quick 10 ps simulation of both files, and MDCrow saves all files for later hand