[Paper Review] CellAgent: An LLM-driven Multi-Agent Framework for Automated Single-cell Data Analysis

Yihang Xiao, Jinyi Liu+10 more|arXiv (Cornell University)|Jul 13, 2024

Gene Regulatory Network Analysis6 citations

TL;DR

CellAgent is an LLM-driven multi-agent framework that autonomously executes end-to-end single-cell RNA-seq analysis by coordinating Planner, Executor, and Evaluator roles, with self-iterative optimization to ensure high-quality results.

ABSTRACT

Single-cell RNA sequencing (scRNA-seq) data analysis is crucial for biological research, as it enables the precise characterization of cellular heterogeneity. However, manual manipulation of various tools to achieve desired outcomes can be labor-intensive for researchers. To address this, we introduce CellAgent (http://cell.agent4science.cn/), an LLM-driven multi-agent framework, specifically designed for the automatic processing and execution of scRNA-seq data analysis tasks, providing high-quality results with no human intervention. Firstly, to adapt general LLMs to the biological field, CellAgent constructs LLM-driven biological expert roles - planner, executor, and evaluator - each with specific responsibilities. Then, CellAgent introduces a hierarchical decision-making mechanism to coordinate these biological experts, effectively driving the planning and step-by-step execution of complex data analysis tasks. Furthermore, we propose a self-iterative optimization mechanism, enabling CellAgent to autonomously evaluate and optimize solutions, thereby guaranteeing output quality. We evaluate CellAgent on a comprehensive benchmark dataset encompassing dozens of tissues and hundreds of distinct cell types. Evaluation results consistently show that CellAgent effectively identifies the most suitable tools and hyperparameters for single-cell analysis tasks, achieving optimal performance. This automated framework dramatically reduces the workload for science data analyses, bringing us into the "Agent for Science" era.

Motivation & Objective

• Automate end-to-end scRNA-seq data analysis without human intervention.
• Leverage specialized biological expert roles to plan, execute, and evaluate analyses.
• Enable hierarchical planning and self-iterative optimization to improve outputs.

Proposed method

• Introduce three LLM-driven biological expert roles: Planner (high-level task planning), Executor (subtask execution and code generation), and Evaluator (quality assessment and optimization).
• Implement a hierarchical decision-making mechanism to coordinate Planner and Executors across subtasks.
• Incorporate a self-iterative optimization loop where Evaluator guides Executor to refine plans, with exception handling for code execution.
• Provide a memory and tool-retrieval system to manage history and available analysis tools, executed in a code sandbox for safety.
• Utilize GPT-4V for evaluating batch correction and trajectory visualization, and GPT-4 for aggregating cell type annotations from multiple tools.

Experimental results

Research questions

• RQ1 Can CellAgent autonomously decompose and execute complex scRNA-seq analysis tasks from natural language inputs?
• RQ2 Does the multi-agent collaboration improve task completion rate and result quality compared to single-model baselines?
• RQ3 How do hierarchical planning and self-iterative optimization affect preprocessing, batch correction, cell type annotation, and trajectory inference?
• RQ4 What is the impact of integrated tools, memory, and code sandboxing on robustness and reproducibility of results.

Key findings

• CellAgent achieved a 92% comprehensive task completion rate across the benchmark, outperforming GPT-4 alone.
• On batch correction tasks, CellAgent achieved top scores in both batch correction and bio-conservation across nine datasets.
• CellAgent showed superior average accuracy in cell type annotation across multiple tissues and organisms, with high agreement to expert annotations on PBMC data.
• In trajectory inference, CellAgent achieved the best overall score among compared methods and demonstrated biologically interpretable trajectories.
• The framework consistently identified suitable tools and hyperparameters for single-cell analyses, matching or surpassing existing tools in several tasks.