Skip to main content
QUICK REVIEW

[论文解读] BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments

Yusuf Roohani, Lee, Andrew|arXiv (Cornell University)|May 27, 2024
Evolutionary Algorithms and ApplicationsComputer Science被引用 15
一句话总结

BioDiscoveryAgent 使用基于大模型的代理,具备文献访问和批评工具,设计遗传扰动实验,在五轮内在单基因情境中比贝叶斯优化基线多约18%命中率,且在两基因扰动中准确度翻倍。

ABSTRACT

Agents based on large language models have shown great potential in accelerating scientific discovery by leveraging their rich background knowledge and reasoning capabilities. In this paper, we introduce BioDiscoveryAgent, an agent that designs new experiments, reasons about their outcomes, and efficiently navigates the hypothesis space to reach desired solutions. We demonstrate our agent on the problem of designing genetic perturbation experiments, where the aim is to find a small subset out of many possible genes that, when perturbed, result in a specific phenotype (e.g., cell growth). Utilizing its biological knowledge, BioDiscoveryAgent can uniquely design new experiments without the need to train a machine learning model or explicitly design an acquisition function as in Bayesian optimization. Moreover, BioDiscoveryAgent, using Claude 3.5 Sonnet, achieves an average of 21% improvement in predicting relevant genetic perturbations across six datasets, and a 46% improvement in the harder task of non-essential gene perturbation, compared to existing Bayesian optimization baselines specifically trained for this task. Our evaluation includes one dataset that is unpublished, ensuring it is not part of the language model's training data. Additionally, BioDiscoveryAgent predicts gene combinations to perturb more than twice as accurately as a random baseline, a task so far not explored in the context of closed-loop experiment design. The agent also has access to tools for searching the biomedical literature, executing code to analyze biological datasets, and prompting another agent to critically evaluate its predictions. Overall, BioDiscoveryAgent is interpretable at every stage, representing an accessible new paradigm in the computational design of biological experiments with the potential to augment scientists' efficacy.

研究动机与目标

  • 需要高效识别在不训练新模型的情况下产生期望表型的基因扰动。
  • 开发一个基于 LLM 的代理,设计遗传扰动实验并在闭环中推理结果。
  • 利用生物医学文献检索、基因特征查询,以及 AI 批评者来增强解释和人机交互反馈。
  • 展示代理在多个数据集上改进命中发现,并扩展到基因对扰动。

提出的方法

  • 将一个 LLM(Claude v1 Anthropic)作为中心决策者,在多轮中生成基因扰动计划。
  • 在每一轮中,向 LLM 提供任务描述和前一轮结果的提示;LLM 输出下一步要扰动的基因集合。
  • 结合工具:通过 PubMed API 的文献检索,提供前5条摘要和行引用的引文;基于基因特征的相似性/差异性搜索;一个 AI 批评者重新评估预测。
  • 采用两步基因选择策略:允许最初不受限制的建议,然后在需要时对剩余基因进行汇总,以拓宽搜索。
  • 提供结构化的响应格式(反思、研究计划、解决方案)以促进决策的可解释性和可追溯性。
  • 用每轮模拟扰动128个基因、五轮,在与多种 ML 基线和随机策略比较中进行评估。
Figure 1: An AI agent for closed-loop experiment design. (a) Conventional machine learning approach for experiment design involves training a machine learning model in every experimental round, scoring all perturbations and defining an acquisition function for selecting genes to perturb in the next
Figure 1: An AI agent for closed-loop experiment design. (a) Conventional machine learning approach for experiment design involves training a machine learning model in every experimental round, scoring all perturbations and defining an acquisition function for selecting genes to perturb in the next

实验结果

研究问题

  • RQ1一个基于 LLM 的代理是否能够设计遗传扰动实验,在识别命中方面超越传统基于获取函数的基线?
  • RQ2整合文献检索、特征引导的基因相似性和 AI 批评是否提升闭环实验设计?
  • RQ3BioDiscoveryAgent 在单基因和组合(2 基因)扰动任务在真实数据集上的表现如何?
  • RQ4在多轮中,先验知识与观测结果对决策质量的贡献有多大?

主要发现

  • BioDiscoveryAgent 配合全部工具在五个单基因数据集的第5轮后命中率平均领先基线18%。
  • 聚焦于非必需基因时,性能提升至相对于基线的29%。
  • 对于两基因扰动,BioDiscoveryAgent 在五轮累计命中上比随机采样高出130%。
  • 使用所有工具时表现最好;No-Tools 变体在大多数数据集上落后。
  • 代理能够给出可解释的预测,并带有文献引用和通路层面的推理(如 IFNG 调控、线粒体通路等)。
  • Prompt+Observation(同时使用任务描述和前一轮结果)在决策方面优于仅 Prompt 或仅 Observation,尤其是在早期轮次。
Figure 2: BioDiscoveryAgent workflow with all tools over a single experimental round . Prompts and agent responses have been summarized. See Appendix F for full trace. (a) The input to the agent is the description of the problem. (b) In case of the literature search tool, the LLM first determines ap
Figure 2: BioDiscoveryAgent workflow with all tools over a single experimental round . Prompts and agent responses have been summarized. See Appendix F for full trace. (a) The input to the agent is the description of the problem. (b) In case of the literature search tool, the LLM first determines ap

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。