[Paper Review] AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework

Zihang Zeng, Jiaquan Zhang+3 more|arXiv (Cornell University)|Mar 3, 2026

Scientific Computing and Data Management0 citations

TL;DR

The paper proposes a Bayesian adversarial multi-agent framework (Task Manager, Solution Generator, Evaluator) implemented as a low-code platform to improve robust AI-for-Science code generation across diverse LLMs by co-evolving code, tests, and prompts.

ABSTRACT

Large Language Models (LLMs) demonstrate potentials for automating scientific code generation but face challenges in reliability, error propagation in multi-agent workflows, and evaluation in domains with ill-defined success metrics. We present a Bayesian adversarial multi-agent framework specifically designed for AI for Science (AI4S) tasks in the form of a Low-code Platform (LCP). Three LLM-based agents are coordinated under the Bayesian framework: a Task Manager that structures user inputs into actionable plans and adaptive test cases, a Code Generator that produces candidate solutions, and an Evaluator providing comprehensive feedback. The framework employs an adversarial loop where the Task Manager iteratively refines test cases to challenge the Code Generator, while prompt distributions are dynamically updated using Bayesian principles by integrating code quality metrics: functional correctness, structural alignment, and static analysis. This co-optimization of tests and code reduces dependence on LLM reliability and addresses evaluation uncertainty inherent to scientific tasks. LCP also streamlines human-AI collaboration by translating non-expert prompts into domain-specific requirements, bypassing the need for manual prompt engineering by practitioners without coding backgrounds. Benchmark evaluations demonstrate LCP's effectiveness in generating robust code while minimizing error propagation. The proposed platform is also tested on an Earth Science cross-disciplinary task and demonstrates strong reliability, outperforming competing models.

Motivation & Objective

• Address reliability and error propagation in multi-agent LLM code generation for scientific tasks.
• Enable non-experts to convert vague domain prompts into executable, domain-consistent requirements.
• Co-evolve code, test cases, and prompts using a non-LLM Bayesian updating rule to reduce dependency on any single model.
• Demonstrate robustness and cross-domain effectiveness on Earth Science and AI-for-Science benchmarks across multiple base models.

Proposed method

• Three-agent architecture: Task Manager (Challenger), Solution Generator (Solver), and Evaluator that iteratively co-optimizes plans, test cases, and code.
• Bayesian updating of prompts: p(Prompt^{t+1}_{ij}|S_3^t) ∝ p(S_3^t|Prompt^{t}_{ij}) p(Prompt^{t}_{ij}), enabling recursive refinement without relying on a single LLM.
• A priori estimation with Bayesian Optimization: embed generated codes via AST/code embeddings and predict performance based on structural similarity to tested codes to guide expensive evaluations.
• Adversarial test case generation (ATC): TM crafts challenging yet solvable test cases to push SG, improving robustness and reducing error propagation.
• Iterative evaluation framework: compute test case scores S1, code scores S2, and prompt scores S3 to drive Bayesian prompt updates and selection of candidate prompts.
• Sample code pool management: maintain and expand a pool of sample codes with high-guidance quality, incorporating new high-performing codes from SG.

Figure 1: Comparison between three code generation paradigms: Single LLM generator, multi-agent role playing and the proposed Bayesian adversarial multi-agent framework.

Experimental results

Research questions

• RQ1 Can a Bayesian adversarial multi-agent framework improve reliability and robustness of AI-for-Science code generation across diverse LLMs?
• RQ2 Does an adversarial test-case generation mechanism mitigate error propagation in multi-agent code generation pipelines?
• RQ3 How does the framework perform relative to state-of-the-art baselines on AI-for-Science benchmarks and general code-generation benchmarks?
• RQ4 Can non-expert domain users benefit from the low-code platform to translate vague prompts into executable scientific workflows without expert prompt engineering?
• RQ5 What is the impact of Bayesian iterative prompt updates on solution quality over multiple iterations?

Key findings

• The framework yields robust solutions and reduces error propagation across base models from 1.7B to 235B, with notable gains on Earth Science benchmarks.
• On SciCode, smaller open-source models with the framework reach performance closer to or surpass larger models in certain settings (e.g., Qwen3-14b with framework matching larger baselines in some cases).
• On ScienceAgentBench with GPT-4o, the framework achieves state-of-the-art Valid Execution Rate (VER) and competitive SR/CBS scores.
• Iterative Bayesian co-updating improves performance over iterations, with ATC (Adversarial Test Cases) providing additional gains in later iterations.
• The framework exhibits robustness to prompt quality, narrowing the gap between basic prompts and expert prompts, and enabling non-experts to achieve strong results.

Figure 2: Overview of the Bayesian adversarial multi-agent framework. The three red arrows indicate fusion of the user-approved plan, test cases, and codes into prompts, the distribution of which is recursively updated under the Bayesian framework. $S_{1}$ , $S_{2}$ and $S_{3}$ are the scores comput