Skip to main content
QUICK REVIEW

[论文解读] InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training

Ziyun Zhang, Zezhou Wang|arXiv (Cornell University)|Jan 7, 2026
Web Data Mining and AnalysisComputer Science被引用 0
一句话总结

tldr: InfiniteWeb 自动生成具可验证评估器的功能性、多样化网页环境,用于 GUI 代理训练,相较基线在功能正确性和代理性能方面有提升。

ABSTRACT

GUI agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants. However, training such agents is hindered by the scarcity of suitable environments. We present InfiniteWeb, a system that automatically generates functional web environments at scale for GUI agent training. While LLMs perform well on generating a single webpage, building a realistic and functional website with many interconnected pages faces challenges. We address these challenges through unified specification, task-centric test-driven development, and a combination of website seed with reference design image to ensure diversity. Our system also generates verifiable task evaluators enabling dense reward signals for reinforcement learning. Experiments show that InfiniteWeb surpasses commercial coding agents at realistic website construction, and GUI agents trained on our generated environments achieve significant performance improvements on OSWorld and Online-Mind2Web, demonstrating the effectiveness of proposed system.

研究动机与目标

  • Motivate the need for scalable, diverse, and verifiable web environments to train GUI agents.
  • Propose a unified specification plus task-centric test-driven development to ensure cross-page consistency and correctness.
  • Introduce design-guided frontend and seed-based diversity to produce visually varied websites.
  • Provide automatic evaluators to yield dense reinforcement learning rewards for training agents.
  • Demonstrate improvements in website realism and GUI agent performance over baselines.

提出的方法

  • Unified Specification: derive data models and interfaces from generated tasks to ensure cross-page consistency.
  • Task-Centric Backend with Test-Driven Development (TCTDD): generate and iteratively fix task-relevant code until all tests pass.
  • Design-Guided Frontend: use a reference design image to extract visual features and guide page generation for visual diversity.
  • Evaluator Generation: create task-specific evaluators using instrumentation variables for dense RL reward signals.
  • Seed + Design Image Guidance: combine a website seed with a design image to promote functional and visual diversity.
  • Evaluation methodology: compare functional correctness and visual quality against baselines on WebGen-Bench and assess agent Training impact on OSWorld and Online-Mind2Web.

实验结果

研究问题

  • RQ1Can automated generation of end-to-end, multi-page websites deliver realistic, functional environments for GUI agent training?
  • RQ2Does a unified specification plus task-centric correctness improve cross-page consistency and reduce functional bugs in generated websites?
  • RQ3Can design images and seeds provide sufficient visual and functional diversity to prevent overfitting of GUI agents?
  • RQ4Do automatically generated evaluators provide dense, informative rewards that improve RL-based GUI agent training?
  • RQ5How well do agents trained in InfiniteWeb environments transfer to real-world web and desktop tasks?

主要发现

  • InfiniteWeb achieves the highest overall WebGen-Bench score of 85.6% among compared baselines.
  • Task-centric correctness with TCTDD improves functional accuracy by approximately 5 points.
  • Dense reward signals via instrumentation enable learning from 4.4× more tasks than binary reward approaches.
  • Training on InfiniteWeb-generated environments yields +6.9 percentage points on OSWorld and +5.7 points on Online-Mind2Web.
  • Visual quality evaluations show our method consistently wins 69–85% against baselines in LLM-as-Judge assessments.
  • Agent training on InfiniteWeb environments improves OSWorld and Online-Mind2Web performance, indicating transfer to real-world tasks.

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。