[Paper Review] AutoFlow: Automated Workflow Generation for Large Language Model Agents

Zelong Li, Shuyuan Xu+7 more|arXiv (Cornell University)|Jul 1, 2024

Business Process Modeling and Analysis4 citations

TL;DR

AutoFlow is a framework that automatically generates natural language workflows for LLM-based agents using reinforcement learning, enabling effective and reliable task execution. It achieves over 40% performance improvement over baselines like CoRE when using open-source LLMs, demonstrating superior efficiency and readability in complex task solving.

ABSTRACT

Recent advancements in Large Language Models (LLMs) have shown significant progress in understanding complex natural language. One important application of LLM is LLM-based AI Agent, which leverages the ability of LLM as well as external tools for complex-task solving. To make sure LLM Agents follow an effective and reliable procedure to solve the given task, manually designed workflows are usually used to guide the working mechanism of agents. However, manually designing the workflows requires considerable efforts and domain knowledge, making it difficult to develop and deploy agents on massive scales. To address these issues, we propose AutoFlow, a framework designed to automatically generate workflows for agents to solve complex tasks. AutoFlow takes natural language program as the format of agent workflow and employs a workflow optimization procedure to iteratively optimize the workflow quality. Besides, this work offers two workflow generation methods: fine-tuning-based and in-context-based methods, making the AutoFlow framework applicable to both open-source and closed-source LLMs. Experimental results show that our framework can produce robust and reliable agent workflows. We believe that the automatic generation and interpretation of workflows in natural language represent a promising paradigm for solving complex tasks, particularly with the rapid development of LLMs. The source code of this work is available at https://github.com/agiresearch/AutoFlow.

Motivation & Objective

• Address the high human effort and domain expertise required to manually design reliable workflows for LLM-based agents.
• Enable scalable deployment of LLM agents by automating workflow creation without manual engineering.
• Support both open-source and closed-source LLMs through dual generation methods: fine-tuning and in-context learning.
• Improve workflow quality through iterative reinforcement learning optimization using performance rewards.
• Maintain human readability of generated workflows while enhancing task-solving performance over manually designed counterparts.

Proposed method

• Represents agent workflows as natural language programs for interpretability and ease of human understanding.
• Employs a two-model setup: a generator LLM to produce workflows and a frozen interpreter LLM to execute and evaluate them.
• Uses reinforcement learning (REINFORCE) with performance metrics (e.g., CLIP, BERT, ViT scores) as rewards to fine-tune the generator LLM over 30 iterations.
• Applies Low-Rank Adaptation (LoRA) with rank 8 for efficient fine-tuning of open-source LLMs like Mixtral.
• Supports in-context learning for closed-source LLMs (e.g., GPT-4) without parameter updates, enabling broad compatibility.
• Iteratively refines workflows by optimizing the generator based on feedback from the interpreter’s execution and evaluation.

Experimental results

Research questions

• RQ1 Can natural language workflows be automatically generated by LLMs to replace manually designed workflows in LLM agents?
• RQ2 How effective is reinforcement learning in optimizing LLM-generated workflows for complex task execution?
• RQ3 Can the AutoFlow framework achieve better performance than human-designed or existing baseline workflows?
• RQ4 Does combining different LLMs as generator and interpreter (e.g., Mixtral + GPT-4) yield synergistic performance gains?
• RQ5 To what extent do generated workflows remain interpretable and readable to humans while improving task performance?

Key findings

• AutoFlow outperforms the best baseline, CoRE, by over 40% in average performance when using Mixtral as the interpreter LLM.
• With GPT-4 as the interpreter, AutoFlow achieves a 5% improvement over CoRE, demonstrating consistent superiority.
• The highest average score on the OpenAGI benchmark was achieved by AutoFlow using GPT-4 as the generator and Mixtral as the interpreter.
• AutoFlow-generated workflows achieved the highest scores across all task types in both tables, indicating robustness across diverse tasks.
• The framework maintains high readability of generated workflows while significantly reducing manual design effort.
• The synergistic combination of different LLMs (e.g., Mixtral for generator, GPT-4 for interpreter) leads to improved performance, suggesting complementary strengths.