[Paper Review] Flows: Building Blocks of Reasoning and Collaborating AI

Martin Josifoski, Lars Klein+9 more|arXiv (Cornell University)|Aug 2, 2023

Reinforcement Learning in Robotics44 references4 citations

TL;DR

This paper introduces Flows, a modular, message-based framework for structuring complex reasoning and collaboration in AI systems, enabling recursive composition of self-contained computational units. Evaluated on competitive coding, Flows improved solve rates by +21 (AI-only) and +54 (human–AI) points, demonstrating that systematic, structured interactions significantly enhance generalization and performance over unstructured prompting.

ABSTRACT

Recent advances in artificial intelligence (AI) have produced highly capable and controllable systems. This creates unprecedented opportunities for structured reasoning as well as collaboration among multiple AI systems and humans. To fully realize this potential, it is essential to develop a principled way of designing and studying such structured interactions. For this purpose, we introduce the conceptual framework Flows. Flows are self-contained building blocks of computation, with an isolated state, communicating through a standardized message-based interface. This modular design simplifies the process of creating Flows by allowing them to be recursively composed into arbitrarily nested interactions and is inherently concurrency-friendly. Crucially, any interaction can be implemented using this framework, including prior work on AI-AI and human-AI interactions, prompt engineering schemes, and tool augmentation. We demonstrate the potential of Flows on competitive coding, a challenging task on which even GPT-4 struggles. Our results suggest that structured reasoning and collaboration substantially improve generalization, with AI-only Flows adding +21 and human-AI Flows adding +54 absolute points in terms of solve rate. To support rapid and rigorous research, we introduce the aiFlows library embodying Flows. The aiFlows library is available at https://github.com/epfl-dlab/aiflows. Data and Flows for reproducing our experiments are available at https://github.com/epfl-dlab/cc_flows.

Motivation & Objective

• To address the lack of a principled, scalable abstraction for modeling complex, structured interactions among AI systems and humans.
• To enable systematic study of reasoning and collaboration patterns in LLM-based systems beyond ad-hoc control flows.
• To reduce complexity in designing multi-agent and human-AI workflows through modularity and isolation of computational units.
• To support rigorous, reproducible research by providing a standardized infrastructure with logging and composability.
• To demonstrate that structured interaction patterns significantly improve generalization and performance in challenging reasoning tasks.

Proposed method

• Flows are self-contained, goal-driven computational units with isolated state and standardized message-based interfaces for communication.
• Atomic Flows are built from basic tools (e.g., code execution, search), while Composite Flows compose multiple Flows into nested, hierarchical interactions.
• The framework supports recursive composition and asynchronous execution, enabling complex, dynamic workflows.
• The aiFlows library implements the Flows abstraction, providing tools for defining, composing, logging, and executing Flows.
• Experiments use a competitive coding benchmark with controlled data leakage and variance analysis to evaluate interaction patterns.
• Feedback mechanisms, including reflection and grounding, are integrated into Flows to assess refinement and collaboration quality.

Experimental results

Research questions

• RQ1 How do structured reasoning and collaboration patterns in Flows affect generalization in complex reasoning tasks?
• RQ2 What is the impact of human involvement in high-level reasoning on solve performance in competitive coding?
• RQ3 How do different refinement strategies (e.g., reflection, grounding) influence the effectiveness of AI-generated solutions?
• RQ4 To what extent is the performance gain from Flows dependent on data contamination and variance in model outputs?
• RQ5 Can Flows support meta-reasoning and autonomous behavior through recursive, asynchronous composition?

Key findings

• AI-only Flows improved the solve rate by +21 absolute points on a competitive coding benchmark, demonstrating the benefit of structured reasoning.
• Human–AI Flows achieved a +54 absolute point improvement, highlighting the significant value of human feedback in high-level reasoning.
• The effectiveness of refinement patterns like reflection and grounding is not universal and depends on the quality of feedback and solution grounding.
• Data contamination significantly affects results, and error bars must be rigorously reported to ensure valid conclusions.
• Problem decomposition is effective only when intermediate steps are of high quality and well-grounded in feedback.
• The Flows framework enables asynchronous, modular, and composable workflows, supporting meta-reasoning and scalable collaboration.