[Paper Review] Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow

Wenqi Zhang, Yongliang Shen+4 more|arXiv (Cornell University)|Jun 12, 2023

Data Quality and Management10 citations

TL;DR

Data-Copilot is an LLM-based system that autonomously designs interfaces and dispatches workflows to manage, process, and visualize massive heterogeneous data without human assistance.

ABSTRACT

Industries such as finance, meteorology, and energy generate vast amounts of data daily. Efficiently managing, processing, and displaying this data requires specialized expertise and is often tedious and repetitive. Leveraging large language models (LLMs) to develop an automated workflow presents a highly promising solution. However, LLMs are not adept at handling complex numerical computations and table manipulations and are also constrained by a limited context budget. Based on this, we propose Data-Copilot, a data analysis agent that autonomously performs querying, processing, and visualization of massive data tailored to diverse human requests. The advancements are twofold: First, it is a code-centric agent that receives human requests and generates code as an intermediary to handle massive data, which is quite flexible for large-scale data processing tasks. Second, Data-Copilot involves a data exploration phase in advance, which explores how to design more universal and error-free interfaces for real-time response. Specifically, it actively explores data sources, discovers numerous common requests, and abstracts them into many universal interfaces for daily invocation. When deployed in real-time requests, Data-Copilot only needs to invoke these pre-designed interfaces, transforming raw data into visualized outputs (e.g., charts, tables) that best match the user's intent. Compared to generating code from scratch, invoking these pre-designed and compiler-validated interfaces can significantly reduce errors during real-time requests. Additionally, interface workflows are more efficient and offer greater interpretability than code. We open-sourced Data-Copilot with massive Chinese financial data, such as stocks, funds, and news, demonstrating promising application prospects.

Motivation & Objective

• Motivate the need to efficiently handle massive heterogeneous data across domains.
• Propose an LLM-based system that designs and deploys autonomous data management interfaces.
• Enable end-to-end data processing, analysis, forecasting, and visualization via self-designed workflows.
• Demonstrate a demonstrator in the Chinese financial domain to validate the approach.

Proposed method

• Interface Design: use a self-request process to generate diverse data-related interfaces (data acquisition, processing, prediction, visualization, DataFrame manipulation) described in natural language and code, iteratively refining them offline.
• Interface Merging: merge similar interfaces to create generalized tools, enabling a compact, reusable interface library.
• Interface Implementation: generate implementation code for each interface from the refined descriptions, enabling scalable deployment across platforms.
• Interface Dispatch: parse user intent, plan a multi-step workflow using the self-designed interfaces, and dispatch them in sequential, parallel, or loop structures in real time.
• Offline design vs online dispatch: separate offline interface design from online interface dispatch to transform raw data into user-friendly outputs.
• Multi-form Output: produce graphics, tables, and textual summaries, along with a workflow recap for transparency.

Experimental results

Research questions

• RQ1 How can an LLM autonomously design versatile interfaces to manage, process, analyze, predict, and visualize large-scale heterogeneous data?
• RQ2 Can an LLM-driven system plan and dispatch complex workflows in real time to satisfy diverse user requests across data domains?
• RQ3 To what extent can self-designed interfaces handle data from multiple sources (e.g., stocks, funds, economics, news) and present results in multiple formats?
• RQ4 What are the limitations and stability concerns of a fully autonomous, interface-driven data workflow system?

Key findings

• Data-Copilot can autonomously design a library of data interfaces and merge similar tools to generalize functionality.
• It can dispatch planned workflows that execute data acquisition, processing, prediction, and visualization in parallel, sequential, or loop structures.
• The system supports multi-form outputs (graphics, tables, text) and provides a workflow summary for transparency.
• A demonstrator using Chinese financial data shows access to stocks, funds, economic data, live news, and various visualizations.
• Experiments indicate effective planning and execution using GPT-4 for design and GPT-3.5 Turbo for dispatch, with real-time visualization of results.