[Paper Review] Large Language Model Agent in Financial Trading: A Survey

Han Ding, Yinheng Li+4 more|arXiv (Cornell University)|Jul 26, 2024

Stock Market Forecasting Methods9 citations

TL;DR

This paper surveys how LLMs are used as trading agents or alpha miners in financial markets, summarizing architectures, data inputs, backtest performance, and challenges.

ABSTRACT

Trading is a highly competitive task that requires a combination of strategy, knowledge, and psychological fortitude. With the recent success of large language models(LLMs), it is appealing to apply the emerging intelligence of LLM agents in this competitive arena and understanding if they can outperform professional traders. In this survey, we provide a comprehensive review of the current research on using LLMs as agents in financial trading. We summarize the common architecture used in the agent, the data inputs, and the performance of LLM trading agents in backtesting as well as the challenges presented in these research. This survey aims to provide insights into the current state of LLM-based financial trading agents and outline future research directions in this field.

Motivation & Objective

• Identify common architectures for LLM-based trading agents (LLM as Trader vs LLM as Alpha Miner) and their rationales.
• Catalog data inputs used by LLM trading agents (numerical, textual, visual, simulated) and how they are processed.
• Review backtesting performance, evaluation metrics, and baselines to assess LLM agents in finance.
• Highlight limitations, current trends in LLM selection, and propose avenues for future research.

Proposed method

• Systematically review 27 papers on LLMs for financial trading.
• Categorize architectures into two main types: LLM as Trader and LLM as Alpha Miner.
• Summarize data inputs into four groups: numerical, textual, visual, simulated.
• Summarize evaluation methods including backtesting, metrics, and baselines.
• Discuss limitations and future directions, including model accessibility and integration challenges.

Figure 2. Histogram of base LLM used by Finance Agent (one paper may contain multiple agent)

Experimental results

Research questions

• RQ1 What are the common architectures used for LLM-powered trading agents?
• RQ2 What data types are utilized by LLM trading agents and how are they integrated?
• RQ3 What is the current performance and what are the limitations of LLM trading agents in backtesting?
• RQ4 What future directions and challenges are emphasized for LLM-based financial trading?

Key findings

• LLM architectures for trading are broadly categorized as LLM as a Trader and LLM as an Alpha Miner.
• Agents primarily rely on textual data, complemented by numerical data converted to text, and increasingly explore memory, reflection, and debate mechanisms.
• Backtesting on real market data shows strong performance with annualized returns reported in the 15%–30% range relative to baselines.
• Open-source LLMs exist in the literature but OpenAI models (GPT-3.5/4) dominate usage due to performance, while latency and privacy concerns persist with closed-source models.
• Multimodal data (including visual data like charts) is being explored, but financial visual data remains underutilized.
• Evaluation often lacks consideration of trading costs and broader market coverage beyond US/Chinese stock markets, signaling areas for more comprehensive testing.