[Paper Review] When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments

Chong Zhang, Xinyi Liu+11 more|arXiv (Cornell University)|Jul 15, 2024

Stock Market Forecasting MethodsDecision Sciences3 citations

TL;DR

This paper introduces StockAgent, a multi-agent LLM-based framework that simulates realistic stock trading environments to study how external factors like macroeconomics, policy changes, and market events influence AI-driven trading behaviors. By preventing test data leakage and leveraging LLMs to model diverse investor strategies, StockAgent reveals distinct behavioral patterns—such as reduced herding and increased strategy diversity—especially between GPT and Gemini agents, offering insights into LLM-based investment reliability and personalization.

ABSTRACT

Can AI Agents simulate real-world trading environments to investigate the impact of external factors on stock trading activities (e.g., macroeconomics, policy changes, company fundamentals, and global events)? These factors, which frequently influence trading behaviors, are critical elements in the quest for maximizing investors' profits. Our work attempts to solve this problem through large language model based agents. We have developed a multi-agent AI system called StockAgent, driven by LLMs, designed to simulate investors' trading behaviors in response to the real stock market. The StockAgent allows users to evaluate the impact of different external factors on investor trading and to analyze trading behavior and profitability effects. Additionally, StockAgent avoids the test set leakage issue present in existing trading simulation systems based on AI Agents. Specifically, it prevents the model from leveraging prior knowledge it may have acquired related to the test data. We evaluate different LLMs under the framework of StockAgent in a stock trading environment that closely resembles real-world conditions. The experimental results demonstrate the impact of key external factors on stock market trading, including trading behavior and stock price fluctuation rules. This research explores the study of agents' free trading gaps in the context of no prior knowledge related to market data. The patterns identified through StockAgent simulations provide valuable insights for LLM-based investment advice and stock recommendation. The code is available at https://github.com/MingyuJ666/Stockagent.

Motivation & Objective

• To develop a simulation framework that models real-world stock trading dynamics without relying on historical data leakage.
• To investigate how external factors—such as economic indicators, policy changes, and company reports—affect AI agent trading behaviors and profitability.
• To evaluate differences in trading strategies and decision-making tendencies between distinct LLMs (e.g., GPT vs. Gemini) in a controlled, event-driven market environment.
• To enable interpretable, customizable agent-based experiments for testing investment strategies, sentiment dynamics, and risk exposure in simulated markets.
• To provide a foundation for assessing the reliability and bias profiles of LLM-based stock recommendations and quantitative trading systems.

Proposed method

• Design a multi-agent AI system where each agent is driven by a large language model (LLM), simulating individual investors with distinct decision-making styles.
• Implement an event-driven simulation engine that models real-world market conditions, including news events, earnings reports, interest rate changes, and lending mechanisms.
• Introduce a test data isolation mechanism to prevent LLMs from accessing or memorizing future market data, eliminating test set leakage.
• Use LLMs to generate trading decisions based on contextual market information, external events, and agent-specific roles (e.g., growth vs. value investor).
• Integrate customizable prompts and role definitions to control agent behavior and simulate different investment styles and risk preferences.
• Enable dynamic interaction between agents through shared information channels (e.g., BBS for sentiment sharing) while allowing for selective disabling to study behavioral impacts.

Figure 2. The workflow of trading simulation.

Experimental results

Research questions

• RQ1 How do external factors such as macroeconomic shifts, policy changes, and company fundamentals influence the trading behaviors and profitability of LLM-based AI agents in a simulated market?
• RQ2 To what extent do inherent biases and reasoning patterns in different LLMs (e.g., GPT vs. Gemini) shape distinct trading strategies and performance outcomes?
• RQ3 How does the absence of prior knowledge about test data affect the emergence of systematic trading behaviors, such as herding or trend-following, in LLM-driven agents?
• RQ4 What is the impact of disabling specific information channels (e.g., BBS, loan functions) on agent risk-taking, profit variance, and market competition?
• RQ5 Can LLM-based agents simulate personalized investment styles and diverse strategies without relying on historical data, and how does this affect simulation realism?

Key findings

• GPT-driven agents exhibited more independent and diverse trading behaviors compared to Gemini agents, with reduced herding and trend-following tendencies.
• Disabling non-first-round loan functions and BBS information sharing led to more conservative trading behavior across agents, indicating that information and leverage mechanisms shape risk exposure.
• Removing profit reports or interest rate change events caused some agents to shift from loss to profit, highlighting the critical role of fundamental data in agent performance.
• Eliminating access to any financial information increased profit and loss variance between agents, indicating heightened market competition and reduced predictability.
• Despite exposure to the same external environment, LLM agents retained distinct intrinsic trading tendencies, suggesting that model-specific biases persist even in context-aware, one-step decisions.
• The simulation revealed that LLM-based agents can embody personalized investment styles, supporting their use in modeling heterogeneous investor behavior and testing strategy robustness.

Figure 3. The demonstration of stock market investment. In our simulation, agents make investment decisions based on multiple external sources of information.