[Paper Review] Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review

Tidor-Vlad Pricope|arXiv (Cornell University)|May 31, 2021

Stock Market Forecasting Methods26 citations

TL;DR

This paper surveys deep reinforcement learning (DRL) approaches for automated, low-frequency quantitative stock trading, highlighting promising results alongside significant gaps in real-time testing and profitability validation.

ABSTRACT

Algorithmic stock trading has become a staple in today's financial market, the majority of trades being now fully automated. Deep Reinforcement Learning (DRL) agents proved to be to a force to be reckon with in many complex games like Chess and Go. We can look at the stock market historical price series and movements as a complex imperfect information environment in which we try to maximize return - profit and minimize risk. This paper reviews the progress made so far with deep reinforcement learning in the subdomain of AI in finance, more precisely, automated low-frequency quantitative stock trading. Many of the reviewed studies had only proof-of-concept ideals with experiments conducted in unrealistic settings and no real-time trading applications. For the majority of the works, despite all showing statistically significant improvements in performance compared to established baseline strategies, no decent profitability level was obtained. Furthermore, there is a lack of experimental testing in real-time, online trading platforms and a lack of meaningful comparisons between agents built on different types of DRL or human traders. We conclude that DRL in stock trading has showed huge applicability potential rivalling professional traders under strong assumptions, but the research is still in the very early stages of development.

Motivation & Objective

• Assess how DRL has been applied to end-to-end trading agents in low-frequency stock markets.
• Categorize methods into critic-only, actor-only, and actor-critic DRL approaches and summarize their strengths/weaknesses.
• Evaluate data quality, observability, exploration-exploitation challenges, and reward designs in financial DRL.
• Identify gaps between proof-of-concept results and real-time, live trading viability.

Proposed method

• Classify reviewed studies by RL paradigm: critic-only, actor-only, and actor-critic.
• Summarize core algorithms used (DQN/DRQN, GRU-DQN, GDQN, GDPG, PPO, A2C, DDPG).
• Discuss reward functions and risk metrics (e.g., Sharpe ratio, Sortino ratio).
• Compare evaluation settings, datasets, and performance metrics across studies.
• Highlight limitations such as small action spaces, lack of transaction-cost modeling, and absence of broad real-time testing.
• Provide synthesis on potential for real-time deployment and market-beating performance.

Experimental results

Research questions

• RQ1 Can DRL-based trading agents achieve super-human performance on minute-to-daily timeframes under realistic constraints?
• RQ2 How do critic-only, actor-only, and actor-critic DRL methods compare in live-like trading environments?
• RQ3 What are the main barriers to real-time online testing and deployment of DRL trading agents?
• RQ4 What role do reward design and risk measures play in shaping DRL trading performance?

Key findings

• DRL shows potential for trading on minute to daily timeframes, with prototype systems achieving notable performance in selected markets.
• Critic-only approaches (e.g., DQN) are common but face challenges with continuous action and state spaces, and reward sensitivity.
• Actor-only methods offer continuous action spaces but can require more data and longer training times.
• Actor-critic approaches (PPO, A2C, DDPG) provide robustness to changing data distributions and instability during learning.
• Ensemble DRL strategies (combining PPO, A2C, DDPG) on Dow Jones stocks show robust performance and risk management advantages over single-model approaches.
• Across studies, many are proof-of-concept with limited real-time testing, and reported profitability varies; real-time online deployment and direct human-trader comparisons are still scarce.