[Paper Review] Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning

Minxuan Hu, Ziheng Chen+2 more|arXiv (Cornell University)|Feb 1, 2026

Risk and Portfolio Optimization0 citations

TL;DR

The paper introduces two shortfall-aware reinforcement learning frameworks, Adaptive-QLBS and Replication Learning of Option Pricing (RLOP), to enhance hedging under market frictions, showing reduced tail risk and lower trading costs on SPY and XOP options.

ABSTRACT

The deployment of autonomous AI agents in derivatives markets has widened a practical gap between static model calibration and realized hedging outcomes. We introduce two reinforcement learning frameworks, a novel Replication Learning of Option Pricing (RLOP) approach and an adaptive extension of Q-learner in Black-Scholes (QLBS), that prioritize shortfall probability and align learning objectives with downside sensitive hedging. Using listed SPY and XOP options, we evaluate models using realized path delta hedging outcome distributions, shortfall probability, and tail risk measures such as Expected Shortfall. Empirically, RLOP reduces shortfall frequency in most slices and shows the clearest tail-risk improvements in stress, while implied volatility fit often favors parametric models yet poorly predicts after-cost hedging performance. This friction-aware RL framework supports a practical approach to autonomous derivatives risk management as AI-augmented trading systems scale.

Motivation & Objective

• Address the misalignment between static pricing calibration and real hedging performance in derivatives markets.
• Develop reinforcement learning frameworks that optimize shortfall probability rather than replication error.
• Integrate transaction costs and market frictions into hedging decision processes.

Proposed method

• Model option hedging as an MDP with state X_t and hedging action a_t, incorporating self-financing constraints and transaction costs.
• Extend QLBS to Adaptive-QLBS by making the value function adapted to filtration and introducing a backward, discounting structure for portfolio value.
• Introduce Replication Learning of Option Pricing (RLOP), a forward, replication-based RL approach that emphasizes minimizing shortfalls at maturity.
• Parametrize hedging policies with neural networks (ResNet-style) and train via REINFORCE with a baseline using simulated geometric Brownian motion paths.
• Evaluate hedging performance on realized-path distributions under transaction costs, using metrics like PnL net, shortfall probability, and Expected Shortfall (ES).

Experimental results

Research questions

• RQ1 Does integrating shortfall probability into the RL reward structure improve hedging stability under frictions?
• RQ2 How do Adaptive-QLBS and RLOP perform relative to parametric models (BS, JD, SV) in terms of tail risk and execution costs?
• RQ3 Can RL-based hedging policies reduce turnover while maintaining or improving downside protection across regimes?

Key findings

• RLOP reduces shortfall frequency in most slices and shows the clearest tail-risk improvements in stress conditions.
• Adaptive-QLBS and RLOP reveal that IVRMSE-based diagnostics favor parametric models for static pricing, but RL policies improve realized-path hedging under costs.
• RL policies consistently achieve a systematic cost advantage by lowering trading turnover under the same daily rebalancing schedule.
• Tail risk analyses (ES at 5% and 10%, and shortfall probability) indicate RL methods, especially RLOP, reduce extreme after-cost losses in stressed regimes (e.g., 2020Q1).
• QLBS tends to be a replication-oriented stabilizer, while RLOP emphasizes implementability and downside control under frictions.