[Paper Review] Algorithmic Monitoring: Measuring Market Stress with Machine Learning

Marc Schmitt|arXiv (Cornell University)|Feb 5, 2026

Financial Markets and Investment Strategies0 citations

TL;DR

The paper builds MSPI, an equity-only, real-time, probabilistic stress measure for the U.S. stock market one month ahead, using cross-sectional fragility signals and lasso-logit, and compares it to nonlinear learners.

ABSTRACT

I construct a Market Stress Probability Index (MSPI) that estimates the probability of high stress in the U.S. equity market one month ahead using information from the cross-section of individual stocks. Using CRSP daily data, each month is summarized by a set of interpretable cross-sectional fragility signals and mapped into a forward-looking stress probability via an L1-regularized logistic regression in a real-time expanding-window design. Out of sample, MSPI tracks major stress episodes and improves discrimination and accuracy relative to a parsimonious benchmark based on lagged market return and realized volatility, delivering calibrated stress probabilities on an economically meaningful scale. Further, I illustrate how MSPI can be used as a probability-based measurement object in financial econometrics. The resulting index provides a transparent and easily updated measure of near-term equity-market stress risk.

Motivation & Objective

• Motivate the need for algorithmic, real-time market-stress monitoring in electronic markets.
• Construct an equity-only, transparent Market Stress Probability Index (MSPI) that outputs calibrated one-month-ahead stress probabilities.
• Compare a sparse lasso-logit MSPI to nonlinear learners (random forests, gradient boosting) under a real-time expanding-window protocol.
• Demonstrate MSPI’s calibration, discrimination, and relation to realized market outcomes.
• Highlight MSPI as a usable measurement object for financial econometrics.

Proposed method

• Define cross-sectional fragility signals from CRSP daily stock data (dispersion, tails, skewness, kurtosis, trading intensity).
• Aggregate signals monthly to form X_t as predictors for stress.
• Define stress monthly indicator S_t based on a return cutoff and a volatility quantile q_{t-1}(α).
• Model MSPI_t as a one-month-ahead stress probability using lasso-logit: MSPI_t = Λ(β_0 + X_t'β), with L1 regularization.
• Employ an expanding-window real-time learning protocol with initial 120-month training and fixed λ chosen via time-series cross-validation.
• Benchmark against a parsimonious benchmark using lagged market return and realized volatility with L2 regularization.
• Assess nonlinear models (random forests, gradient boosting) with Platt-calibration within each training window for probability outputs.

Figure 1: MSPI out-of-sample. The solid line plots the predicted probability that the next month is a stress month. Dots mark months for which stress is realized in the subsequent month under the definition in equation ( 8 ).

Experimental results

Research questions

• RQ1 Can an equity-only, real-time cross-sectional fragility framework produce a calibrated one-month-ahead probability of market stress?
• RQ2 How does a sparse lasso-logit MSPI compare to nonlinear learners in discrimination and probability accuracy under a real-time expanding-window design?
• RQ3 Is MSPI's probability output stable and economically interpretable for monitoring and econometric use?
• RQ4 Do MSPI levels and innovations relate to subsequent realized market outcomes like volatility?

Key findings

• MSPI achieves out-of-sample AUC of 0.800 and PR-AUC of 0.538, outperforming the benchmark (AUC 0.752, PR-AUC 0.444).
• Nonlinear learners (gradient boosting, random forest) do not consistently dominate MSPI in discrimination or calibrated probability accuracy.
• MSPI records the lowest Brier score (0.106) and log loss (0.352) among evaluated models, indicating strong probability accuracy.
• MSPI probability path is smoother and more interpretable for monitoring than jagged nonlinear models.
• MSPI’s mean predicted probability (~0.180) is close to realized stress rate (~0.159), reflecting good calibration.
• MSPI signals rise ahead of major stress episodes (e.g., 2008-09 crisis, 2020 COVID) and cluster over time.

Figure 2: Out-of-sample stress probability - model horse race (calibrated). MSPI (black) is the L1-logit probability of next-month stress. Random forest (blue) and gradient boosting (orange) are estimated under the same expanding-window design and Platt-calibrated in real time within each training w